Question and answer model training method and device

Abstract

The invention discloses a question and answer model training method and device, and relates to the technical field of machine learning. A specific implementation mode of the method comprises the steps of obtaining question data and standard reply data generated in the online question answering process of a question answering model from training data, inputting the question data into the question answering model to obtain corresponding reply data, performing evaluation in combination with the standard reply data and the corresponding reply data to obtain an evaluation result, and outputting the evaluation result. And performing parameter adjustment on the question and answer model based on the evaluation result to obtain an optimized question and answer model. According to the embodiment, automatic optimization of the question and answer model based on the online question and answer data is realized, and the accuracy of data generated by the question and answer model is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a training method and device for a question-answering model. Background Art

[0002] In recent years, the rapid development of large language models (LLMs) has driven their widespread application in multiple fields, particularly in user-oriented chat scenarios and RAG (Retrieval-Augmented Generation) question-answering scenarios. Currently, the application of large models in customer service has become a key technology focus, with companies exploring how to use them to improve customer service efficiency and quality. However, in the actual application of large models, effectively improving reasoning efficiency and accuracy while reducing costs remains a key challenge facing the industry. Currently, model optimization mostly relies on manual prompt engineering and model fine-tuning, making it difficult to adapt to rapidly changing business needs and user scenarios.

[0003] Existing solutions often treat model training and online inference as independent links, resulting in deviations between model training data and actual inference data, affecting model effectiveness. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a method and device for training a question-answering model, which can perform offline training and optimization of the question-answering model based on online question-answering data, and directly use the optimized question-answering model for online question-answering reasoning, so that model training and online reasoning are closely linked, thereby improving the accuracy of the reasoning results and reasoning effect of the question-answering model.

[0005] To achieve the above objectives, according to one aspect of an embodiment of the present invention, a method for training a question-answering model is provided, comprising:

[0006] Acquiring training data, and extracting question-answer pairs consisting of question data and standard answer data from the training data, wherein the training data includes question-answer pair data generated during an online question-answering process of the question-answering model and first training data obtained by processing the adopted results;

[0007] Inputting the question data in each question-answer pair into the question-answer model so that the question-answer model generates answer data corresponding to the question data in each question-answer pair;

[0008] Performing an evaluation based on the standard answer data in each question-answer pair and the answer data corresponding to the question data in each question-answer pair to obtain an evaluation result;

[0009] Based on the evaluation results, the parameters of the question-answering model are adjusted to obtain an optimized question-answering model, so as to perform online question-answering based on the optimized question-answering model.

[0010] Optionally, the evaluation result includes an evaluation result generated based on at least one target prompt word corresponding to at least one node to be optimized in the question-answering model; the target prompt word is generated by the question-answering model based on the original prompt word; and the parameter adjustment of the question-answering model based on the evaluation result includes: determining the improved prompt word corresponding to at least one node to be optimized in the question-answering model based on the evaluation result; and adjusting the parameters of the question-answering model based on the improved prompt word.

[0011] Optionally, determining the improved prompt words corresponding to at least one node to be optimized in the question-answering model based on the evaluation result includes: based on the evaluation result, adding preset prompt word example data to the target prompt words to generate improved prompt words corresponding to at least one node to be optimized in the question-answering model.

[0012] Optionally, determining the improved prompt words corresponding to at least one node to be optimized in the question-answering model based on the evaluation results includes: based on the evaluation results, driving the target prompt words through multiple rounds of optimization through a large model, and using a global optimization strategy to find the global optimal solution to generate improved prompt words corresponding to at least one node to be optimized in the question-answering model.

[0013] Optionally, determining the improved prompt words corresponding to at least one node to be optimized in the question-answering model based on the evaluation result includes: based on the evaluation result, performing multiple rounds of optimization on the target prompt words by adding preset prompt word example data to the target prompt words, and using a global optimization strategy to find the global optimal solution to generate improved prompt words corresponding to at least one node to be optimized in the question-answering model.

[0014] Optionally, the evaluation result includes an evaluation result generated based on hyperparameters corresponding to at least one node to be optimized in the question-answering model; the parameter adjustment of the question-answering model based on the evaluation result includes: determining target hyperparameters corresponding to at least one node to be optimized in the question-answering model based on the evaluation result; and adjusting the parameters of the question-answering model based on the target hyperparameters.

[0015] Optionally, determining the target hyperparameters corresponding to at least one node to be optimized in the question-answering model based on the evaluation results includes: based on the evaluation results, determining the target hyperparameters corresponding to at least one node to be optimized in the question-answering model by traversing all parameters within the parameter search range.

[0016] Optionally, the first training data is obtained by processing the question and answer pair data and adoption results generated during the online question and answer process of the question and answer model in the following manner: sampling the question and answer pair data whose adoption results are adopted during the online question and answer process of the question and answer model to generate first question and answer pair data; sampling the question and answer pair data whose adoption results are not adopted during the online question and answer process of the question and answer model to generate recommended answer data corresponding to the question data in the question and answer pair data, and using the question and answer pair data consisting of the question data and the corresponding recommended answer data as the second question and answer pair data; generating the first training data based on the first question and answer pair data and the second question and answer pair data.

[0017] Optionally, the training data further includes second training data generated based on question-answer pair data consisting of manually annotated standard question data and standard answer data.

[0018] According to another aspect of an embodiment of the present invention, a training apparatus for a question-answering model is provided, comprising:

[0019] an acquisition module for acquiring training data and extracting question-answer pairs consisting of question data and standard answer data from the training data, wherein the training data includes question-answer pair data generated during online question-answering by the question-answering model and first training data obtained by processing the adopted results;

[0020] a generation module, configured to input the question data in each question-answer pair into the question-answer model, so that the question-answer model generates answer data corresponding to the question data in each question-answer pair;

[0021] An evaluation module, configured to evaluate the standard answer data in each question-answer pair and the answer data corresponding to the question data in each question-answer pair to obtain an evaluation result;

[0022] An optimization module is used to adjust the parameters of the question-answering model based on the evaluation results to obtain an optimized question-answering model, so as to perform online question-answering based on the optimized question-answering model.

[0023] Optionally, the evaluation result includes an evaluation result generated based on at least one target prompt word corresponding to at least one node to be optimized in the question-answering model; the target prompt word is generated by the question-answering model based on the original prompt word; the optimization module is also used to: determine the improved prompt word corresponding to at least one node to be optimized in the question-answering model based on the evaluation result; and adjust the parameters of the question-answering model based on the improved prompt word.

[0024] Optionally, the optimization module is further configured to: based on the evaluation result, add preset prompt word example data to the target prompt word to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

[0025] Optionally, the optimization module is also used to: based on the evaluation results, drive the target prompt words through multiple rounds of optimization through a large model, and use a global optimization strategy to find the global optimal solution to generate improved prompt words corresponding to no less than one node to be optimized in the question-answering model.

[0026] Optionally, the optimization module is further used to: based on the evaluation results, perform multiple rounds of optimization on the target prompt word by adding preset prompt word example data to the target prompt word, and use a global optimization strategy to find the global optimal solution to generate an improved prompt word corresponding to no less than one node to be optimized in the question-answering model.

[0027] Optionally, the evaluation result includes an evaluation result generated based on hyperparameters corresponding to at least one node to be optimized in the question-answering model; the optimization module is further used to: determine target hyperparameters corresponding to at least one node to be optimized in the question-answering model based on the evaluation result; and adjust parameters of the question-answering model based on the target hyperparameters.

[0028] Optionally, the optimization module is further used to: based on the evaluation result, determine the target hyperparameters corresponding to at least one node to be optimized in the question-answering model by traversing all parameters within the parameter search range.

[0029] Optionally, the first training data is obtained by processing the question and answer pair data and adoption results generated during the online question and answer process of the question and answer model in the following manner: sampling the question and answer pair data whose adoption results are adopted during the online question and answer process of the question and answer model to generate first question and answer pair data; sampling the question and answer pair data whose adoption results are not adopted during the online question and answer process of the question and answer model to generate recommended answer data corresponding to the question data in the question and answer pair data, and using the question and answer pair data consisting of the question data and the corresponding recommended answer data as the second question and answer pair data; generating the first training data based on the first question and answer pair data and the second question and answer pair data.

[0030] Optionally, the training data further includes second training data generated based on question-answer pair data consisting of manually annotated standard question data and standard answer data.

[0031] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the question-answering model provided by an embodiment of the present invention.

[0032] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the training method of the question-answering model provided by an embodiment of the present invention is implemented.

[0033] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements the training method of the question-answering model provided by an embodiment of the present invention.

[0034] One embodiment of the above invention has the following advantages or beneficial effects: by constructing training data based on the question-answer pair data obtained in the online question-answering process of the question-answering model, and training and optimizing the question-answering model based on the training data, the response data generated by the question-answering model in the training process can be evaluated, and the parameters of the question-answering model can be adjusted according to the evaluation results, thereby ensuring the accuracy of the optimized question-answering model and improving the training optimization efficiency of the model. The present invention optimizes the question-answering model offline based on the online question-answering data, and directly uses the optimized question-answering model for online question-answering reasoning, so that the model training and online reasoning are closely linked, thereby improving the accuracy of the reasoning results and the reasoning effect of the question-answering model. At the same time, the training data is the data determined based on the adoption results of the question-answering pair data in the online question-answering process, which can be directly used for the training optimization of the question-answering model, reducing the amount of manually labeled training data and improving the model training efficiency.

[0035] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0037] Figure 1 2 is a schematic diagram of the main steps of the training method of the question-answering model according to an embodiment of the present invention;

[0038] Figure 2 Schematic diagram of the application and training optimization process of the question-answering model provided by an embodiment of the present invention;

[0039] Figure 3 2 is a schematic diagram of main modules of a training device for a question-answering model according to an embodiment of the present invention;

[0040] Figure 4 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0041] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0043] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions disclosed in this invention all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken with respect to user personal information to prevent unauthorized access to user personal information data and to safeguard the security of user personal information, network security, and national security.

[0044] With the increasing popularity of large model applications, performance requirements for large models are becoming increasingly stringent. To improve the accuracy of data generated by large models, there is an urgent need for a technical solution that can effectively connect model training and online inference, and achieve automatic optimization and continuous improvement of prompts. This can enhance the performance and efficiency of large models in practical applications and better meet the growing business needs of enterprises.

[0045] Current large-model optimization methods have certain limitations and cannot fully utilize the capabilities of large models. Regarding prompt word optimization, existing inference optimization tools are divided into unsupervised and supervised methods, distinguished by whether labeled data is required. Unsupervised optimization typically leverages the generalization capabilities of large models, using the original prompt words as input to generate optimized prompt words with the help of the large model. The optimization effect is usually very limited, and the optimization results are difficult to evaluate. Supervised optimization requires users to prepare large amounts of training data for training and evaluation, requiring a significant amount of manpower for data preparation and labeling. Furthermore, these two types of tools are only responsible for offline optimization and lack the ability to deploy online services. This leads to a disconnect between offline optimization and online inference, often using one framework for offline optimization and another for online inference.

[0046] In terms of model weight optimization, model training and inference prompt word design are usually the responsibility of two different groups of people. The cost of model training is high, but the inference prompt words change frequently, resulting in a situation where the data used for large model parameter training is significantly different from the data used for online inference. This makes it impossible to achieve unification, making online inference overly dependent on the generalization capabilities of the large model.

[0047] To address the above technical problems existing in the prior art, the present invention provides a method for training a question-answering model. This method can optimize the question-answering model offline based on online question-answering data, and directly use the optimized question-answering model for online question-answering reasoning. This method closely links model training and online reasoning, avoids the disconnection between offline model optimization and online reasoning, and improves the accuracy and effectiveness of the question-answering model's reasoning results. Furthermore, the training data, which is the data determined based on the adopted results of the question-answering pairs during the online question-answering process, can be directly used for training and optimizing the question-answering model, reducing the amount of manually labeled training data and improving model training efficiency.

[0048] The specific implementation process of the question-answering model training method according to an embodiment of the present invention is described below with reference to the accompanying drawings.

[0049] Figure 1 FIG. 1 is a schematic diagram of the main steps of the training method of the question-answering model according to an embodiment of the present invention. Figure 1 As shown, the training method of the question-answering model of the present invention mainly includes steps S101 to S104.

[0050] Step S101, obtain training data, and extract question-answer pairs consisting of question data and standard answer data from the training data. The training data includes question-answer pair data generated during the online question-answering process of the question-answering model and the first training data obtained by processing the adopted results.

[0051] Before optimizing and training the question-answering model, you first need to obtain training data for the model. This training data primarily consists of question-answer pairs—specifically, question data and corresponding standard responses. This data is extracted from the training data and used in the subsequent model training process. The training data consists of question-answer pairs generated during the online question-answering process when the question-answering model is used, processed based on the user's final response.

[0052] The question-answering model analyzes and identifies the question data input by the user, generates corresponding answer data, and forms a question-answer pair with the input question data and the generated answer data. The question-answer pair data is then further processed based on whether the generated answer data is accepted by the user to generate first training data and store it in the training data. This data can be selected from any time period as needed, for example, from the past day's question-answer pair data, the past hour's question-answer pair data, and so on.

[0053] Step S102: input the question data in each question-answer pair into the question-answer model so that the question-answer model generates answer data corresponding to the question data in each question-answer pair.

[0054] When training a question-answering model, you first need to input the extracted question data into the constructed question-answering model to obtain the corresponding response data generated by the question-answering model after analyzing and identifying the question data. This response data is the initial response data output by the question-answering model, that is, the response data before the next optimization operation. This response data can be used as reference data for subsequent adjustment and optimization of the question-answering model parameters.

[0055] Step S103: evaluating the standard answer data in each question-answer pair and the answer data corresponding to the question data in each question-answer pair to obtain an evaluation result.

[0056] After obtaining the response data, the current question-answering model needs to be evaluated. This can be achieved by evaluating the response data generated by the current question-answering model based on the question data. During the evaluation, the response data needs to be evaluated according to the preset evaluation rules and the standard response data corresponding to the question data as a reference to obtain the corresponding evaluation results.

[0057] In one embodiment, the operation of performing evaluation based on evaluation rules mentioned above can be implemented in the following two ways: one is automatic evaluation, in which the big model evaluates whether the response data is correct based on set standards (for example, a prompt word specified by the user); the other is based on staff feedback, in which the question and the response data of the big model are pushed to the evaluation platform, and the staff annotates the data to determine whether the model's response data is correct.

[0058] In one embodiment, the evaluation process includes, for example, performing evaluation by calculating the similarity between the standard answer data and the corresponding answer data, and evaluating the accuracy of the current answer data based on the calculated similarity value.

[0059] Step S104: Adjust the parameters of the question-answering model based on the evaluation results to obtain an optimized question-answering model, and perform online question-answering based on the optimized question-answering model.

[0060] After obtaining the evaluation results, the parameters of the question-answering model need to be adjusted based on the evaluation results to achieve model optimization and obtain an optimized question-answering model. The evaluation results can be used to measure the accuracy of the question-answering model's output. The resulting optimized model can be used in subsequent online question-answering processes. Specifically, after the optimized question-answering model is launched online, it can be used to generate response data.

[0061] According to the training method of the question-answering model provided by the embodiment of the present invention, training data is constructed based on the question-answering pair data obtained in the online question-answering process of the question-answering model, and the question-answering model is trained and optimized based on the training data. Then, the response data generated by the question-answering model in the training process can be evaluated, and the parameters of the question-answering model can be adjusted according to the evaluation results to ensure the accuracy of the optimized question-answering model and improve the training optimization efficiency of the model. By performing offline training and optimization on the question-answering model based on the online question-answering data, and directly using the optimized question-answering model for online question-answering reasoning, the model training and online reasoning are closely linked, which improves the accuracy of the reasoning results and the reasoning effect of the question-answering model. At the same time, the training data is the data determined based on the adoption results of the question-answering pair data in the online question-answering process, which can be directly used for the training optimization of the question-answering model, reducing the amount of manually labeled training data and improving the efficiency of model training.

[0062] In one embodiment, the evaluation result includes an evaluation result generated based on at least one target prompt word corresponding to at least one node to be optimized in the question-answering model; the target prompt word is generated by the question-answering model based on the original prompt word; the question-answering model is parameter adjusted based on the evaluation result, including: determining an improved prompt word corresponding to at least one node to be optimized in the question-answering model based on the evaluation result; and adjusting the question-answering model parameters based on the improved prompt word.

[0063] When evaluating the question-answering model, multiple target prompt words of multiple nodes to be optimized of the question-answering model can be evaluated, that is, the accuracy of the prompt words of the question-answering model can be evaluated to obtain corresponding evaluation results. Among them, the target prompt words are generated based on the original prompt words of the node to be optimized of the question-answering model. Specifically, the original prompt words are input into the large model, and multiple target prompt words are generated using the large model. For each node to be optimized, it is necessary to evaluate the accuracy of the response data generated based on each target prompt word, and determine the target prompt word with the highest accuracy as the optimized prompt word of the current node to be optimized. Use the optimized prompt word to replace the original prompt word of the current node to be optimized to complete the prompt word adjustment operation of the question-answering model.

[0064] In one embodiment, determining an improved prompt word corresponding to at least one node to be optimized in the question-answering model based on the evaluation results includes: based on the evaluation results, adding preset prompt word example data to the target prompt word to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

[0065] In one embodiment, an improved prompt word corresponding to at least one node to be optimized in the question-answering model is determined based on the evaluation results, including: based on the evaluation results, driving the target prompt word through multiple rounds of optimization by a large model, and using a global optimization strategy to find a global optimal solution to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

[0066] In one embodiment, an improved prompt word corresponding to at least one node to be optimized in the question-answering model is determined based on the evaluation results, including: based on the evaluation results, performing multiple rounds of optimization on the target prompt word by adding preset prompt word example data to the target prompt word, and using a global optimization strategy to find a global optimal solution to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

[0067] When optimizing prompt words, the evaluation results are obtained by assessing the accuracy of the large-scale model's response data generated based on the target prompt word. Specifically, the accuracy values generated based on the responses corresponding to different prompt words are included. Improved prompt words can be determined by adding example data. Specifically, by enriching the prompt words with few-shot examples / demonstration data (a learning method that allows a model to effectively complete a classification or task even when only a small number of examples per category are available during training), the large-scale language model outputs response data that conforms to the style and specifications of the example / demonstration data. Few-shot examples represent question-answer pairs. Demonstration data is determined based on the intermediate steps in the process from question to answer generation. The steps involved in the demonstration data are determined by a computational graph involving multiple inference steps. The demonstration data is automatically generated by the system based on the training data. Using a pre-set teacher model, the system completes the entire inference process based on the training data input. The intermediate results of all steps are collected to construct the demonstration data until the required number of demonstration data is generated. Among them, the calculation graph is equivalent to a logical link. The various links established for the user's questions constitute the calculation graph. For example, for the user's questions, the intention recognition link is set first, and then the data analysis link is set. If the user's question may be ambiguous, the information collection link is also required. The various links are connected to form the user's calculation graph.

[0068] When determining the improved prompt words, it can also be achieved through prompt word text optimization. Specifically, the evaluation results include the evaluation results of multiple target prompt words corresponding to each node to be optimized, wherein each node to be optimized has multiple target prompt words corresponding to it. For each node to be optimized, through multiple rounds of optimization, the optimal prompt word among the multiple target prompt words is determined (the accuracy of the response data generated based on the prompt word is the highest), the determined optimal prompt word is used as the improved prompt word, and the improved prompt word is used to replace the original prompt word of the node to be optimized to complete the optimization process. Among them, the target prompt word is the prompt word in the intermediate process during the training of the large model, and the determined improved prompt word is the optimal prompt word among the target prompt words. The specific process of multiple rounds of optimization is:

[0069] 1. Traverse all nodes to be optimized in the large model, take the original prompt word as input, and use the large model to generate multiple new target prompt words as initial input;

[0070] 2. Traverse according to the parameter "depth", where depth represents the number of rounds:

[0071] (1) Traverse all nodes to be optimized

[0072] a. Traverse and evaluate all candidate sets of prompt words to be evaluated (a set consisting of multiple generated target prompt words);

[0073] b. Select and set the best prompt word for the current node.

[0074] (2) Traverse all nodes to be optimized

[0075] a. Generate new candidate sets using the large model based on the previous candidate sets and corresponding evaluation results;

[0076] b. Add the new candidate set to the candidate set to be evaluated;

[0077] c. Select and set the best global prompt word.

[0078] When determining improved prompt words, you can also combine adding sample data with prompt word text optimization. That is, based on the prompt word text optimization process described above, add sample data to the obtained global optimal prompt word to obtain the final improved prompt word, and then use this final improved prompt word to replace the original prompt word of the node to be optimized.

[0079] The training method for the question-answering model provided in an embodiment of the present invention can optimize the prompt words of the question-answering model based on the training data, thereby greatly improving the accuracy of the response data generated by the question-answering model.

[0080] In one embodiment, the evaluation result includes an evaluation result generated based on hyperparameters corresponding to at least one node to be optimized in the question-answering model; adjusting the parameters of the question-answering model based on the evaluation result includes: determining target hyperparameters corresponding to at least one node to be optimized in the question-answering model based on the evaluation result; and adjusting the parameters of the question-answering model based on the target hyperparameters.

[0081] Optimizing the question-answering model may also include optimizing hyperparameters. The corresponding evaluation results include the evaluation results generated for the hyperparameters of at least one node to be optimized. Specifically, based on the evaluation results of the hyperparameters of each node to be optimized, the optimal hyperparameter result is determined as the target hyperparameter, and the target hyperparameter is used to replace and adjust the parameters in the question-answering model.

[0082] In one embodiment, the target hyperparameters corresponding to at least one node to be optimized in the question-answering model are determined based on the evaluation results, including: based on the evaluation results, by traversing all parameters within the parameter search range, determining the target hyperparameters corresponding to at least one node to be optimized in the question-answering model.

[0083] In one embodiment, determining the target hyperparameters corresponding to at least one node to be optimized in the question-answering model based on the evaluation results may also include: based on the evaluation results, determining the target hyperparameters corresponding to at least one node to be optimized in the question-answering model by randomly sampling parameters within the parameter search range.

[0084] In one embodiment, determining target hyperparameters corresponding to at least one node to be optimized in the question-answering model based on the evaluation results may also include: based on the evaluation results, using a global optimization strategy to find a global optimal solution, and selecting target hyperparameters corresponding to at least one node to be optimized in the question-answering model from parameters within the parameter search range.

[0085] The target hyperparameters can be determined by using different hyperparameter search algorithms within a specified range. A grid search algorithm can be used to determine the target hyperparameters. Specifically, all possible parameter combinations within the parameter search range are traversed to determine the optimal hyperparameters as the target hyperparameters. Alternatively, a random search algorithm can be used to determine the target hyperparameters. Specifically, parameters within the parameter search range are randomly sampled. When the number of samples reaches a certain value, the optimal hyperparameter is selected as the target hyperparameter. In this case, with a sufficient number of samples, there is a high probability of finding near-optimal parameters. Alternatively, a Bayesian optimization algorithm (a global optimization strategy algorithm) can be used to determine the target hyperparameters. Specifically, previous search information is used to guide the selection of new search points, exploring the parameter space formed by the parameter search range. The target hyperparameters are then determined by learning the shape of the objective function to predict the location of the optimal parameters.

[0086] The training method of the question-answering model provided by the embodiment of the present invention can train and optimize the hyperparameters of the question-answering model based on the constructed training data, thereby further improving the accuracy of the data generated by the question-answering model.

[0087] In one embodiment, in addition to prompt word optimization and hyperparameter optimization, question-answering model training optimization can also be achieved through node optimization and model weight optimization. Specifically, during node optimization, the entire question-answering model reasoning task can be represented as a computational graph, with each subtask described as a node. Common subtasks include data processing, knowledge data recall, large-model reasoning, multimodal data analysis, risk control and security checks, and intent recognition. A complete computational graph is a directed graph composed of subtask nodes, and the node optimization unit is a unit that uses the large-model to automatically optimize the user-defined directed computational graph. Atomic operations used to update the computational graph can include adding, deleting, and moving nodes. When optimizing nodes in the question-answering model, the node optimization unit first uses the large-model to convert the computational graph source code into structured text information. This text information includes node functional descriptions and data transfer relationships between nodes. It then guides the large-model to analyze how to optimize the current process. Based on the analysis results, it then determines the atomic operations to be performed and updates the computational graph based on the determined atomic operations. The user can specify a "learning rate" to control the intensity of the update. The node optimization method mainly relies on the analysis and summary capabilities of the large model and does not require additional training data.

[0088] When optimizing model weights, you can use the training data to call the training framework to automatically fine-tune the model and obtain the optimized model weight parameters. Based on the optimized model weight parameters, you can adjust and optimize the question-answering model.

[0089] In one embodiment, the first training data is obtained by processing the question and answer pair data and the adoption results generated during the online question and answer process of the question and answer model in the following manner: sampling the question and answer pair data whose adoption results are adopted generated during the online question and answer process of the question and answer model to generate first question and answer pair data; generating the question and answer pair data whose adoption results are not adopted generated during the online question and answer process of the question and answer model, generating recommended answer data corresponding to the question data in the question and answer pair data, and using the question data and the corresponding recommended answer data as the second question and answer pair data; generating the first training data based on the first question and answer pair data and the second question and answer pair data.

[0090] The generation of the first training data is achieved through the online response data of the question-answering model. Specifically, for the question-answering data generated online, it is necessary to first obtain its adoption result, that is, whether it is adopted by the user. If the user adopts it, it means that the generated question-answering data is accurate data, and sampling is performed directly to obtain the first question-answering data. If the user does not adopt it, it means that the generated question-answering data is not accurate enough, and it needs to be processed to obtain the second question-answering data. The processing operation includes correcting the question-answering data that is not adopted, generating the recommended response data corresponding to it, and replacing the recommended response data as the response data in the question-answering data to obtain the second question-answering data. The first question-answering data and the second question-answering data are merged to obtain the first training data.

[0091] The training method of the question-answering model provided by the embodiment of the present invention can construct training data based on the data in the online question-answering process of the question-answering model, thereby improving the efficiency of constructing training data while ensuring the model training effect.

[0092] In one embodiment, the training data also includes second training data generated from question-answer pairs consisting of manually annotated standard question data and standard answer data. In addition to data collected based on online Q&A, the training data may also include second training data consisting of manually annotated standard question data and corresponding standard answer data. This second training data has extremely high accuracy, ensuring the accuracy of the results of the question-answering model optimization.

[0093] In one embodiment, the trained and optimized question-answering model is tested using test data. After the test passes, the model is launched. The test passes require that the accuracy of the response data generated by the question-answering model meets a preset threshold.

[0094] According to the training method of the question-answering model provided by the embodiment of the present invention, training data is constructed based on the question-answering pair data obtained in the online question-answering process of the question-answering model, and the question-answering model is trained and optimized based on the training data. The training process may specifically include: evaluating the response data generated by the question-answering model during the training process, and adjusting the parameters of the question-answering model according to the evaluation results, to ensure the accuracy of the optimized question-answering model and improve the training optimization efficiency of the model. The present invention optimizes the question-answering model offline based on the online question-answering data, and directly uses the optimized question-answering model for online question-answering reasoning, so that the model training and online reasoning are closely linked, thereby improving the accuracy of the reasoning results and the reasoning effect of the question-answering model. At the same time, the training data is the data determined based on the adoption results of the question-answering pair data in the online question-answering process, which can be directly used for the training optimization of the question-answering model, avoiding the difficulty of manually annotating the training data, reducing the amount of training data that needs to be constructed, and improving the efficiency of determining the training data. In addition, the training optimization of the question-answering model includes a variety of optimization methods, including prompt word optimization, hyperparameter optimization, node optimization, and model weight optimization. These methods can ensure the optimization effect while achieving global optimization, greatly improving the optimization effect of the model and avoiding falling into the dilemma of local optimality.

[0095] Figure 2 The figure is a schematic diagram of the application and training optimization process of the question-answering model provided by an embodiment of the present invention. The online inference process of the question-answering model includes an online user inputting a question into the question-answering model and obtaining answer data output by the model. If the answer data is accepted by the user, the question-answer pair data consisting of the answer data and the question is sampled, and the sampled first training data is stored in the training set database. If the answer data is not accepted by the user, a customer service response is provided based on the question input by the user, that is, the customer service staff manually responds to obtain recommended answer data, and the question-answer pair data constructed from the question and recommended answer data is stored as second training data in the training set database. In addition, the data in the training set database may also include annotated data that has been pre-annotated on a labeling platform. The annotated data is data obtained by customer service staff manually annotating the question, specifically data obtained by the customer service staff through operations performed on the labeling platform. For the data in the training set database, training data and test data sets are constructed through sampling. The training data can then be used to train and optimize the question-answering model. When training and optimizing a question-answering model, questions from the training data are fed into the model to generate the model's output. These responses are then evaluated against the corresponding standard responses, optimizing the model based on the evaluation results. The optimized model is then evaluated using the test dataset. If the test passes, the optimized model is released online.

[0096] Figure 3Schematic diagram of the main modules of the training device 300 for the question-answering model provided by an embodiment of the present invention. Figure 3 As shown, the training device 300 of the question-answering model mainly includes an acquisition module 301, a generation module 302, an evaluation module 303 and an optimization module 304.

[0097] An acquisition module 301 is configured to acquire training data and extract question-answer pairs consisting of question data and standard answer data from the training data, wherein the training data includes question-answer pair data generated during online question-answering by the question-answering model and first training data obtained by processing the adopted results;

[0098] A generation module 302 is configured to input the question data in each question-answer pair into a question-answer model so that the question-answer model generates answer data corresponding to the question data in each question-answer pair;

[0099] An evaluation module 303 is configured to evaluate the standard answer data in each question-answer pair and the answer data corresponding to the question data in each question-answer pair to obtain an evaluation result;

[0100] The optimization module 304 is used to adjust the parameters of the question-answering model based on the evaluation results to obtain an optimized question-answering model, so as to perform online question-answering based on the optimized question-answering model.

[0101] According to the training device of the question-answering model provided by the embodiment of the present invention, training data is constructed based on the question-answering pair data obtained during the online question-answering process of the question-answering model, and the question-answering model is trained and optimized based on the training data. Then, the response data generated by the question-answering model during the training process can be evaluated, and the parameters of the question-answering model can be adjusted according to the evaluation results to ensure the accuracy of the optimized question-answering model and improve the training optimization efficiency of the model. By performing offline training optimization on the question-answering model based on the online question-answering data and directly using the optimized question-answering model for online question-answering reasoning, the model training and online reasoning are closely linked, thereby improving the accuracy of the reasoning results and the reasoning effect of the question-answering model. At the same time, the training data is the data determined based on the adoption results of the question-answering pair data during the online question-answering process, and can be directly used for the training optimization of the question-answering model, reducing the amount of manually labeled training data and improving the efficiency of model training.

[0102] In one embodiment, the evaluation result includes an evaluation result generated based on at least one target prompt word corresponding to at least one node to be optimized in the question-answering model; the target prompt word is generated by the question-answering model based on the original prompt word; the optimization module 304 is further used to: determine an improved prompt word corresponding to at least one node to be optimized in the question-answering model based on the evaluation result; and adjust parameters of the question-answering model based on the improved prompt word.

[0103] In one embodiment, the optimization module 304 is further configured to: based on the evaluation result, add preset prompt word example data to the target prompt word to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

[0104] In one embodiment, the optimization module 304 is further used to: based on the evaluation results, drive the target prompt word through multiple rounds of optimization through the large model, and use the global optimization strategy to find the global optimal solution to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

[0105] In one embodiment, the optimization module 304 is further used to: based on the evaluation results, perform multiple rounds of optimization on the target prompt word by adding preset prompt word example data to the target prompt word, and use a global optimization strategy to find the global optimal solution to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

[0106] In one embodiment, the evaluation result includes an evaluation result generated based on hyperparameters corresponding to at least one node to be optimized in the question-answering model; the optimization module 304 is also used to: determine target hyperparameters corresponding to at least one node to be optimized in the question-answering model based on the evaluation result; and adjust the parameters of the question-answering model based on the target hyperparameters.

[0107] In one embodiment, the optimization module 304 is further used to: determine the target hyperparameters corresponding to at least one node to be optimized in the question-answering model by traversing all parameters within the parameter search range based on the evaluation results.

[0108] In one embodiment, the first training data is obtained by processing the question and answer pair data and the adoption results generated during the online question and answer process of the question and answer model in the following manner: sampling the question and answer pair data whose adoption results are adopted generated during the online question and answer process of the question and answer model to generate first question and answer pair data; generating the question and answer pair data whose adoption results are not adopted generated during the online question and answer process of the question and answer model, generating recommended answer data corresponding to the question data in the question and answer pair data, and using the question data and the corresponding recommended answer data as the second question and answer pair data; generating the first training data based on the first question and answer pair data and the second question and answer pair data.

[0109] In one embodiment, the training data further includes second training data generated based on question-answer pair data consisting of manually annotated standard question data and standard answer data.

[0110] According to the training device of the question-answering model provided by the embodiment of the present invention, training data is constructed based on the question-answering pair data obtained in the online question-answering process of the question-answering model, and the question-answering model is trained and optimized based on the training data. The training process may specifically include: evaluating the response data generated by the question-answering model during the training process, and adjusting the parameters of the question-answering model according to the evaluation results, to ensure the accuracy of the optimized question-answering model and improve the training optimization efficiency of the model. The present invention optimizes the question-answering model offline based on the online question-answering data, and directly uses the optimized question-answering model for online question-answering reasoning, so that the model training and online reasoning are closely linked, thereby improving the accuracy of the reasoning results and the reasoning effect of the question-answering model. At the same time, the training data is the data determined based on the adoption results of the question-answering pair data in the online question-answering process, and can be directly used for the training optimization of the question-answering model, avoiding the difficulty of manually labeling the training data, reducing the amount of training data that needs to be constructed, and improving the efficiency of determining the training data. In addition, the training optimization of the question-answering model includes a variety of optimization methods, including prompt word optimization, hyperparameter optimization, node optimization, and model weight optimization. These methods can ensure the optimization effect while achieving global optimization, greatly improving the optimization effect of the model and avoiding falling into the dilemma of local optimality.

[0111] Figure 4 An exemplary system architecture 400 is shown to which the training method or apparatus for a question-answering model according to an embodiment of the present invention may be applied.

[0112] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0113] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Terminal devices 401, 402, and 403 can be installed with various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0114] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0115] Server 405 can be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by users using terminal devices 401, 402, and 403. The backend management server can analyze and perform training on received data, such as training requests, and provide feedback (for example, a trained question-answering model—for example only) to the terminal device.

[0116] It should be noted that the training method of the question-answering model provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the training device of the question-answering model is generally set in the server 405 .

[0117] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0118] Reference below Figure 5 , which shows a schematic structural diagram of a computer system 500 of a terminal device or server suitable for implementing an embodiment of the present invention. Figure 5 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0119] like Figure 5 As shown, computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of system 500 are also stored in RAM 503. CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0120] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 508 including devices such as a hard disk; and a communication section 509 including a network interface card such as a LAN card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. Removable media 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read from the removable media can be installed in the storage section 508 as needed.

[0121] In particular, according to embodiments disclosed herein, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed herein include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509 and / or installed from removable media 511. When executed by central processing unit (CPU) 501, the computer program performs the aforementioned functions defined in the system of the present invention.

[0122] It should be noted that the computer-readable medium described in the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] The units or modules involved in the embodiments of the present invention may be implemented in software or in hardware. The units or modules described may also be provided in a processor. For example, they may be described as: a processor including an acquisition module, a generation module, an evaluation module, and an optimization module. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves. For example, the acquisition module may also be described as a "module for acquiring training data and extracting question-answer pairs consisting of question data and standard answer data from the training data."

[0125] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiments, or may exist independently without being incorporated into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the device, the device includes:

[0126] Acquire training data and extract question-answer pairs consisting of question data and standard answer data from the training data, wherein the training data includes question-answer pair data generated during online question-answering of the question-answering model and first training data obtained by processing the adopted results;

[0127] Input the question data in each question-answer pair into the question-answering model so that the question-answering model generates answer data corresponding to the question data in each question-answer pair;

[0128] Performing an evaluation based on the standard answer data in each question-answer pair and the answer data corresponding to the question data in each question-answer pair to obtain an evaluation result;

[0129] Based on the evaluation results, the parameters of the question-answering model are adjusted to obtain an optimized question-answering model, and online question-answering is performed based on the optimized question-answering model.

[0130] According to the technical solution of an embodiment of the present invention, by constructing training data based on the question-answer pair data obtained in the online question-answering process of the question-answering model, and training and optimizing the question-answering model based on the training data, the response data generated by the question-answering model during the training process can be evaluated, and the parameters of the question-answering model can be adjusted according to the evaluation results to ensure the accuracy of the optimized question-answering model and improve the training optimization efficiency of the model. The present invention optimizes the question-answering model offline based on the online question-answering data, and directly uses the optimized question-answering model for online question-answering reasoning, so that the model training and online reasoning are closely linked, thereby improving the accuracy of the reasoning results and the reasoning effect of the question-answering model. At the same time, the training data is the data determined based on the adoption results of the question-answering pair data in the online question-answering process, which can be directly used for the training optimization of the question-answering model, reducing the amount of manually labeled training data and improving the model training efficiency.

[0131] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A training method for a question-answering model, characterized in that: include: Acquiring training data, and extracting question-answer pairs consisting of question data and standard answer data from the training data, wherein the training data includes question-answer pair data generated during an online question-answering process of the question-answering model and first training data obtained by processing the adopted results; Inputting the question data in each question-answer pair into the question-answer model so that the question-answer model generates answer data corresponding to the question data in each question-answer pair; Performing an evaluation based on the standard answer data in each question-answer pair and the answer data corresponding to the question data in each question-answer pair to obtain an evaluation result; Based on the evaluation results, the parameters of the question-answering model are adjusted to obtain an optimized question-answering model, so as to perform online question-answering based on the optimized question-answering model.

2. The method according to claim 1, characterized in that The evaluation result includes an evaluation result generated based on at least one target prompt word corresponding to at least one node to be optimized in the question-answering model; The target prompt word is generated by the question-answering model based on the original prompt word; The adjusting parameters of the question-answering model based on the evaluation result includes: Determining, based on the evaluation result, an improvement prompt word corresponding to at least one node to be optimized in the question-answering model; Parameters of the question-answering model are adjusted based on the improved prompt words.

3. The method according to claim 2, characterized in that The step of determining, based on the evaluation result, an improvement prompt word corresponding to at least one node to be optimized in the question-answering model includes: Based on the evaluation result, preset prompt word example data is added to the target prompt word to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

4. The method according to claim 2, characterized in that The step of determining, based on the evaluation result, an improvement prompt word corresponding to at least one node to be optimized in the question-answering model includes: Based on the evaluation results, the target prompt words are optimized for multiple rounds through a large model drive, and a global optimization strategy is used to find the global optimal solution to generate improved prompt words corresponding to at least one node to be optimized in the question-answering model.

5. The method according to claim 2, characterized in that The step of determining, based on the evaluation result, an improvement prompt word corresponding to at least one node to be optimized in the question-answering model includes: Based on the evaluation results, the target prompt word is optimized for multiple rounds by adding preset prompt word example data to the target prompt word, and a global optimization strategy is used to find the global optimal solution to generate an improved prompt word corresponding to at least one node to be optimized in the question-answering model.

6. The method according to claim 1, characterized in that The evaluation result includes an evaluation result generated based on the hyperparameters corresponding to at least one node to be optimized in the question-answering model; The adjusting parameters of the question-answering model based on the evaluation result includes: Determine, based on the evaluation result, a target hyperparameter corresponding to at least one node to be optimized in the question-answering model; Parameter adjustment is performed on the question-answering model based on the target hyperparameters.

7. The method according to claim 6, characterized in that Determining target hyperparameters corresponding to at least one node to be optimized in the question-answering model based on the evaluation result includes: Based on the evaluation result, by traversing all parameters within the parameter search range, target hyperparameters corresponding to at least one node to be optimized in the question-answering model are determined.

8. The method according to claim 1, characterized in that The first training data is obtained by processing the question-answer pair data and the adopted results generated during the online question-answering process of the question-answering model in the following manner: The adopted result generated in the online question-answering process of the question-answering model is sampled as the adopted question-answer pair data to generate the first question-answer pair data; For question-answer pair data whose adopted result is not adopted during the online question-answering process of the question-answering model, generating recommended answer data corresponding to the question data in the question-answer pair data, and using the question data and the corresponding recommended answer data as the second question-answer pair data; The first training data is generated based on the first question-answer pair data and the second question-answer pair data.

9. The method according to any one of claims 1 to 8, characterized in that The training data also includes second training data generated based on question-answer pair data consisting of manually labeled standard question data and standard answer data.

10. A training device for a question-answering model, characterized in that: include: an acquisition module for acquiring training data and extracting question-answer pairs consisting of question data and standard answer data from the training data, wherein the training data includes question-answer pair data generated during online question-answering by the question-answering model and first training data obtained by processing the adopted results; a generation module, configured to input the question data in each question-answer pair into the question-answer model, so that the question-answer model generates answer data corresponding to the question data in each question-answer pair; An evaluation module, configured to evaluate the standard answer data in each question-answer pair and the answer data corresponding to the question data in each question-answer pair to obtain an evaluation result; An optimization module is used to adjust the parameters of the question-answering model based on the evaluation results to obtain an optimized question-answering model, so as to perform online question-answering based on the optimized question-answering model.

11. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

12. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images