Question answer obtaining method and device based on distillation model, equipment and medium

Abstract

The invention relates to the technical field of natural language processing and financial science and technology, and particularly discloses a question answer obtaining method and device based on a distillation model, computer equipment and a storage medium. The initial answer sequence of the to-be-answered question is obtained through the distillation model corresponding to the initial question and answer model, the distillation model can significantly reduce the parameter quantity and the calculation complexity, and then the obtaining efficiency of the initial answer sequence is improved. And secondly, the initial answer sequence is verified through the target question and answer model corresponding to the initial question and answer model, the target answer is obtained, errors in the initial answer sequence can be corrected, and the accuracy of the answer is ensured. Moreover, the target question and answer model only needs to verify the sequence, the whole sequence does not need to be generated, the task load is low, and the efficiency of obtaining the question answer is improved.

Description

Technical Field

[0001] The present application relates to the fields of natural language processing and financial technology, and in particular to a method, device, computer equipment and storage medium for obtaining answers to questions based on a distillation model. Background Art

[0002] With the rapid development of artificial intelligence technology, natural language processing has gradually been applied to various fields. In the financial field, the use of large language models to predict and generate questions raised by users, such as financial consulting and financial policy consulting, can effectively improve the efficiency of financial business processing. However, since the reasoning of large language models uses autoregressive generation, in the process of reasoning the answer to the question, each time a word token is generated, the model needs to access all parameters and transfer the parameters from the memory to the computing unit for calculation. The process is cumbersome, resulting in low efficiency in obtaining answers to questions. Therefore, how to improve the efficiency of obtaining answers to questions has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides a method, apparatus, computer device and storage medium for obtaining answers to questions based on a distillation model to improve the efficiency of obtaining answers to questions.

[0004] In a first aspect, the present application provides a method for obtaining a question answer based on a distillation model, the method comprising:

[0005] Obtaining questions to be answered, and determining an initial question-answering model based on the questions to be answered;

[0006] Acquire a target question-answering model and a distillation model corresponding to the initial question-answering model, and process the question sequence of the questions to be answered based on the distillation model to obtain an initial answer sequence corresponding to the questions to be answered;

[0007] Concatenating the question sequence with the initial answer sequence to obtain a target sequence;

[0008] Based on the target question-answering model and the target sequence, the initial answer sequence is verified to obtain the target answer to the question to be answered.

[0009] In a second aspect, the present application further provides a device for obtaining a question answer based on a distillation model, the device comprising:

[0010] An initial question-answering model determination module is used to obtain questions to be answered and determine an initial question-answering model based on the questions to be answered;

[0011] An initial answer sequence acquisition module is used to obtain a target question-answering model and a distillation model corresponding to the initial question-answering model, and based on the distillation model, process the question sequence of the questions to be answered to obtain an initial answer sequence corresponding to the questions to be answered;

[0012] A target sequence acquisition module, used for concatenating the question sequence with the initial answer sequence to obtain a target sequence;

[0013] The target answer acquisition module is used to verify the initial answer sequence based on the target question-answering model and the target sequence to obtain the target answer to the question to be answered.

[0014] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the method for obtaining answers to questions based on the distillation model as described above when executing the computer program.

[0015] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the method for obtaining answers to questions based on the distillation model as described above.

[0016] The present application discloses a method, device, computer equipment and storage medium for obtaining answers to questions based on a distillation model, obtaining questions to be answered, and determining an initial question-answering model based on the questions to be answered; obtaining a target question-answering model and a distillation model corresponding to the initial question-answering model, processing the question sequence of the questions to be answered based on the distillation model, and obtaining an initial answer sequence corresponding to the questions to be answered; splicing the question sequence with the initial answer sequence to obtain a target sequence; verifying the initial answer sequence based on the target question-answering model and the target sequence to obtain a target answer to the question to be answered. The present application obtains an initial answer sequence for the questions to be answered through a distillation model corresponding to the initial question-answering model, and the distillation model can significantly reduce the number of parameters and computational complexity, thereby improving the efficiency of obtaining the initial answer sequence. Secondly, the initial answer sequence is verified through the target question-answering model corresponding to the initial question-answering model to obtain the target answer, and the errors in the initial answer sequence can be corrected to ensure the accuracy of the answer. In addition, the target question-answering model only needs to verify the sequence, and does not need to generate the entire sequence, so the task volume is low, which improves the efficiency of obtaining the answer to the question. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic flow chart of a first embodiment of a method for obtaining a question answer based on a distillation model provided in an embodiment of the present application;

[0019] Figure 2 is a schematic flow chart of a second embodiment of a method for obtaining a question answer based on a distillation model provided in an embodiment of the present application;

[0020] Figure 3 is a schematic flow chart of a third embodiment of a method for obtaining a question answer based on a distillation model provided in an embodiment of the present application;

[0021] Figure 4 A schematic block diagram of a device for obtaining a question answer based on a distillation model provided in an embodiment of the present application;

[0022] Figure 5 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] The embodiments of the present application provide a method, device, computer equipment and storage medium for obtaining answers to questions based on a distillation model. Among them, the method for obtaining answers to questions based on a distillation model can be applied to a server. The initial answer sequence of the question to be answered is obtained through the distillation model corresponding to the initial question and answer model. The distillation model can significantly reduce the number of parameters and computational complexity, thereby improving the efficiency of obtaining the initial answer sequence. Secondly, the initial answer sequence is verified through the target question and answer model corresponding to the initial question and answer model to obtain the target answer, and the errors in the initial answer sequence can be corrected to ensure the accuracy of the answer. In addition, the target question and answer model only needs to verify the sequence, and there is no need to generate the entire sequence. The task volume is low, which improves the efficiency of obtaining the answer to the question. Among them, the server can be an independent server or a server cluster.

[0028] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] See also Figure 1 , Figure 1 This is a schematic flow chart of a method for obtaining answers to questions based on a distillation model provided in an embodiment of the present application. The method for obtaining answers to questions based on a distillation model can be applied to a server, and is used to obtain an initial answer sequence for questions to be answered through a distillation model corresponding to the initial question-answering model. The distillation model can significantly reduce the number of parameters and computational complexity, thereby improving the efficiency of obtaining the initial answer sequence. Secondly, the initial answer sequence is verified through the target question-answering model corresponding to the initial question-answering model to obtain the target answer, and errors in the initial answer sequence can be corrected to ensure the accuracy of the answer. In addition, the target question-answering model only needs to verify the sequence, and there is no need to generate the entire sequence. The task volume is low, which improves the efficiency of obtaining answers to questions.

[0030] like Figure 1 As shown, the method for obtaining the answer to the question based on the distillation model specifically includes steps S101 to S104.

[0031] S101, obtaining a question to be answered, and determining an initial question-answering model based on the question to be answered;

[0032] In one embodiment, user input is received and preprocessed to obtain questions to be answered, wherein the preprocessing may include steps such as cleaning data, removing irrelevant characters, word segmentation, and standardization.

[0033] In one embodiment, keywords in the question to be answered are identified, and the corresponding Q&A model is used as the initial Q&A model.

[0034] Further, determining the initial Q&A model based on the question to be answered includes: parsing the keywords of the question to be answered to obtain the keywords corresponding to the question to be answered; based on the keywords, searching in a preset model database to obtain the Q&A model corresponding to the keywords as the initial Q&A model.

[0035] In one embodiment, the Q&A models obtained through pre-training are stored in the model database. Among them, the Q&A models include general models applicable to a wide range of topics and specific models for specific fields (such as the financial field). Each Q&A model corresponds to a set of keywords, and these keywords represent the expertise or capabilities of the model.

[0036] In one embodiment, natural language processing technology is used to parse the question to be answered by the user to extract the key information in the question. Specifically, the question text is tokenized to break the sentence into individual words or phrases. Common stop words (such as "of", "and", "is", etc.) are removed, as these words usually do not carry important information. The parts of speech of the tokenized words are tagged to identify nouns, verbs, etc. Based on the content and context of the question, the keywords reflecting the core content of the question are extracted.

[0037] In one embodiment, the extracted keywords are used to query in a preset model database to find the Q&A model that best matches the keywords. Specifically, matching algorithms (such as cosine similarity, Jaccard similarity, etc.) can be applied to evaluate the similarity between the keywords and the model annotations.

[0038] In one embodiment, according to the matching degree and other performance metrics (such as response time, user feedback, etc.), the retrieved models are sorted and filtered, and the most matching model is selected as the initial Q&A model.

[0039] In one embodiment, the question type and field can be determined according to the keywords. For example, when the keyword "insurance" is detected, the question to be answered is classified into the financial field. Therefore, the Q&A model obtained according to the keywords can correspond to the field of the question, improving the accuracy of the question answer.

[0040] S102. Obtain the target Q&A model and the distillation model corresponding to the initial Q&A model, and based on the distillation model, process the question sequence of the question to be answered to obtain the initial answer sequence corresponding to the question to be answered;

[0041] In one embodiment, the target question-answering model is obtained by adjusting the initial question-answering model. Specifically, a training data set is obtained, and the target question-answering model is adjusted according to the training data set. The training data set can be a data set uploaded by the user according to actual needs, or a data set extracted from historical data according to keywords of the question to be answered.

[0042] In one embodiment, a data set is extracted from historical data according to the keywords of the question to be answered. For example, if the question to be answered is "Which is better, insurance A or insurance B?", the keyword is insurance, all insurance-related professional data is obtained from the historical data, a training data set is generated, and the initial question-answering model is adjusted to obtain a target question-answering model, so that the target question-answering model can generate accurate answers.

[0043] In one embodiment, the distilled model is a model into which the knowledge of the target question answering model is distilled, retaining the key knowledge of the target question answering model but with fewer layers and dimensions than the target question answering model.

[0044] In one embodiment, the question sequence refers to a series of word-unit tokens obtained after preprocessing and word segmentation of the questions to be answered.

[0045] In a specific embodiment, the questions to be answered are preprocessed, including word segmentation, stop word removal, stem extraction, etc., in preparation for input into the distillation model. The preprocessed questions are converted into a format that the model can understand, usually a series of word-unit token sequences. The question sequence is input into the distillation model, and the distillation model uses an autoregressive method to generate an initial answer sequence.

[0046] S103, concatenating the question sequence with the initial answer sequence to obtain a target sequence;

[0047] In one embodiment, a splicing point (eg, the last token of the question sequence) is determined, and based on the splicing point, the question sequence and the initial answer sequence are directly connected to form a new sequence, namely, the target sequence.

[0048] For example, assuming that the question sequence is "token1, token2, token3", and the initial answer sequence is "token4, token5, token6", the target sequence obtained by splicing is "token1, token2, token3, token4, token5, token6".

[0049] In another embodiment, special markers or separators may be added between the question sequence and the initial answer sequence to help the model understand the boundaries of the sequences.

[0050] S104: Based on the target question-answering model and the target sequence, the initial answer sequence is verified to obtain a target answer to the question to be answered.

[0051] In one embodiment, the target sequence is input into the target question answering model, and the model generates predictions for each token, including the possible next token for each token.

[0052] In one embodiment, the prediction of the target question-answering model is compared with the token in the initial answer sequence. If the token predicted by the model is consistent with the token in the initial answer sequence, the token is retained. If not, the token in the initial answer sequence is modified or discarded based on the prediction of the target question-answering model.

[0053] In one embodiment, the retained and modified tokens are integrated to form a target answer sequence based on the prediction of the target question-answering model.

[0054] Exemplarily, for the target sequence "token1, token2, token3, token4, token5, token6", the target question answering model first predicts the next possible token of token3 based on token1, token2, and token3. If the possible token is consistent with token4, token4 is retained, and the next possible token of token4 is predicted based on the target question answering model. If the possible token is consistent with token5, token5 is retained, and the next possible token of token5 is predicted based on the target question answering model. If the possible token is inconsistent with token6, token6 is deleted, and the possible token is used as the new token6.

[0055] In one embodiment, the target answer sequence is language polished to ensure the fluency and readability of the answer, and the format of the answer, such as punctuation, capitalization, etc., is adjusted to meet language standards.

[0056] In one embodiment, the verified and corrected target answer sequence is converted into a text format and presented to the user through a user interface.

[0057] The above embodiment provides a method, device, computer equipment and storage medium for obtaining answers to questions based on a distillation model, obtaining questions to be answered, and determining an initial question-answering model based on the questions to be answered; obtaining a target question-answering model and a distillation model corresponding to the initial question-answering model, processing the question sequence of the questions to be answered based on the distillation model, and obtaining an initial answer sequence corresponding to the questions to be answered; splicing the question sequence with the initial answer sequence to obtain a target sequence; verifying the initial answer sequence based on the target question-answering model and the target sequence to obtain the target answer to the question to be answered. The present application obtains an initial answer sequence for the questions to be answered through a distillation model corresponding to the initial question-answering model, and the distillation model can significantly reduce the number of parameters and computational complexity, thereby improving the efficiency of obtaining the initial answer sequence. Secondly, the initial answer sequence is verified through the target question-answering model corresponding to the initial question-answering model to obtain the target answer, and the errors in the initial answer sequence can be corrected to ensure the accuracy of the answer. In addition, the target question-answering model only needs to verify the sequence, and does not need to generate the entire sequence, so the task volume is low, which improves the efficiency of obtaining the answer to the question.

[0058] See also Figure 2 , Figure 2 This is a schematic flow chart of a method for obtaining answers to questions based on a distillation model provided in an embodiment of the present application. The method for obtaining answers to questions based on a distillation model can be applied to a server to perform distillation processing on a target question-answering model to obtain a lightweight distillation model, significantly reducing the number of parameters and computational complexity, thereby improving the efficiency of obtaining an initial answer sequence.

[0059] like Figure 2 As shown, the method for obtaining the answer to the question based on the distillation model specifically includes steps S201 to S202.

[0060] S201, obtaining a training data set, adjusting the initial question-answering model based on the training data set, and obtaining a target question-answering model corresponding to the initial question-answering model;

[0061] S202: Distill the target question-answering model to obtain a distilled model corresponding to the initial question-answering model.

[0062] In one embodiment, a training data set suitable for a specific field (such as the financial field, the medical field, etc.) is collected. The training data set can be a data set uploaded by the user according to actual needs, or it can be a data set extracted from historical data according to the keywords of the question to be answered. For example, if the question to be answered is "Which is better, insurance A or insurance B?", the keyword is insurance, and all insurance-related professional data are obtained in the historical data to generate a training data set.

[0063] In one embodiment, training parameters such as learning rate, batch size, training cycle, etc. are set. The initial question-answering model is trained using the training data set so that the model learns specific knowledge and patterns in the data set and implements model fine-tuning. The model performance is evaluated on the validation set and the test set to ensure the effectiveness and generalization ability of the model.

[0064] In one embodiment, the target question-answering model is used as the teacher model, and the number of layers and dimensions of the teacher model are reduced to obtain a student model. The student model is trained using a training data set to implement model distillation and obtain a distillation model corresponding to the target question-answering model.

[0065] Furthermore, the distillation of the target question and answer model to obtain the distillation model corresponding to the initial question and answer model includes: processing the number of layers and dimensions of the target question and answer model to obtain a student model; obtaining a first prediction result of the target question and answer model and a second prediction result of the student model based on the training data set; obtaining a loss function of the student model based on the first prediction result, the second prediction result and the loss function of the initial question and answer model; and training the student model based on the training data set and the loss function to obtain a distillation model corresponding to the initial question and answer model.

[0066] In a specific embodiment, the number of layers of the target question-answering model is reduced to half (or the number of layers set according to actual needs) to reduce the complexity of the model.

[0067] In a specific embodiment, the dimension of each layer is also reduced to half (or the dimension set according to actual needs), further reducing the number of parameters of the model.

[0068] In one embodiment, parameters of the attention layer are extracted from every other layer of the teacher model, and after dimensionality reduction processing, they are used to initialize the corresponding layer of the student model.

[0069] In one embodiment, a training data set is input into a target question-answering model to generate a first prediction result, and the same training data set is input into a student model to generate a second prediction result.

[0070] In one embodiment, the loss function of the target question-answering model is obtained based on the probability distribution of the first prediction result and the probability distribution of the second prediction result, and the cosine distance between the first prediction result and the second prediction result is calculated. The loss function of the target question-answering model, the loss function of the initial question-answering model, and the cosine distance between the first prediction result and the second prediction result are substituted into the formula for obtaining the loss function of the student model to obtain the loss function of the student model.

[0071] In one embodiment, a suitable optimization algorithm (such as SGD, Adam, etc.) is selected to update the parameters of the student model, and the loss function of the student model is used to calculate the loss value of the student model. Through multiple iterative training, the parameters of the student model are continuously updated to reduce the value of the loss function. When the loss value meets the requirements, the student model at this time is saved as a distillation model.

[0072] Furthermore, the loss function of the student model is obtained based on the first prediction result, the second prediction result and the loss function of the initial question-answering model, including: obtaining a first loss function based on the probability distribution of the first prediction result and the probability distribution of the second prediction result, and using the loss function of the initial question-answering model as the second loss function; obtaining the cosine distance between the target question-answering model prediction result and the student model prediction result; obtaining the loss function of the student model based on a preset weight coefficient, the first loss function, the second loss function and the cosine distance.

[0073] In one embodiment, the loss function of the target question-answering model is obtained based on the probability distribution of the first prediction result and the probability distribution of the second prediction result. Specifically, the first loss function of the target question-answering model is obtained by the formula:

[0074]

[0075] Among them, L CE is the loss function of the target question answering model, t i is the probability distribution of the first prediction result, s i is the probability distribution of the second prediction result.

[0076] In one embodiment, the probability distribution can be expressed as:

[0077]

[0078] Among them, p i represents the probability of the i-th category, z i is the i-th element in the original output vector z, T is the temperature parameter, and N is the total number of categories.

[0079] In one embodiment, the first prediction result and the second prediction result may be in vector form. If the first prediction result and the second prediction result are not in vector form, they are converted into vector form. The cosine distance between the two prediction result vectors is calculated.

[0080] In one embodiment, the loss function of the student model is obtained based on the first prediction result and the second prediction result. Specifically, the loss function of the student model is obtained by the formula:

[0081] Loss=ω1LCE +ω2L mlm +ω3L cos

[0082] Among them, ω1, ω2, ω3 are the weights of each loss, L CE is the loss function of the target question answering model, L mlm is the loss function used in the initial question-answering model training, i.e., the second loss function, L cos is the cosine distance between the first prediction result of the target question answering model and the second prediction result of the student model.

[0083] In the above embodiment, the initial question-answering model is fine-tuned according to the training data set to obtain the target question-answering model, so that the target question-answering model is more closely matched to the domain of the question to be answered, thereby improving the accuracy of the model in predicting the answer. The target question-answering model is distilled to obtain a lightweight distillation model, which significantly reduces the number of parameters and computational complexity, thereby improving the efficiency of obtaining answers.

[0084] See also Figure 3 , Figure 3 This is a schematic flow chart of a method for obtaining answers to questions based on a distillation model provided in an embodiment of the present application. The method for obtaining answers to questions based on a distillation model can be applied to a server to verify a sequence through a target question-answering model, thereby reducing the amount of model tasks and improving the efficiency of obtaining answers to questions.

[0085] like Figure 3 As shown, the method for obtaining the answer to the question based on the distillation model specifically includes steps S301 to S303.

[0086] S301, predicting the next word-gram of each word-gram in the target sequence based on the target question-answering model to obtain a predicted word-gram;

[0087] S302, based on the predicted word-gram, verify each word-gram in the initial answer sequence to obtain a verification result;

[0088] S303: Based on the verification result and the predicted word, modify the initial answer sequence to obtain the target answer.

[0089] Furthermore, based on the verification result and the predicted word-gram, the initial answer sequence is modified to obtain the target answer, including: if the verification result is that the predicted word-gram is inconsistent with the word-gram in the initial answer sequence, then the word-gram in the initial answer sequence is replaced with the predicted word-gram to obtain the target answer.

[0090] In one embodiment, the target sequence is input into the target question answering model. For each word in the target sequence, the target question answering model predicts its next possible word token.

[0091] In one embodiment, the next word-gram predicted by the target question-answering model is compared with the corresponding word-gram in the initial answer sequence to determine whether the corresponding word-gram in the initial answer sequence is consistent with the next word-gram predicted by the model.

[0092] In one embodiment, if the word-gram in the initial answer sequence is consistent with the predicted word-gram, it is marked as correct. If it is inconsistent, it is marked as wrong or to be corrected.

[0093] In one embodiment, word-grams marked as correct in the verification result are retained in the initial answer sequence.

[0094] In one embodiment, for the word-gram marked as an error, correction is performed according to the prediction of the target question-answering model. Specifically, the error word-gram and all word-grams after the word-gram position are replaced with the corresponding word-gram predicted by the target question-answering model.

[0095] Exemplarily, for the target sequence "token1, token2, token3, token4, token5, token6", the target question answering model first predicts the next possible token of token3 based on token1, token2, and token3. If the possible token is consistent with token4, token4 is retained, and the next possible token of token4 is predicted based on the target question answering model. If the possible token is consistent with token5, token5 is retained, and the next possible token of token5 is predicted based on the target question answering model. If the possible token is inconsistent with token6, token6 is deleted, and the possible token is used as the new token6.

[0096] In one embodiment, the retained and modified word-grams are combined into a new answer sequence, namely the target answer.

[0097] In another embodiment, necessary post-processing is performed on the target answer, such as grammar checking, text polishing, etc., to ensure the readability and accuracy of the answer.

[0098] In the above embodiment, the sequence is verified by the target question-answering model without generating the entire sequence, which reduces the model workload and improves the efficiency of obtaining answers to questions.

[0099] See also Figure 4 , Figure 4The embodiment of the present application provides a schematic block diagram of a device for obtaining a question answer based on a distillation model, and the device for obtaining a question answer based on a distillation model is used to execute the aforementioned method for obtaining a question answer based on a distillation model. The device for obtaining a question answer based on a distillation model can be configured on a server.

[0100] like Figure 4 As shown, the question answer obtaining device 400 based on the distillation model includes:

[0101] An initial question-answering model determination module 401 is used to obtain a question to be answered and determine an initial question-answering model based on the question to be answered;

[0102] An initial answer sequence obtaining module 402 is used to obtain a target question answering model and a distillation model corresponding to the initial question answering model, and based on the distillation model, process the question sequence of the question to be answered to obtain an initial answer sequence corresponding to the question to be answered;

[0103] A target sequence acquisition module 403 is used to concatenate the question sequence with the initial answer sequence to obtain a target sequence;

[0104] The target answer obtaining module 404 is used to verify the initial answer sequence based on the target question-answering model and the target sequence to obtain the target answer to the question to be answered.

[0105] Furthermore, the initial answer sequence obtaining module 402 includes:

[0106] A target question-answering model acquisition submodule is used to obtain a training data set, adjust the initial question-answering model based on the training data set, and obtain a target question-answering model corresponding to the initial question-answering model;

[0107] The distillation model acquisition submodule is used to distill the target question-answering model to obtain a distillation model corresponding to the initial question-answering model.

[0108] Furthermore, the distillation model obtains a submodule, including:

[0109] A student model obtaining unit, used to process the number of layers and dimensions of the target question-answering model to obtain a student model;

[0110] A prediction result obtaining unit, used to obtain a first prediction result of the target question-answering model and a second prediction result of the student model based on the training data set;

[0111] A loss function obtaining unit, configured to obtain the loss function of the student model based on the first prediction result, the second prediction result and the loss function of the initial question-answering model;

[0112] A distillation model obtaining unit is used to train the student model based on the training data set and the loss function to obtain a distillation model corresponding to the initial question-answering model.

[0113] Furthermore, the loss function obtaining unit includes:

[0114] A loss function obtaining subunit, configured to obtain a first loss function based on the probability distribution of the first prediction result and the probability distribution of the second prediction result, and use the loss function of the initial question-answering model as a second loss function;

[0115] A cosine distance obtaining subunit, used to obtain the cosine distance between the prediction result of the target question answering model and the prediction result of the student model;

[0116] The loss function obtaining subunit is used to obtain the loss function of the student model based on a preset weight coefficient, the first loss function, the second loss function and the cosine distance.

[0117] Furthermore, the target answer obtaining module 404 includes:

[0118] A predicted word unit is used to predict the next word of each word in the target sequence based on the target question-answering model to obtain a predicted word;

[0119] A verification result obtaining unit, configured to verify each word in the initial answer sequence based on the predicted word to obtain a verification result;

[0120] The target answer obtaining unit is used to modify the initial answer sequence based on the verification result and the predicted word to obtain the target answer.

[0121] Furthermore, the target answer obtaining unit includes:

[0122] The answer correction subunit is used to replace the word in the initial answer sequence with the predicted word to obtain the target answer if the verification result is that the predicted word is inconsistent with the word in the initial answer sequence.

[0123] Furthermore, the initial question-answer model determination module 401 includes:

[0124] A keyword obtaining unit, used for performing keyword analysis on the question to be answered to obtain keywords corresponding to the question to be answered;

[0125] The model determination unit is used to search in a preset model database based on the keyword to obtain a question-answering model corresponding to the keyword as the initial question-answering model.

[0126] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0127] The above-mentioned device can be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.

[0128] See also Figure 5 , Figure 5 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0129] See also Figure 5 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0130] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any method for obtaining a question answer based on a distillation model.

[0131] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0132] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any method for obtaining the answer to the question based on the distillation model.

[0133] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0135] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0136] Obtaining questions to be answered, and determining an initial question-answering model based on the questions to be answered;

[0137] Obtaining a target question-answering model and a distillation model corresponding to the initial question-answering model, and processing the question sequence of the questions to be answered based on the distillation model to obtain an initial answer sequence corresponding to the questions to be answered;

[0138] Concatenating the question sequence with the initial answer sequence to obtain a target sequence;

[0139] Based on the target question-answering model and the target sequence, the initial answer sequence is verified to obtain the target answer to the question to be answered.

[0140] In one embodiment, when acquiring the target question-answering model and the distillation model corresponding to the initial question-answering model, the processor is used to implement:

[0141] Acquire a training data set, and adjust the initial question-answering model based on the training data set to obtain a target question-answering model corresponding to the initial question-answering model;

[0142] The target question-answering model is distilled to obtain a distilled model corresponding to the initial question-answering model.

[0143] In one embodiment, when the processor implements distillation of the target question-answering model to obtain the distilled model corresponding to the initial question-answering model, it is configured to implement:

[0144] Processing the number of layers and dimensions of the target question-answering model to obtain a student model;

[0145] Based on the training data set, obtaining a first prediction result of the target question-answering model and a second prediction result of the student model;

[0146] Based on the first prediction result, the second prediction result and the loss function of the initial question-answering model, obtaining the loss function of the student model;

[0147] Based on the training data set and the loss function, the student model is trained to obtain a distillation model corresponding to the initial question-answering model.

[0148] In one embodiment, when the processor implements the loss function of the student model based on the first prediction result, the second prediction result, and the loss function of the initial question-answering model to obtain the loss function of the student model, it is used to implement:

[0149] Based on the probability distribution of the first prediction result and the probability distribution of the second prediction result, a first loss function is obtained, and the loss function of the initial question-answering model is used as a second loss function;

[0150] Obtaining the cosine distance between the prediction result of the target question-answering model and the prediction result of the student model;

[0151] Based on the preset weight coefficient, the first loss function, the second loss function and the cosine distance, the loss function of the student model is obtained.

[0152] In one embodiment, when the processor verifies the initial answer sequence based on the target question-answering model and the target sequence to obtain the target answer to the question to be answered, it is used to implement:

[0153] Based on the target question-answering model, predict the next word-gram of each word-gram in the target sequence to obtain a predicted word-gram;

[0154] Based on the predicted word-gram, verify each word-gram in the initial answer sequence to obtain a verification result;

[0155] Based on the verification result and the predicted word, the initial answer sequence is modified to obtain the target answer.

[0156] In one embodiment, when the processor amends the initial answer sequence based on the verification result and the predicted word to obtain the target answer, it is configured to implement:

[0157] If the verification result is that the predicted word-gram is inconsistent with the word-gram in the initial answer sequence, the word-gram in the initial answer sequence is replaced with the predicted word-gram to obtain the target answer.

[0158] In one embodiment, when determining the initial question-answering model based on the question to be answered, the processor is used to implement:

[0159] Perform keyword analysis on the question to be answered to obtain keywords corresponding to the question to be answered;

[0160] Based on the keyword, a search is performed in a preset model database to obtain a question-answering model corresponding to the keyword as the initial question-answering model.

[0161] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the methods for obtaining answers to questions based on a distillation model provided in the embodiments of the present application.

[0162] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Medi a Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc., equipped on the computer device.

[0163] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for obtaining a question answer based on a distillation model, characterized in that: include: Obtaining questions to be answered, and determining an initial question-answering model based on the questions to be answered; Acquire a target question-answering model and a distillation model corresponding to the initial question-answering model, and process the question sequence of the questions to be answered based on the distillation model to obtain an initial answer sequence corresponding to the questions to be answered; Concatenating the question sequence with the initial answer sequence to obtain a target sequence; Based on the target question-answering model and the target sequence, the initial answer sequence is verified to obtain the target answer to the question to be answered.

2. The method for obtaining the answer to a question based on a distillation model according to claim 1, characterized in that: The obtaining of a target question answering model and a distillation model corresponding to the initial question answering model includes: Acquire a training data set, and adjust the initial question-answering model based on the training data set to obtain a target question-answering model corresponding to the initial question-answering model; The target question-answering model is distilled to obtain a distilled model corresponding to the initial question-answering model.

3. The method for obtaining answers to questions based on a distillation model according to claim 2, characterized in that: The step of distilling the target question-answering model to obtain a distilled model corresponding to the initial question-answering model includes: Processing the number of layers and dimensions of the target question-answering model to obtain a student model; Based on the training data set, obtaining a first prediction result of the target question-answering model and a second prediction result of the student model; Based on the first prediction result, the second prediction result and the loss function of the initial question-answering model, obtaining the loss function of the student model; Based on the training data set and the loss function, the student model is trained to obtain a distillation model corresponding to the initial question-answering model.

4. The method for obtaining the answer to a question based on a distillation model according to claim 3, characterized in that: The step of obtaining the loss function of the student model based on the first prediction result, the second prediction result, and the loss function of the initial question-answering model includes: Based on the probability distribution of the first prediction result and the probability distribution of the second prediction result, a first loss function is obtained, and the loss function of the initial question-answering model is used as a second loss function; Obtaining the cosine distance between the prediction result of the target question-answering model and the prediction result of the student model; Based on the preset weight coefficient, the first loss function, the second loss function and the cosine distance, the loss function of the student model is obtained.

5. The method for obtaining answers to questions based on a distillation model according to claim 1, characterized in that: The step of verifying the initial answer sequence based on the target question-answering model and the target sequence to obtain the target answer to the question to be answered includes: Based on the target question-answering model, predict the next word-gram of each word-gram in the target sequence to obtain a predicted word-gram; Based on the predicted word-gram, verify each word-gram in the initial answer sequence to obtain a verification result; Based on the verification result and the predicted word, the initial answer sequence is modified to obtain the target answer.

6. The method for obtaining answers to questions based on a distillation model according to claim 5, characterized in that: The step of modifying the initial answer sequence based on the verification result and the predicted word to obtain the target answer includes: If the verification result is that the predicted word-gram is inconsistent with the word-gram in the initial answer sequence, the word-gram in the initial answer sequence is replaced with the predicted word-gram to obtain the target answer.

7. The method for obtaining a question answer based on a distillation model according to any one of claims 1 to 6, characterized in that: The determining of the initial question-answering model based on the question to be answered includes: Perform keyword analysis on the question to be answered to obtain keywords corresponding to the question to be answered; Based on the keyword, a search is performed in a preset model database to obtain a question-answering model corresponding to the keyword as the initial question-answering model.

8. A device for obtaining answers to questions based on a distillation model, characterized in that: include: An initial question-answering model determination module is used to obtain questions to be answered and determine an initial question-answering model based on the questions to be answered; An initial answer sequence acquisition module is used to obtain a target question-answering model and a distillation model corresponding to the initial question-answering model, and based on the distillation model, process the question sequence of the questions to be answered to obtain an initial answer sequence corresponding to the questions to be answered; A target sequence acquisition module, used for concatenating the question sequence with the initial answer sequence to obtain a target sequence; The target answer acquisition module is used to verify the initial answer sequence based on the target question-answering model and the target sequence to obtain the target answer to the question to be answered.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the method for obtaining the answer to a question based on the distillation model as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the method for obtaining a question answer based on a distillation model as described in any one of claims 1 to 7.

Images