Question and answer model training method and device, electronic equipment and storage medium

Abstract

Embodiments of the invention provide a question and answer model training method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a sample question text and a corresponding specified field cue word thereof; fusing the sample question text and the corresponding specified field cue word to obtain a fused question text; performing mask processing on the fusion problem text to obtain a masked problem text; performing feature coding processing on the masked problem text to obtain a coding feature corresponding to the masked problem text; and training a pre-trained question and answer model according to the coding features to obtain a trained question and answer model. The question-answering accuracy of the question-answering model in the specified field can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, device, electronic device and storage medium for a question-answering model. Background Art

[0002] In the field of natural language processing, question-answering systems are designed to accurately respond to various questions, especially when focusing on knowledge in a specific field, which requires high professionalism and accuracy. Although many current question-answering models have the ability to understand a wide range of general languages, once they are involved in highly specialized specific fields such as medicine and finance, they expose significant shortcomings, resulting in low accuracy in question-answering in specific fields. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a training method, device, electronic device and storage medium for a question-answering model, which can improve the accuracy of question-answering of the question-answering model in a specified field.

[0004] To achieve the above objectives, a first aspect of an embodiment of the present application proposes a method for training a question-answering model, the method comprising:

[0005] Obtain sample question text and its corresponding prompt words for a specified field;

[0006] Fusion of the sample question text and its corresponding designated field prompt words to obtain a fused question text;

[0007] Performing masking processing on the fused question text to obtain a masked question text;

[0008] Performing feature encoding processing on the masked question text to obtain encoding features corresponding to the masked question text;

[0009] The pre-trained question-answering model is trained according to the encoding features to obtain a trained question-answering model.

[0010] In some embodiments, the masked question text includes a mask mark, and the pre-trained question-answering model is trained according to the encoding feature to obtain the trained question-answering model, including:

[0011] Inputting the encoded features into the pre-trained question-answering model, predicting first probabilities that different response words in a first vocabulary corresponding to the response words appear in the response text corresponding to the sample question text, and second probabilities that the real question words corresponding to the mask mark belong to different question words in a second vocabulary corresponding to the question words;

[0012] Calculating a first loss value according to the first probability;

[0013] Calculating a second loss value according to the second probability;

[0014] The pre-trained question-answering model is trained according to the first loss value and the second loss value to obtain a trained question-answering model.

[0015] In some embodiments, calculating a first loss value according to the first probability includes:

[0016] Get the response text label corresponding to the sample question text;

[0017] determining a first probability of a target response word appearing in the response text label from the first probabilities;

[0018] A first loss value is calculated based on the first probability of the target response vocabulary.

[0019] In some embodiments, calculating the second loss value according to the second probability includes:

[0020] Determining a real question word corresponding to the mask mark in the fused question text;

[0021] Performing one-hot encoding processing on the real question vocabulary to obtain a one-hot encoding of the real question vocabulary;

[0022] The second loss value is calculated according to the second probability and the one-hot encoding.

[0023] In some embodiments, before training the pre-trained question-answering model according to the encoding features to obtain the trained question-answering model, the method further includes:

[0024] Obtaining a target parameter matrix related to a specified field in the parameter matrix of the pre-trained question-answering model;

[0025] Decomposing the target parameter matrix into a low-rank matrix;

[0026] The step of training the pre-trained question-answering model according to the encoding features to obtain a trained question-answering model includes:

[0027] The pre-trained question-answering model is trained according to the encoding features to adjust the parameters of the low-rank matrix to obtain a trained question-answering model.

[0028] In some embodiments, the pre-trained question-answering model is a pre-trained multilingual question-answering model, the parameter matrix of the pre-trained multilingual question-answering model includes parameter matrices corresponding to different languages, the sample question texts and their corresponding designated domain prompt words include sample question texts in different languages ​​and their corresponding designated domain prompt words, and the fusing of the sample question texts and their corresponding designated domain prompt words to obtain the fused question text includes:

[0029] The sample question texts in different languages ​​and their corresponding answer text labels and designated domain prompt words are integrated to obtain integrated question texts in different languages;

[0030] The step of performing masking on the fused question text to obtain a masked question text includes:

[0031] Masking is performed on the fused question texts in different languages ​​to obtain masked question texts in different languages;

[0032] The step of performing feature encoding processing on the masked question text to obtain encoding features corresponding to the masked question text includes:

[0033] Perform feature encoding processing on the masked question texts in different languages ​​to obtain encoding features corresponding to the masked question texts in different languages;

[0034] The step of training the pre-trained question-answering model according to the encoding features to adjust the parameters of the low-rank matrix to obtain a trained question-answering model includes:

[0035] The pre-trained multilingual question-answering model is trained according to the encoding features corresponding to the masked question texts in different languages ​​to adjust the parameters of the low-rank matrix and the parameters of the parameter matrix corresponding to the different languages ​​to obtain a trained multilingual question-answering model.

[0036] In some embodiments, the pre-trained multilingual question answering model is trained according to the encoding features corresponding to the masked question texts in different languages ​​to adjust the parameters of the low-rank matrix and the parameters of the parameter matrices corresponding to the different languages ​​to obtain the trained multilingual question answering model, further comprising:

[0037] Receive input question text;

[0038] Performing feature encoding processing on the input question text to obtain encoding features corresponding to the input question text;

[0039] The encoding features corresponding to the input question text are input into the trained multilingual question-answering model to obtain a reply text corresponding to the input question text.

[0040] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a training device for a question-answering model, comprising:

[0041] An acquisition unit, used for acquiring a sample question text and its corresponding prompt words for a specified field;

[0042] A fusion unit, used for fusing the sample question text and its corresponding designated field prompt word to obtain a fused question text;

[0043] A masking unit, used for performing masking processing on the fused question text to obtain a masked question text;

[0044] An encoding unit, used for performing feature encoding processing on the masked question text to obtain encoding features corresponding to the masked question text;

[0045] A training unit is used to train the pre-trained question-answering model according to the encoding features to obtain a trained question-answering model.

[0046] In some embodiments, the masked question text includes a mask mark, a training unit, for:

[0047] Inputting the encoded features into the pre-trained question-answering model, predicting first probabilities that different response words in a first vocabulary corresponding to the response words appear in the response text corresponding to the sample question text, and second probabilities that the real question words corresponding to the mask mark belong to different question words in a second vocabulary corresponding to the question words;

[0048] Calculating a first loss value according to the first probability;

[0049] Calculating a second loss value according to the second probability;

[0050] The pre-trained question-answering model is trained according to the first loss value and the second loss value to obtain a trained question-answering model.

[0051] In some embodiments, the training unit is used to:

[0052] Get the response text label corresponding to the sample question text;

[0053] determining a first probability of a target response word appearing in the response text label from the first probabilities;

[0054] A first loss value is calculated based on the first probability of the target response vocabulary.

[0055] In some embodiments, the training unit is used to:

[0056] Determining a real question word corresponding to the mask mark in the fused question text;

[0057] Performing one-hot encoding processing on the real question vocabulary to obtain a one-hot encoding of the real question vocabulary;

[0058] The second loss value is calculated according to the second probability and the one-hot encoding.

[0059] In some embodiments, the training unit is used to:

[0060] Obtaining a target parameter matrix related to a specified field in the parameter matrix of the pre-trained question-answering model;

[0061] Decomposing the target parameter matrix into a low-rank matrix;

[0062] The pre-trained question-answering model is trained according to the encoding features to adjust the parameters of the low-rank matrix to obtain a trained question-answering model.

[0063] In some embodiments, the pre-trained question-answering model is a pre-trained multi-language question-answering model, the parameter matrix of the pre-trained multi-language question-answering model includes parameter matrices corresponding to different languages, the sample question texts and their corresponding designated domain prompt words include sample question texts in different languages ​​and their corresponding designated domain prompt words, and the fusion unit is used to:

[0064] The sample question texts in different languages ​​and their corresponding answer text labels and designated domain prompt words are integrated to obtain integrated question texts in different languages;

[0065] The mask unit is used for:

[0066] Masking is performed on the fused question texts in different languages ​​to obtain masked question texts in different languages;

[0067] The encoding unit is used for:

[0068] Perform feature encoding processing on the masked question texts in different languages ​​to obtain encoding features corresponding to the masked question texts in different languages;

[0069] The training unit is used to:

[0070] The pre-trained multilingual question-answering model is trained according to the encoding features corresponding to the masked question texts in different languages ​​to adjust the parameters of the low-rank matrix and the parameters of the parameter matrix corresponding to the different languages ​​to obtain a trained multilingual question-answering model.

[0071] In some embodiments, the training device of the question-answering model further includes an input unit, the input unit being configured to:

[0072] Receive input question text;

[0073] Performing feature encoding processing on the input question text to obtain encoding features corresponding to the input question text;

[0074] The encoding features corresponding to the input question text are input into the trained multilingual question-answering model to obtain a reply text corresponding to the input question text.

[0075] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0076] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0077] The training method, device, electronic device and storage medium of the question-answering model proposed in the present application obtain sample question texts and their corresponding reply text labels and designated field prompt words; fuse the sample question texts and their corresponding designated field prompt words to obtain a fused question text; perform masking on the fused question text to obtain a masked question text; perform feature encoding on the masked question text to obtain encoding features corresponding to the masked question text; and train a pre-trained question-answering model based on the encoding features to obtain a trained question-answering model, thereby guiding the question-answering model to focus on the designated field through designated field prompt words, further deepening the question-answering model's understanding and representation of the knowledge and language structure of the designated field through masking, and enabling the question-answering model to better use the designated field knowledge to generate reply texts through model training, thereby improving the accuracy of the question-answering model in the designated field. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 is a flow chart of a method for training a question-answering model provided in an embodiment of the present application;

[0079] Figure 2 It is a scenario diagram of the training method of the question-answering model provided in the embodiment of the present application;

[0080] Figure 3 yes Figure 1 Flow chart of step S105 in FIG.

[0081] Figure 4 yes Figure 3 Flow chart of step S202 in FIG.

[0082] Figure 5 yes Figure 3 Flow chart of step S203 in FIG.

[0083] Figure 6 yes Figure 1 The steps before step 105 and the flowchart of step 105;

[0084] Figure 7 yes Figure 1 Flow chart of steps 102 to 105 in FIG.

[0085] Figure 8 yes Figure 7 The flowchart after step 604 in FIG.

[0086] Fig. 9 yes Figure 1 The flowchart after step 105 in FIG.

[0087] Fig.10 It is a structural schematic diagram of a training device for a question-answering model provided in an embodiment of the present application;

[0088] Fig.11 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0090] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0091] The embodiments of the present application provide a method, device, electronic device and storage medium for training a question-answering model, aiming to improve the accuracy of question-answering of the question-answering model in a specified field.

[0092] The training method, device, electronic device and storage medium of the question-answering model provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the training method of the question-answering model in the embodiments of the present application is described.

[0093] The training method of the question-answering model provided in the embodiment of the present application, the field of artificial intelligence technology. The training method of the question-answering model provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the training method of the question-answering model, etc., but is not limited to the above forms.

[0094] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0095] Figure 1 is an optional flowchart of the training method of the question-answering model provided in the embodiment of the present application. Figure 2 It is a scenario diagram of the training method of the question-answering model provided in an embodiment of the present application. The training method of the question-answering model may include but is not limited to steps S101 to S105.

[0096] Step S101, obtaining a sample question text and its corresponding designated field prompt word.

[0097] Among them, sample question text refers to various question statements that people may ask in specified fields, such as finance, medicine, science and technology, tourism or education. These questions should cover various common situations in the specified field so that the question-answering model can learn a sufficiently diverse way of asking questions to accurately respond to various problems in subsequent practical applications. For example, in the financial field, sample question text refers to various specific question statements that people may ask in daily financial activities or in the process of consulting on financial knowledge. These questions cover many aspects such as savings, loans, investments, characteristics of financial products, financial market conditions, etc., reflecting the user's demand for financial-related information in actual scenarios.

[0098] Specified field prompt words: These are specific keywords or phrases that guide the question-answering model to focus on the professional knowledge system in the specified field when dealing with problems. They clearly tell the question-answering model to understand and answer questions from the perspective of the specified field, so as to avoid situations where the answers deviate too much from the professional scope of the specified field.

[0099] Taking the financial field as an example, we first need to collect a large number of questions and their accurate answers from actual scenarios in the financial field, and then mark these questions and answers as sample question texts and reply text labels. At the same time, determine the appropriate prompt words for the specified field so that the question-answering model can be guided to give accurate answers in the specified field. It should be noted that any method provided by the relevant technology can be used to determine the prompt words for the specified field, and no specific restrictions are made here.

[0100] Step S102, merging the sample question text and its corresponding designated field prompt words to obtain a fused question text.

[0101] Among them, after obtaining the sample question text and its corresponding designated field prompt word, each sample question text and its corresponding designated field prompt word can be fused to form a new question text, namely, the fused question text, so that a clear signal can be given to the model at the beginning of the question, so that it can give priority to finding answers from the professional knowledge system in the designated field in the subsequent processing process, thereby improving the accuracy of the question-answering model in the designated field question-answering tasks.

[0102] For example, assuming the sample question text is "How is the interest rate of a bank's fixed deposit determined?" and the specified domain prompt is "[Answer from a financial perspective]", then the fused question text is "[Answer from a financial perspective] How is the interest rate of a bank's fixed deposit determined?".

[0103] Step S103, masking is performed on the fused question text to obtain a masked question text, where the masked question text includes a mask mark.

[0104] Among them, masking is a natural language processing technology that replaces some words in the fused question text with special masking tags (such as [MASK]) according to certain rules to form a masked question text. Its main purpose is to train the question-answering model to predict the masked words based on the context, so as to enhance the question-answering model's ability to understand the text semantics and language structure, and in the context of a specified field, it helps the question-answering model better grasp the relationship between words in the specified field and the expression of the text.

[0105] The fused question text after masking includes parts that are replaced with masked tags. The model needs to predict the original words in these masked positions based on the context and domain knowledge.

[0106] In this embodiment, after the fused question text is obtained, specific words in the fused question text can be selected and replaced with mask marks according to a preset mask rule. The mask rule can be to randomly select words, select words of a specific part of speech (such as nouns, verbs, etc.), select words related to key concepts in a specified field, etc. Then, a masked question text including mask marks is generated.

[0107] For example, suppose the fused question text is "[Answer from a financial perspective] How is the interest rate of a bank time deposit determined?", and suppose that "interest rate" is masked according to the selection of words related to key financial concepts as masking rules, and the masked question text is obtained: "[Answer from a financial perspective] How is the [MASK] of a bank time deposit determined?"

[0108] Step S104, performing feature encoding processing on the masked question text to obtain encoding features corresponding to the masked question text.

[0109] Among them, feature encoding processing is to convert the masked question text into a digital representation that the model can understand and process, that is, encoding features. It is usually achieved by using technologies such as word embedding, which maps the words in the text to a low-dimensional vector space so that different words have different representations in this space. In addition, the semantic relationship between words and the grammatical structure of the text are also considered during the encoding process to provide the model with more comprehensive and accurate input information, so that it can better learn and train based on these features.

[0110] In this embodiment, each vocabulary in the masked question text can be encoded using a selected feature encoding technology, such as a word embedding technology. First, the masked question text is subjected to necessary preprocessing. This may include but is not limited to word segmentation processing (for languages ​​such as Chinese that require word segmentation), removing stop words, and converting the text into a format that the model can process. Taking the masked question text "[Answer from a financial perspective] How is the [MASK] of a bank time deposit determined?" as an example, a Chinese word segmentation tool such as jieba word segmentation can be used to segment it, and the result after word segmentation is: "[Answer from a financial perspective]", "Bank", "Time Deposit", "Of", "[MASK]", "Yes", "How", "Determine", "Of". Then, stop words are removed. Stop words are words that have no substantial contribution or less contribution to semantic expression in the text, such as "Of", "Yes", etc. After removing stop words, the text becomes: "[Answer from a financial perspective]", "Bank", "Time Deposit", "[MASK]", "How", "Determine", which is converted into the vector representation required by the model (such as by word embedding and other technologies), such as mapping each vocabulary with the vector in the word embedding matrix to obtain the vector representation corresponding to each vocabulary. Then, these vectors are adjusted and combined according to the semantic relationship between words, the grammatical structure of the text, etc., and finally the encoding features composed of these vectors are formed.

[0111] Step S105: train the pre-trained question-answering model according to the encoding features to obtain a trained question-answering model.

[0112] Among them, the pre-trained question-answering model is a model that has been preliminarily trained on large-scale text data. It has a certain language understanding and generation ability, and can process the input text and try to generate corresponding answers. In this embodiment, it is necessary to further use the encoding features related to the specified field to perform targeted training on it so that it can better adapt to the question-answering tasks in the specified field, and obtain a trained question-answering model.

[0113] In this embodiment, after obtaining the encoded features of the masked question text, the encoded features of the masked question text can be input into the pre-trained question-answering model, and the model will process the input according to its own structure and algorithm and output the result. Then, the loss value can be calculated by comparing the difference between the output result of the model and the corresponding reply text label, and the difference between the fused question text, etc. Then, the back propagation algorithm is used to adjust the parameters of the model according to the loss value. After multiple rounds of such training process, the trained question-answering model is finally obtained.

[0114] Steps S101 to S105 shown in the embodiment of the present application are as follows: obtaining a sample question text and its corresponding reply text label and a designated field prompt word; fusing the sample question text and its corresponding designated field prompt word to obtain a fused question text; performing masking on the fused question text to obtain a masked question text; performing feature encoding on the masked question text to obtain encoding features corresponding to the masked question text; and training a pre-trained question-answering model based on the encoding features to obtain a trained question-answering model, thereby guiding the question-answering model to focus on the designated field through designated field prompt words, further deepening the question-answering model's understanding and representation of the knowledge and language structure of the designated field through masking, and enabling the question-answering model to better use the designated field knowledge to generate reply text through model training, thereby improving the accuracy of the question-answering model in the designated field.

[0115] See also Figure 3 In some embodiments, the masked question text includes a mask mark, and step S105 may include but is not limited to steps S201 to S204:

[0116] Step S201, input the encoded features into the pre-trained question-answering model, and predict the first probability that different response words in the first vocabulary corresponding to the response words appear in the response text corresponding to the sample question text, and the second probability that the real question words corresponding to the mask mark belong to different question words in the second vocabulary corresponding to the question words.

[0117] The first vocabulary is a set of all possible response words that may appear in the response texts given to various sample question texts in the specified field. For example, when answering questions about bank interest rates, the words that may appear, such as "annual interest rate", "base interest rate", "upward", and "downward", are all in this first vocabulary.

[0118] The first probability refers to the quantitative representation of the likelihood of different response words in the first vocabulary appearing in the response text corresponding to the sample question text. After the input encoding features are processed by the pre-trained question-answering model, the probability of each response word appearing in the correct response to a given sample question text is predicted.

[0119] Mask tags refer to special tags used to replace some words when masking the fusion question text, such as [MASK]. The model needs to predict the real words replaced by the mask tags based on context and other information.

[0120] The second vocabulary corresponding to the question vocabulary is a set of all possible words related to the specified field problem. These words are the words that may be used when asking the specified field problem. For example, in the financial field, words such as "deposit", "loan", "interest rate", and "credit card" are all in this second vocabulary. It is different from the first vocabulary, which is mainly a vocabulary collection for problem expression.

[0121] The second probability represents a quantitative representation of the probability that the real question word corresponding to the mask mark belongs to different question words in the second vocabulary. That is, the model processes the input encoding features to predict the probability that the real word replaced by the mask mark is each different question word in the second vocabulary.

[0122] Specifically, first, the encoding features obtained in the previous step, such as the encoding features of the masked question text "[Answer from a financial perspective] How is the [MASK] of a bank time deposit determined?", are input into the pre-trained question-answering model after feature encoding. Then, the model processes the input encoding features based on its internal neural network structure, the language knowledge that has been learned, and the preliminary understanding of the specified field (from pre-training). During the processing, the model predicts the first probability of each response word in the first vocabulary appearing in the response text corresponding to the sample question text based on the current input encoding features and its own parameter settings. For example, for the response word "base interest rate", the model may predict that its probability of appearing in the correct answer to the question is 0.3; for "market competition", the probability may be 0.2, etc. At the same time, the model also predicts the second probability that the real question word corresponding to the mask mark, such as [MASK] in the above example, belongs to different question words in the second vocabulary corresponding to the question word. For example, for "interest rate" in the second vocabulary, the model predicts that the probability that it is the real problem word replaced by the masked token is 0.4; for "deposit term", the model predicts that the probability that it is the real problem word replaced by the masked token may be 0.1, and so on.

[0123] For example, suppose we have a pre-trained question-answering model that has been preliminarily trained on large-scale financial-related text data. We input the encoded features for "[Answer from a financial perspective] How is the [MASK] of a bank time deposit determined?" into the model, and the first vocabulary contains response words such as "base interest rate", "market competition", and "supply and demand of funds". The model predicts that the first probability of "base interest rate" appearing in the response text corresponding to the sample question text is 0.3, the first probability of "market competition" is 0.2, the first probability of "supply and demand of funds" is 0.1, etc. The second vocabulary contains question words such as "interest rate", "deposit term", and "bank operation". The model predicts that the second probability of the real question word corresponding to [MASK] is "interest rate" is 0.4, the second probability of "deposit term" is 0.1, the second probability of "bank operation" is 0.05, etc.

[0124] Step S202: Calculate a first loss value according to the first probability.

[0125] The first loss value is a quantitative indicator used to measure the difference between the probability of the reply words in the first vocabulary table predicted by the pre-trained question-answering model appearing in the reply text corresponding to the sample question text and the actual situation (i.e., the appearance of the reply words in the reply text label). By calculating the first loss value, we can know the accuracy of the model in predicting reply words, so as to adjust and optimize the model later.

[0126] In this embodiment, after obtaining the first probability that different response words in the first vocabulary corresponding to the response words appear in the response text corresponding to the sample question text, the first loss value can be calculated based on the first probability that different response words in the first vocabulary corresponding to the response words appear in the response text corresponding to the sample question text.

[0127] Step S203: Calculate a second loss value according to the second probability.

[0128] Among them, the second loss value is a quantitative indicator that measures the degree of difference between the probability that the real question words corresponding to the masked tokens belong to different question words in the second vocabulary corresponding to the question words predicted by the model and the actual situation (that is, the actual situation of the real question words replaced by the masked tokens). It reflects the accuracy of the model in predicting the masked words.

[0129] In this embodiment, after obtaining the second probability that the real problem vocabulary corresponding to the mask mark belongs to different problem vocabulary in the second vocabulary table corresponding to the problem vocabulary, the second loss value can be calculated based on the second probability that the real problem vocabulary corresponding to the mask mark belongs to different problem vocabulary in the second vocabulary table corresponding to the problem vocabulary.

[0130] Step S204: train the pre-trained question-answering model according to the first loss value and the second loss value to obtain a trained question-answering model.

[0131] Specifically, after obtaining the first loss value and the second loss value corresponding to each question sample text, the first loss value and the second loss value corresponding to each question sample text are combined. Usually, a weighted summation method can be adopted to calculate the sum of the first loss value and the second loss value of all question sample texts through the following formula (1) to obtain a total loss value.

[0132] For example, the calculation formula of the total loss value L can be:

[0133]

[0134] Among them, L 1m represents the first loss value corresponding to the mth problem sample, L 2m It represents the second loss value corresponding to the mth problem sample. γ and (1-γ) are weight coefficients. γ can be adjusted according to the actual situation. For example, γ can be 0.6.

[0135] Then, based on the total loss value, the back propagation algorithm is used to adjust the parameters of the pre-trained question-answering model. The back propagation algorithm calculates a gradient for the model parameters based on the loss value, and then adjusts the parameter values ​​according to the direction and magnitude of the gradient. For example, if the gradient of a parameter is positive, it means that increasing the value of the parameter will increase the loss value, so the value of the parameter should be reduced; conversely, if the gradient is negative, it means that increasing the value of the parameter will reduce the loss value, so the value of the parameter should be increased.

[0136] After multiple rounds of such training, the parameters of the model are continuously adjusted so that the total loss value gradually decreases until a satisfactory effect is achieved or the preset stopping conditions are met (such as reaching a certain number of training rounds, the loss value is less than a certain threshold, etc.). Finally, a trained question-answering model is obtained, which can provide more accurate answers in question-answering tasks in specified fields.

[0137] See also Figure 4 In some embodiments, step S202 may include but is not limited to steps S301 to S303:

[0138] Step S301, obtaining the reply text label corresponding to the sample question text.

[0139] The response text label is the standard answer content that is accurate, complete, and based on the specified domain expertise and industry norms corresponding to each sample question text. It is used to train the question-answering model so that it can learn the correct responses to different questions and accurately answer similar questions in subsequent practical applications.

[0140] A sample question text may be "How is the interest rate of a bank's fixed deposit determined?", and the corresponding response text label of the sample question text may be "The determination of the bank's fixed deposit interest rate is mainly affected by the central bank's benchmark interest rate. At the same time, banks will increase or decrease the interest rate to a certain extent based on the central bank's benchmark interest rate according to their own operating conditions, market competition conditions, and the relationship between supply and demand of funds. For example, the one-year benchmark interest rate announced by the central bank is 1.5%. A bank may increase the one-year fixed deposit interest rate to 1.8% based on its own situation.".

[0141] Step S302: determining a first probability of a target reply word appearing in a reply text tag from the first probabilities.

[0142] The target reply vocabulary is the vocabulary that actually appears in the reply text label corresponding to the sample question text, and exists in the second vocabulary table, that is, the second vocabulary table includes the vocabulary that actually appears in the reply text label corresponding to different sample question texts.

[0143] Specifically, first, obtain the reply text label content for the specified field question. For example, for the question "How is the interest-free period of bank credit cards calculated?", the reply text label may be "The calculation of the interest-free period of bank credit cards usually depends on factors such as the billing date, repayment date, and consumption date. Generally speaking, the time from the consumption date to the billing date is interest-free, and the time from the billing date to the repayment date is the interest-free period. The length of the interest-free period varies from bank to bank." Then, the reply text label is split into words to obtain individual words. For example, after the above reply text label is split, the words "bank", "credit card", "interest-free period", "calculation", "billing date", "repayment date", "consumption date", "interest calculation", "varies from bank to bank" and so on may be obtained. The above words are the target reply words.

[0144] Next, for each target reply word determined, find the corresponding probability value from the first probabilities predicted by the model. For example, the model predicts that the first probability of "bank" is 0.2, the first probability of "credit card" is 0.3, the first probability of "interest-free period" is 0.4, etc. These are the first probabilities of the target reply words that appear in the reply text label.

[0145] Step S303, calculating a first loss value according to the first probability of the target response vocabulary.

[0146] First, the previous steps have obtained the first probability of each target response word that appears in the response text label corresponding to each sample question text. For example, for the target response word "credit card" that appears in the response text label corresponding to a sample question text, the model predicts a first probability of 0.3; for the target response word "interest-free period" that appears in the response text label corresponding to the sample question text, the model predicts a first probability of 0.4, etc.

[0147] Then, a suitable loss function is used to calculate the first loss value corresponding to each sample question text. Usually, the cross entropy loss function is used, and its calculation formula can be:

[0148] L 1m =-∑ i log (p i ); (2)

[0149] Among them, L 1m represents the first loss value corresponding to the mth problem sample, p i is the first probability of the i-th target response word in the response text label corresponding to the m-th sample question text predicted by the model.

[0150] For example, assuming that the target reply words in the reply text label corresponding to the mth sample question text only include "credit card" and "interest-free period", the model predicts that the first probability of "credit card" is 0.3 and the first probability of "interest-free period" is 0.4, then the first loss value L corresponding to the mth sample question text is calculated by the above formula (2): 1m About 2.12.

[0151] By analogy, the first loss value corresponding to each sample question text can be calculated in the same manner as described above.

[0152] See also Figure 5 In some embodiments, step S203 may include but is not limited to steps S401 to S403:

[0153] Step S401, determining the real question words corresponding to the mask mark in the fused question text.

[0154] Among them, the real question words corresponding to the mask mark in the fused question text are the words replaced by the mask mark.

[0155] For example, a masked question text that has been masked can be obtained. For example, the masked question text is "[Answer from a financial perspective] How is the [MASK] of a bank time deposit determined?" Assuming that the masked question text is obtained by masking the fused question text "[Answer from a financial perspective] How is the interest rate of a bank time deposit determined?", it can be determined that the real question word corresponding to the mask mark [MASK] in the fused question text "[Answer from a financial perspective] How is the interest rate of a bank time deposit determined?" is "interest rate".

[0156] Step S402, performing one-hot encoding processing on the real question vocabulary to obtain the one-hot encoding of the real question vocabulary.

[0157] Among them, One-Hot Encoding is a way of encoding categorical variables into numerical forms that can be processed by machine learning algorithms. In this context, it is to convert the vocabulary in a specified field (here refers to the determined real problem vocabulary) into a specific vector representation, so that each vocabulary has a unique encoding vector, in which only one element is 1 and the rest are 0, in order to distinguish different vocabulary.

[0158] For example, suppose we have a vocabulary in the financial field, which contains words such as "interest rate", "deposit", "loan", and "credit card". After the real problem vocabulary (such as "interest rate" determined in step S401) is determined, it is subjected to one-hot encoding. In this vocabulary, the position corresponding to "interest rate" is set to 1, and the positions corresponding to other words are set to 0. Assuming that "interest rate" is the third word in the vocabulary, its one-hot encoding vector may be [0,0,1,0,...] (the 0 elements corresponding to other words are omitted here, and the specific length depends on the size of the vocabulary), so the one-hot encoding of the real problem vocabulary "interest rate" is obtained.

[0159] Step S403, calculating a second loss value according to the second probability and the one-hot encoding.

[0160] First, obtain the second probability predicted by the model. For example, for the real problem word "interest rate" corresponding to [MASK] in the above example, the model predicts that it is the word "interest rate" in the second vocabulary with a probability of 0.8, and the word "deposit term" with a probability of 0.1, etc. These are the second probabilities we want to use. At the same time, we have obtained the unique hot encoding of the real problem word "interest rate" in step S402, such as [0,0,1,0,…].

[0161] Then, the cross entropy loss function is usually used to calculate the second loss value corresponding to each sample question text, and its calculation formula can be:

[0162] L2m =-∑ j y j log (p j ); (3)

[0163] Among them, L 2m Represents the second loss value corresponding to the mth problem sample, y j is the value of the jth element in the one-hot encoding of the real question word corresponding to the mask mark in the fused question text corresponding to the mth question sample (that is, the actual situation of the real question word in the one-hot encoding, if it is this word, the corresponding position is 1, otherwise it is 0), p j is the second probability predicted by the model that the real question word corresponding to the mask token in the fused question text belongs to the jth question word in the second vocabulary.

[0164] For example, for the real question word "interest rate" corresponding to the mask mark in the fused question text obtained by fusing the mth sample question text and its corresponding specified domain prompt word, the value of the corresponding "interest rate" position in its one-hot encoding is 1, and the second probability of "interest rate" predicted by the model is 0.8. Then, the second loss value L corresponding to the mth sample question text is calculated by the above formula (2): 2m About 0.22.

[0165] By analogy, the second loss value corresponding to each sample question text can be calculated in the same manner as described above.

[0166] See also Figure 6 In some embodiments, before step S105, the following steps may also be included but not limited to steps S501 to S502:

[0167] Step S501, obtaining a target parameter matrix related to a specified field in a parameter matrix of a pre-trained question-answering model.

[0168] The parameter matrix is ​​a matrix used to store various parameters inside the model. In a neural network model (such as the common Transformer architecture model for question-answering tasks), the behavior and performance of the model depend on the values ​​of these parameters. These parameters include but are not limited to the connection weights between neurons, bias terms, etc., which together determine how the model processes and transforms the input data to generate the corresponding output. For example, in a simple fully connected neural network layer, the parameter matrix may be the weight matrix connecting the input layer and the hidden layer.

[0169] The target parameter matrix related to the specified domain refers to the matrix composed of those parameters in the parameter matrix of the pre-trained question-answering model that are closely related to the processing of the specified domain knowledge in the context of the specified domain. These parameters may be the parts that have been learned and adjusted to a certain extent on the text related to the specified domain during the pre-training process, or the parameters corresponding to the parts of the model that are responsible for processing specific types of language structures, lexical semantic relationships, etc. that are more important in the specified domain. For example, taking the specified domain as the financial field, the parameters of the model parts related to processing financial terminology understanding, financial business process description, etc. may constitute this target parameter matrix.

[0170] In this embodiment, any method provided by the relevant technology can be used to obtain the target parameter matrix related to the specified field in the parameter matrix of the pre-trained question-answering model, and no specific limitation is made here.

[0171] Step S502, decomposing the target parameter matrix into a low-rank matrix.

[0172] Among them, in the field of machine learning, especially when fine-tuning pre-trained models to adapt to specified domain tasks, decomposing the target parameter matrix into a low-rank matrix is ​​a common technique. The low-rank matrix can approximate the original target parameter matrix with relatively few parameters, so that the model can be adjusted and optimized more efficiently without changing the overall architecture of the model. For example, a large and complex target parameter matrix, after being decomposed into a low-rank matrix, will have reduced computational and storage requirements in the subsequent training process. Here, the number of low-rank matrices including but not limited to 1 is considered. Multiple low-rank matrices can more accurately approximate the original target parameter matrix from different angles, so as to better capture and adjust the model characteristics related to specific fields.

[0173] Taking the low-rank matrix including the first low-rank matrix and the second low-rank matrix as an example, the target parameter matrix can be decomposed by a specific method (such as singular value decomposition, non-negative matrix decomposition, etc.) to obtain two low-rank matrices, namely the first low-rank matrix and the second low-rank matrix. The combination of these two low-rank matrices can approximate the original target parameter matrix to a certain extent, and in the subsequent training process, by adjusting these two low-rank matrices respectively, the target parameter matrix can be optimized, thereby affecting the performance of the model in the specified field.

[0174] In a possible embodiment, first, a suitable low-rank matrix decomposition method is selected. A common method is singular value decomposition (SVD), which can decompose a matrix into the product form of three matrices, and then extract two low-rank matrices from them as the first low-rank matrix and the second low-rank matrix we need. For example, for the target parameter matrix A, through singular value decomposition A = UEV T, according to actual needs from U, E, V T Extract the appropriate parts to combine into a low-rank matrix. Suppose we want to decompose it into two low-rank matrices. One possible approach is to take the first k1 columns of U, the first k1 rows and columns of E, and V T The first k1 rows of E are combined into the first low-rank matrix; then the last k2 columns of U are taken (the value of k2 is determined according to the specific situation, and k1+k2 is equal to the number of columns of U), the last k2 rows and columns of E, and V T The rear V T Of course, if you want to decompose into more low-rank matrices, you can also follow a similar idea to start from U, E, V T Select different parts to combine.

[0175] Step S105 may include but is not limited to step S503:

[0176] Step S503: train the pre-trained question-answering model according to the encoding features to adjust the parameters of the low-rank matrix to obtain a trained question-answering model.

[0177] In a possible implementation, first, the low-rank matrix obtained in step S502 is associated with the pre-trained question-answering model so that they can be adjusted during the training process. This may involve setting a corresponding parameter update mechanism in the code structure of the model so that the two low-rank matrices can be updated according to the training data. Then, the encoded features are input into the pre-trained question-answering model, and the model processes the input according to its own structure and algorithm and outputs the result. Then, the loss value can be calculated by comparing the difference between the output result of the model and the corresponding reply text label in step S101, and the difference between the fused question text, etc. Then, according to the calculated loss value, the back propagation algorithm is used to adjust the parameters of the low-rank matrix. Specifically, the back propagation algorithm calculates a gradient for each parameter of each low-rank matrix according to the loss value, and then adjusts the value of the parameter according to the direction and magnitude of the gradient. For example, if the gradient of a parameter is positive, it means that increasing the value of the parameter will increase the loss value, so the value of the parameter should be reduced; conversely, if the gradient is negative, it means that increasing the value of the parameter will reduce the loss value, so the value of the parameter should be increased. After multiple rounds of such training, the parameters of the low-rank matrix are continuously adjusted to gradually reduce the loss value until a satisfactory effect is achieved or the preset stop condition is met (such as reaching a certain number of training rounds, the loss value is less than a certain threshold, etc.), and finally the trained question-answering model is obtained. If there are multiple low-rank matrices, such as the sum obtained by the previous decomposition, their parameters will be adjusted independently according to their respective contributions to the loss value and the calculated gradient to achieve effective optimization of the model.

[0178] In an optional embodiment, when a pre-trained question-answering model is trained according to the encoding features to adjust the parameters of the low-rank matrix to obtain the trained question-answering model, the encoding features can be input into the pre-trained question-answering model to predict the first probability that different response words in the first vocabulary corresponding to the response words appear in the response text corresponding to the sample question text, and the second probability that the real question words corresponding to the mask mark belong to different question words in the second vocabulary corresponding to the question words; calculate the first loss value according to the first probability; calculate the second loss value according to the second probability; train the pre-trained question-answering model according to the first loss value and the second loss value to adjust the parameters of the low-rank matrix to obtain the trained question-answering model. The specific implementation can be referred to the previous embodiment, which will not be repeated here.

[0179] See also Figure 7 In some embodiments, the pre-trained question-answering model is a pre-trained multi-language question-answering model, the parameter matrix of the pre-trained multi-language question-answering model includes parameter matrices corresponding to different languages, the sample question texts and their corresponding designated domain prompt words include sample question texts in different languages ​​and their corresponding designated domain prompt words, and step S102 may include but is not limited to step S601:

[0180] Step S601, merging sample question texts in different languages ​​and their corresponding designated field prompt words to obtain fused question texts in different languages.

[0181] Among them, the pre-trained multilingual question-answering model is a model that has been preliminarily trained on large-scale multilingual text data. It has a certain language understanding and generation ability, can process input texts in different languages, and try to generate answers in the corresponding languages. For example, it may have learned common expressions, grammatical rules, and some general knowledge in multiple languages ​​such as English, Chinese, French, and German, laying the foundation for subsequent question-answering tasks in different languages ​​in designated fields.

[0182] In the pre-trained multilingual question-answering model of the neural network architecture, the parameter matrix is ​​a matrix form used to store various parameters of the model. These parameters cover many aspects, such as the connection weights between neurons, which determine the strength of signal transmission between different neurons; and bias terms. Different parameter value settings will cause the model to have different understandings and responses to the same input text. The parameter matrix includes parameter matrices corresponding to different languages, which means that for each language, the model has a set of parameters related to it to process the text information of that language.

[0183] Sample question texts in different languages ​​refer to various questions that people may ask about business, knowledge, etc. related to a specified field, expressed in different languages. For example, in English, it may be "How can I open a bank account?", and in Chinese, it may be "How can I open a bank account?", etc. These sample question texts in different languages ​​cover common consulting content in the specified field, with the aim of enabling the model to learn and respond to questions from users in multiple language environments about the specified field.

[0184] The designated field prompts in different languages ​​correspond to the sample question texts in different languages. They are specific keywords or phrases that guide the model to focus on the professional knowledge system in the designated field when processing questions in different languages. They clearly tell the model to understand and answer questions from the perspective of the designated field to avoid the situation where the answer deviates from the professional scope of the designated field. For example, in English it can be "[Answer from the financial perspective]", and in Chinese it can be "[Answer from a financial perspective]".

[0185] The fused question texts in different languages ​​are new text content formed by fusing sample question texts in different languages ​​with their corresponding designated domain prompts in a certain way. Its purpose is to give a clear domain orientation when presenting questions to the model, and to enable the model to understand and handle the question from a professional perspective of a specific language and the financial and banking fields. For example, for the English sample question "How can I open a bank account?" and the designated domain prompt "[Answer from the financial perspective]", the fused question text is "[Answer from the financial perspective]How can I open a bank account?"; for the Chinese sample question "How can I open a bank account?" and the designated domain prompt "[Answer from a financial perspective]", the fused question text is "[Answer from a financial perspective]How can I open a bank account?"

[0186] For example, first, for each language, sample question texts and corresponding designated domain prompt words are obtained. For example, for English, there are a series of sample question texts about the financial field, such as "How can I open a bank account?", "What are the interest rates for different types of bank deposits?", etc., and the corresponding designated domain prompt words are "[Answer from the financial perspective]". Similarly, for Chinese, there are also corresponding sample question texts and designated domain prompt words.

[0187] Then, for each language, the sample question text is simply concatenated with the corresponding designated domain prompt words or combined in a specific format. Taking English as an example, for the sample question text "How can I open a bank account?", it is fused with the designated domain prompt word "[Answer from the financial perspective]" to obtain "[Answer from the financial perspective]How can I open a bank account?". For Chinese, the sample question text "How can I open a bank account?" is fused with the designated domain prompt word "[Answer from the financial perspective]" to obtain "[Answer from the financial perspective]How can I open a bank account?". This fusion operation is performed on the sample question texts of all different languages ​​to obtain fused question texts of different languages.

[0188] Step S103 may include but is not limited to step S602:

[0189] Step S602, masking is performed on the fused question texts in different languages ​​to obtain masked question texts in different languages.

[0190] The masked question texts in different languages ​​are obtained by masking the fused question texts in different languages. Masking is to replace some words in the fused question text with special masking tags (such as [MASK]) according to certain rules to form a new text containing masking tags. For example, for the above English fused question text "[Answerfrom the financial perspective]How can I open a bank account?", assuming that "open" is replaced with [MASK] according to the rules, the masked question text obtained is "[Answer from the financial perspective]How can I[MASK]a bank account?"; for the Chinese fused question text "[Answer from the financial perspective]How can I open a bank account?", assuming that "open" is replaced with [MASK], the masked question text obtained is "[Answer from the financial perspective]How can I[MASK]a bank account?".

[0191] Exemplarily, first, determine the masking rules. The masking rules can be determined according to specific needs and experimental settings. Common rules include randomly selecting words for masking, selecting words of specific parts of speech for masking (such as nouns, verbs, etc.), selecting words related to key concepts in a specified field for masking, etc. For example, the rule is set to select words related to key concepts in the financial field for masking.

[0192] Then, for each language's fusion question text, the mask processing rules are determined. Take the English fusion question text "[Answer from the financial perspective] How can I open a bank account?" as an example. If we follow the above rules and assume that "open" is a word related to the key concepts in the financial field (because it involves the opening of a bank account), then replace "open" with [MASK] to get "[Answer from the financial perspective] How can I [MASK] a bank account?". For the Chinese fusion question text "[Answer from a financial perspective] How can I open a bank account?", we also assume that "open" is a word related to the key concepts in the financial and banking fields, and replace "open" with [MASK] to get "[Answer from a financial perspective] How can I [MASK] a bank account?". All fusion question texts in different languages ​​are subjected to such mask processing to obtain masked question texts in different languages.

[0193] Step S104 may include but is not limited to step S603:

[0194] Step S603, performing feature encoding processing on the masked question texts in different languages ​​to obtain encoding features corresponding to the masked question texts in different languages.

[0195] Among them, the encoding features corresponding to the masked question texts of different languages ​​are a digital representation obtained after feature encoding processing of the masked question texts of different languages. It is usually implemented by using technologies such as word embedding, that is, mapping the words in the text to a low-dimensional vector space. Different words have different representations in this space, and the semantic relationship between words and the grammatical structure of the text are also considered in the encoding process. These sequences composed of word vectors are encoding features, which are the input form that the model can understand and process. For example, for the English masked question text "[Answer from the financial perspective] How can I [MASK] a bank account?", after encoding processing, each word will be mapped to a specific vector, and these vectors are arranged in a certain order to form the encoding features corresponding to the English masked question text; for the Chinese masked question text "[Answer from the financial perspective] How can I [MASK] a bank account?", after encoding processing, the Chinese words will also be mapped to the corresponding vectors to form the encoding features corresponding to the Chinese masked question text.

[0196] For example, first, select a suitable feature encoding technology. Common feature encoding technologies include word embedding technologies, such as Word2Vec, GloVe, etc., or word embedding methods based on deep learning, such as the word embedding layer of BERT. Here, it is assumed that Word2Vec word embedding technology is used.

[0197] Then, the masked question text of each language is encoded using the selected feature encoding technology. Taking the English masked question text "[Answer from the financial perspective] How can I [MASK] a bank account?" as an example, each word in it is mapped to the word vector in the Word2Vec model to obtain the vector corresponding to each word. For example, "[Answer from the financial perspective]" may be mapped to the vector [0.1, 0.2, -0.1, 0.3], "How" may be mapped to the vector [-0.1, 0.3, 0.2, 0.1], "can" may be mapped to the vector [0.3, -0.2, 0.4, -0.1], "I" may be mapped to the vector [-0.2, 0.1, 0.3, -0.1], "[MASK]" may be mapped to the vector [0.4, -0.1, 0.3, 0.2], "a" may be mapped to the vector [-0.1, 0.3, 0.2, 0.1], "bank" may be mapped to the vector [0.3, -0.2, 0.4, -0.1], and "account" may be mapped to the vector [-0.2, 0.1, 0.3, -0.1]. These vectors are arranged in a certain order to form the encoding features corresponding to the English masked question text. For the Chinese masked question text "[Answer from a financial perspective] How can I [MASK] a bank account?", the Word2Vec word embedding technology is also used (assuming that it has been properly trained or adjusted for Chinese) to map Chinese words to word vectors respectively, and obtain the vectors corresponding to the Chinese words. These vectors are arranged in a certain order to constitute the encoding features corresponding to the Chinese masked question text.

[0198] Step S105 may include but is not limited to step S604:

[0199] Step S604, training the pre-trained multilingual question-answering model according to the encoding features corresponding to the masked question texts in different languages, so as to adjust the parameters of the low-rank matrix and the parameters of the parameter matrix corresponding to different languages, and obtain a trained multilingual question-answering model.

[0200] Exemplarily, first, the encoding features corresponding to the masked question texts in different languages ​​are sequentially input into the pre-trained multilingual question-answering model. For example, the encoding features corresponding to the English masked question text "[Answer from the financial perspective] How can I [MASK] a bank account?", the encoding features corresponding to the Chinese masked question text "[Answer from the financial perspective] How can I [MASK] a bank account?", and the encoding features corresponding to the masked question texts in other languages ​​are input.

[0201] Then, the model will process the input encoding features according to its own structure and algorithm, and generate output results corresponding to different languages. For example, for the encoding features of English input, the model will generate output results corresponding to English; for the encoding features of Chinese input, the model will generate output results corresponding to Chinese, and so on.

[0202] Next, the loss value is calculated by comparing the difference between the output results generated by the model and the corresponding reply text labels (the reply text labels corresponding to different languages) and the fused question text (the fused question text corresponding to different languages). For the English sample question text, the corresponding loss value is calculated; for the Chinese sample question text, the corresponding loss value is calculated, and so on. Here, the loss value should be calculated for different languages ​​separately, because different languages ​​have their own characteristics.

[0203] The response text labels in different languages ​​correspond to the sample question texts in different languages. They are accurate and complete standard answers given in the corresponding languages ​​based on the professional knowledge and industry standards in the specified fields. For example, for the English sample question "How can I open a bank account?", the response text label may be "You can go to the nearest bank branch with your valid identification documents, such as your passport or driver's license, and fill out the account opening application form. The bank staff will guide you through the process.", while the response text label corresponding to the Chinese sample question will be a similar accurate answer expressed in Chinese. These response text labels are the goal of model training, that is, to make the answers generated by the model as close to these standard answers as possible.

[0204] Finally, according to the calculated loss value, the back propagation algorithm is used to adjust the parameters of the low-rank matrix and the parameter matrix corresponding to different languages. For example, for the English language part, according to the loss value corresponding to the English sample question text, the parameters in the low-rank matrix related to English (such as the low-rank matrix obtained by assuming decomposition, etc.) and the parameter matrix corresponding to the English language are adjusted; for the Chinese language part, according to the loss value corresponding to the English sample question text, the parameters in the low-rank matrix related to Chinese and the parameter matrix corresponding to the Chinese language are adjusted. Through multiple rounds of such training process, these parameters are continuously adjusted so that the loss value is gradually reduced until a satisfactory effect is achieved or the preset stop condition is met (such as reaching a certain number of training rounds, the loss value is less than a certain threshold, etc.), and finally a trained multilingual question-answering model is obtained.

[0205] In one possible implementation, the output results corresponding to different languages ​​include a first probability that different response words in a first vocabulary corresponding to the response words in different languages ​​appear in the response text corresponding to the sample question text in the different languages, and a second probability that the real question words corresponding to the mask marks included in the masked question texts of the different languages ​​belong to different question words in the second vocabulary corresponding to the question vocabulary of the different languages. The first loss value of the sample question texts in the different languages ​​can be calculated based on the first probability that the different response words in the first vocabulary corresponding to the response words in the different languages ​​appear in the response text corresponding to the sample question texts in the different languages; the second loss value of the sample question texts in the different languages ​​can be calculated based on the second probability that the real question words corresponding to the mask marks included in the masked question texts of the different languages ​​belong to different question words in the second vocabulary corresponding to the question vocabulary of the different languages; the total loss value of the sample question texts in the different languages ​​and the total loss value of the sample question texts in all languages ​​are calculated based on the first loss value of the sample question texts in the different languages ​​and the second loss value of the sample question texts in the different languages, and the parameters of the low-rank matrix related to the different languages ​​and the parameter matrix corresponding to the different languages ​​are adjusted based on the total loss value of the sample question texts in the different languages. For example, for the English language part, a first loss value of the sample question text of the English language is calculated based on the first probability that different response words in the first vocabulary corresponding to the response words of the English language appear in the response text corresponding to the sample question text of the English language; a second loss value of the sample question text of the English language is calculated based on the second probability that the real question words corresponding to the mask marks included in the masked question text of the English language belong to different question words in the second vocabulary corresponding to the question words of the English language; a total loss value of the sample question text of the English language is calculated based on the first loss value of the sample question text of the English language and the second loss value of the sample question text of the English language, and the parameters of the low-rank matrix related to English and the parameters in the parameter matrix corresponding to the English language are adjusted based on the total loss value of the sample question text of the English language; for the Chinese language part, the total loss value of the sample question text of the Chinese language is obtained in the same way to adjust the parameters of the low-rank matrix related to Chinese and the parameters in the parameter matrix corresponding to the Chinese language.

[0206] In an optional embodiment, the low-rank matrix can also be adjusted according to the total loss value of sample question texts corresponding to different languages, and for the English language part, the parameters in the parameter matrix corresponding to the English language are adjusted according to the loss value corresponding to the English sample question text; for the Chinese language part, the parameters in the parameter matrix corresponding to the Chinese language are adjusted according to the loss value corresponding to the English sample question text. Through multiple rounds of such training process, these parameters are continuously adjusted so that the loss value is gradually reduced until a satisfactory effect is achieved or a preset stop condition is met (such as reaching a certain number of training rounds, the loss value is less than a certain threshold, etc.), and finally a trained multilingual question-answering model is obtained.

[0207] In one possible implementation, the output results corresponding to different languages ​​include a first probability that different response words in a first vocabulary corresponding to response words in different languages ​​appear in response texts corresponding to sample question texts in different languages, and a second probability that real question words corresponding to mask marks included in masked question texts in different languages ​​belong to different question words in a second vocabulary corresponding to question words in different languages. The first loss value corresponding to the sample question texts in different languages ​​can be calculated based on the first probability that different response words in the first vocabulary corresponding to response words in different languages ​​appear in response texts corresponding to sample question texts in different languages, such as calculating the first loss value corresponding to the sample question text in the English language based on the first probability that different response words in the first vocabulary corresponding to response words in the English language appear in response texts corresponding to the sample question text in the English language; the first loss value corresponding to the sample question text in the English language can be calculated ... The second probability that the real question vocabulary corresponding to the mask mark belongs to different question vocabulary in the second vocabulary table corresponding to the question vocabulary of different languages ​​is used to calculate the second loss value of the sample question text corresponding to the different languages, such as calculating the second loss value of the sample question text corresponding to the English language based on the second probability that the real question vocabulary corresponding to the mask mark included in the masked question text of the English language belongs to different question vocabulary in the second vocabulary table corresponding to the question vocabulary of the English language; according to the first loss value of the sample question text corresponding to the different languages ​​and the second loss value of the sample question text corresponding to the different languages, calculate the total loss value of the sample question text corresponding to the different languages ​​and the total loss value of the sample question texts corresponding to all languages, adjust the parameters of the parameter matrix corresponding to the different languages ​​according to the total loss value of the sample question texts corresponding to the different languages, and adjust the parameters of the low-rank matrix according to the total loss value of the sample question texts corresponding to all languages.

[0208] It should be noted that, based on the first probability that different response words in the first vocabulary corresponding to the response words in different languages ​​appear in the response text corresponding to the sample question text in different languages, the first loss value of the sample question text corresponding to the different languages ​​is calculated; based on the second probability that the real question words corresponding to the mask marks included in the masked question texts in different languages ​​belong to different question words in the second vocabulary corresponding to the question words in different languages, the specific implementation of calculating the second loss value of the sample question text corresponding to the different languages ​​can be referred to the previous embodiments, which will not be repeated here.

[0209] It should also be noted that how to calculate the total loss value of sample question texts corresponding to different languages ​​and the total loss value of sample question texts corresponding to all languages ​​can be found in the previous embodiments and will not be repeated here.

[0210] For example, the total loss value corresponding to the sample question texts in different languages ​​is obtained by weighted summing the first loss value and the second loss value corresponding to each sample question text in the sample question texts in different languages. For example, the total loss value corresponding to the sample question text in the English language is obtained by weighted summing the first loss value and the second loss value corresponding to each sample question text in the sample question text in the English language.

[0211] For example, the total loss value corresponding to the sample question texts of all languages ​​is obtained by weighted summing the first loss value and the second loss value corresponding to each sample question text in the sample question texts of all languages. For example, the total loss value corresponding to the sample question texts of all languages ​​is obtained by weighted summing the first loss value and the second loss value corresponding to each sample question text in the sample question text of the English language, the first loss value and the second loss value corresponding to each sample question text in the sample question text of the Chinese language, and the first loss value and the second loss value corresponding to each sample question text in the sample question texts of other languages.

[0212] See also Figure 8 In some embodiments, after step S604, the following steps may also include but are not limited to steps S701 to S703:

[0213] Step S701, receiving input question text.

[0214] The input question text refers to the text content that the user enters in the question-and-answer scenario in the specified field and wants to obtain relevant information. Taking the specified field as the financial field as an example, it can be a question expression about financial products, banking business, financial market dynamics, etc. For example, "How is the interest rate of bank fixed deposits determined?" "How is the interest-free period of credit cards calculated?" "What financial products are suitable for novices recently?" etc. These texts are the starting point for starting the question-and-answer process, aiming to obtain accurate responses from the trained question-and-answer model.

[0215] For example, first, in actual application scenarios (such as online customer service systems of financial institutions, financial knowledge question-and-answer platforms, etc.), corresponding input interfaces are set so that users can input their questions about the financial field. This input interface can be in the form of a text box on a web page, an input bar in a mobile application, etc.

[0216] Then, when the user enters the question text in the input interface, the system will obtain the user's input content in real time. For example, if the user enters "What is the annual fee for bank credit cards?" in the input box of the financial knowledge question and answer platform, the system will receive the complete question text and prepare to pass it to the subsequent processing link.

[0217] Step S702, performing feature encoding processing on the input question text to obtain the encoding features corresponding to the input question text.

[0218] Exemplarily, the input question text is first preprocessed as necessary. This may include but is not limited to word segmentation (for languages ​​such as Chinese that require word segmentation), removal of stop words, conversion of the text into a format that the model can process, and other operations. Then, the format that the model can process is converted to obtain the encoding features corresponding to the input question text. The specific implementation process can be found in the previous embodiment, which will not be repeated here.

[0219] Step S703: input the encoding features corresponding to the input question text into the trained multilingual question-answering model to obtain the reply text corresponding to the input question text.

[0220] Exemplarily, first, the coded features corresponding to the input question text obtained by processing in step S702 are input into the trained multilingual question answering model. The model processes the input coded features based on its internal trained parameters (including parameter matrices corresponding to different languages ​​and adjusted low-rank matrices, etc.).

[0221] Secondly, when processing the input encoding features, the model will generate a response text in the corresponding language based on its internal algorithm and structure, combined with the knowledge about the specified field learned during the previous training process and the language characteristics of different languages. For example, for a question text entered in English, the model will generate an English response text; for a question text entered in Chinese, the model will generate a Chinese response text. Specifically:

[0222] The model will use the language patterns, vocabulary collocations and other knowledge learned during the training process to determine the words that may appear in the reply text based on the question text features corresponding to the input encoding features. For example, for the question text "What is the annual fee for bank credit cards?", the model may determine that words or phrases such as "varies depending on the type of card" and "usually between tens and hundreds of yuan" have a higher probability of appearing in the reply text based on the previously learned knowledge and vocabulary collocations about credit card annual fees.

[0223] The model also considers the grammatical rules, word order and other characteristics of different languages ​​to generate reply texts that conform to the language standards. For example, when generating English reply texts, the vocabulary arrangement and sentence construction will be carried out according to the English grammatical rules; when generating Chinese reply texts, the corresponding operations will be carried out according to the Chinese grammatical rules.

[0224] Finally, based on the above selection of response words and consideration of language norms, the model combines the selected response words into a complete response text according to certain language rules and logical order. For example, for the Chinese question text "What is the annual fee for bank credit cards?", the Chinese response text that the model may generate is: "The annual fee for bank credit cards generally varies according to the type of card. The annual fee for common credit cards is generally between tens and hundreds of yuan. For example, the annual fee for ordinary cards may be lower, while the annual fee for gold cards and platinum cards is relatively high. At the same time, some credit cards can reduce or waive the annual fee by meeting certain conditions."

[0225] See also Fig. 9 In some embodiments, after step S105, the process may also include but not be limited to steps S801 to S803.

[0226] Step S801, receiving input question text.

[0227] Step S802: Perform feature encoding processing on the input question text to obtain the encoding features corresponding to the input question text.

[0228] Step S803, inputting the encoding features corresponding to the input question text into the trained question-answering model to obtain the reply text corresponding to the input question text.

[0229] The specific implementation of steps S801 to S802 can refer to the previous embodiments and will not be repeated here.

[0230] See also Fig.10 The embodiment of the present application also provides a training device for a question-answering model, which can implement the training method for the question-answering model, and the device includes:

[0231] An acquisition unit, used for acquiring a sample question text and its corresponding prompt words for a specified field;

[0232] A fusion unit is used to fuse the sample question text and its corresponding designated domain prompt words to obtain a fused question text;

[0233] A masking unit, used for performing masking processing on the fused question text to obtain a masked question text;

[0234] An encoding unit, used for performing feature encoding processing on the masked question text to obtain encoding features corresponding to the masked question text;

[0235] The training unit is used to train the pre-trained question-answering model according to the encoding features to obtain a trained question-answering model.

[0236] In some embodiments, the masked question text includes a mask mark, a training unit, for:

[0237] Input the encoded features into a pre-trained question-answering model, and predict a first probability that different response words in a first vocabulary corresponding to the response words appear in the response text corresponding to the sample question text, and a second probability that the real question words corresponding to the mask mark belong to different question words in a second vocabulary corresponding to the question words;

[0238] Calculate a first loss value according to the first probability;

[0239] Calculate a second loss value according to the second probability;

[0240] The pre-trained question-answering model is trained according to the first loss value and the second loss value to obtain a trained question-answering model.

[0241] In some embodiments, the training unit is used to:

[0242] Get the response text label corresponding to the sample question text;

[0243] determining, from the first probabilities, a first probability of a target response word appearing in the response text label;

[0244] Calculate the first loss value based on the first probability of the target response vocabulary.

[0245] In some embodiments, the training unit is used to:

[0246] Determine the real question words corresponding to the mask tokens in the fused question text;

[0247] Perform one-hot encoding on the real question words to obtain the one-hot encoding of the real question words;

[0248] A second loss value is calculated according to the second probability and the one-hot encoding.

[0249] In some embodiments, the training unit is used to:

[0250] Get the target parameter matrix related to the specified domain in the parameter matrix of the pre-trained question-answering model;

[0251] Decompose the target parameter matrix into a low-rank matrix;

[0252] According to the encoding features, the pre-trained question-answering model is trained to adjust the parameters of the low-rank matrix to obtain the trained question-answering model.

[0253] In some embodiments, the pre-trained question-answering model is a pre-trained multi-language question-answering model, the parameter matrix of the pre-trained multi-language question-answering model includes parameter matrices corresponding to different languages, the sample question texts and their corresponding designated domain prompt words include sample question texts in different languages ​​and their corresponding designated domain prompt words, and the fusion unit is used to:

[0254] The sample question texts in different languages ​​and their corresponding answer text labels and designated domain prompt words are integrated to obtain integrated question texts in different languages;

[0255] Mask unit, used to:

[0256] Masking is performed on the fused question texts in different languages ​​to obtain masked question texts in different languages;

[0257] Coding unit for:

[0258] Perform feature encoding processing on the masked question texts in different languages ​​to obtain encoding features corresponding to the masked question texts in different languages;

[0259] Training unit for:

[0260] According to the encoding features corresponding to the masked question texts in different languages, the pre-trained multilingual question-answering model is trained to adjust the parameters of the low-rank matrix and the parameters of the parameter matrix corresponding to different languages ​​to obtain the trained multilingual question-answering model.

[0261] In some embodiments, the training device of the question-answering model further includes an input unit, the input unit being configured to:

[0262] Receive input question text;

[0263] Perform feature encoding processing on the input question text to obtain the encoding features corresponding to the input question text;

[0264] The encoded features corresponding to the input question text are input into the trained multilingual question-answering model to obtain the reply text corresponding to the input question text.

[0265] The specific implementation of the training device for the question-answering model is basically the same as the specific implementation of the training method for the above-mentioned question-answering model, and will not be repeated here.

[0266] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the training method of the question-answering model when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0267] See also Fig.11 , Fig.11 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0268] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0269] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the training method of the question-answering model in the embodiment of this application;

[0270] Input / output interface 903, used to implement information input and output;

[0271] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0272] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0273] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0274] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the training method of the above-mentioned question-answering model.

[0275] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0276] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0277] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0278] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0279] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0280] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0281] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0282] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0283] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0284] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0285] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A training method for a question-answering model, characterized in that: include: Obtain sample question text and its corresponding prompt words for a specified field; Fusion of the sample question text and its corresponding designated field prompt words to obtain a fused question text; Performing masking processing on the fused question text to obtain a masked question text; Performing feature encoding processing on the masked question text to obtain encoding features corresponding to the masked question text; The pre-trained question-answering model is trained according to the encoding features to obtain a trained question-answering model.

2. The training method of the question-answering model according to claim 1, characterized in that: The masked question text includes a mask mark, and the pre-trained question-answering model is trained according to the encoding feature to obtain a trained question-answering model, including: Inputting the encoded features into the pre-trained question-answering model, predicting first probabilities that different response words in a first vocabulary corresponding to the response words appear in the response text corresponding to the sample question text, and second probabilities that the real question words corresponding to the mask mark belong to different question words in a second vocabulary corresponding to the question words; Calculating a first loss value according to the first probability; Calculating a second loss value according to the second probability; The pre-trained question-answering model is trained according to the first loss value and the second loss value to obtain a trained question-answering model.

3. The training method of the question-answering model according to claim 2, characterized in that: The calculating a first loss value according to the first probability includes: Get the response text label corresponding to the sample question text; determining a first probability of a target response word appearing in the response text label from the first probabilities; A first loss value is calculated based on the first probability of the target response vocabulary.

4. The training method of the question-answering model according to claim 2, characterized in that: The calculating a second loss value according to the second probability includes: Determining a real question word corresponding to the mask mark in the fused question text; Performing one-hot encoding processing on the real question vocabulary to obtain a one-hot encoding of the real question vocabulary; The second loss value is calculated according to the second probability and the one-hot encoding.

5. The training method of the question-answering model according to claim 1, characterized in that: Before training the pre-trained question-answering model according to the encoding features to obtain the trained question-answering model, the method further includes: Obtaining a target parameter matrix related to a specified field in the parameter matrix of the pre-trained question-answering model; Decomposing the target parameter matrix into a low-rank matrix; The step of training the pre-trained question-answering model according to the encoding features to obtain a trained question-answering model includes: The pre-trained question-answering model is trained according to the encoding features to adjust the parameters of the low-rank matrix to obtain a trained question-answering model.

6. The training method of the question-answering model according to claim 5, characterized in that: The pre-trained question-answering model is a pre-trained multi-language question-answering model, the parameter matrix of the pre-trained multi-language question-answering model includes parameter matrices corresponding to different languages, the sample question texts and their corresponding designated field prompt words include sample question texts in different languages ​​and their corresponding designated field prompt words, and the fusion of the sample question texts and their corresponding designated field prompt words to obtain the fused question text includes: The sample question texts in different languages ​​and their corresponding prompt words in the designated field are integrated to obtain integrated question texts in different languages; The step of performing masking on the fused question text to obtain a masked question text includes: Masking is performed on the fused question texts in different languages ​​to obtain masked question texts in different languages; The step of performing feature encoding processing on the masked question text to obtain encoding features corresponding to the masked question text includes: Perform feature encoding processing on the masked question texts in different languages ​​to obtain encoding features corresponding to the masked question texts in different languages; The step of training the pre-trained question-answering model according to the encoding features to adjust the parameters of the low-rank matrix to obtain a trained question-answering model includes: The pre-trained multilingual question-answering model is trained according to the encoding features corresponding to the masked question texts in different languages ​​to adjust the parameters of the low-rank matrix and the parameters of the parameter matrix corresponding to the different languages ​​to obtain a trained multilingual question-answering model.

7. The training method of the question-answering model according to claim 6, characterized in that: After the pre-trained multilingual question answering model is trained according to the encoding features corresponding to the masked question texts in different languages ​​to adjust the parameters of the low-rank matrix and the parameters of the parameter matrices corresponding to the different languages ​​to obtain the trained multilingual question answering model, the method further includes: Receive input question text; Performing feature encoding processing on the input question text to obtain encoding features corresponding to the input question text; The encoding features corresponding to the input question text are input into the trained multilingual question-answering model to obtain a reply text corresponding to the input question text.

8. A training device for a question-answering model, characterized in that: include: An acquisition unit, used for acquiring a sample question text and its corresponding prompt words for a specified field; A fusion unit, used for fusing the sample question text and its corresponding designated field prompt word to obtain a fused question text; A masking unit, used for performing masking processing on the fused question text to obtain a masked question text; An encoding unit, used for performing feature encoding processing on the masked question text to obtain encoding features corresponding to the masked question text; The training unit is used to train the pre-trained question-answering model according to the encoding features to obtain a trained question-answering model.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the training method of the question-answering model described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the training method of the question-answering model described in any one of claims 1 to 7 is implemented.

Images