Model training method, device, electronic device and storage medium
The two-stage training method with redundant statement filtering and contrastive learning addresses the issue of inaccurate semantic associations in language models, enhancing the fluency and accuracy of human-computer dialogue systems.
Patent Information
- Application Number
- CN202011566127.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-25
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2040-12-25
AI Technical Summary
Existing language models for human-computer dialogue systems struggle with inaccurate semantic associations between question and answer pairs, leading to suboptimal training effects and fluency in human-computer interactions.
A two-stage training method involving filtering redundant statements to create positive and negative samples, ensuring strict semantic correspondence in question and answer pairs, and utilizing contrastive learning to enhance the language model's training process.
Ensures accurate and fluent human-computer dialogue by precisely delineating positive and negative samples, improving the language model's ability to learn intrinsic semantic connections.
Smart Images
Figure CN114691815B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a model training method, device, electronic device and storage medium. Background Art
[0002] With the development of artificial intelligence (AI), various intelligent robots are increasingly entering people's lives, such as service robots, autonomous vending robots, and other commercial robots. To facilitate user experience, intelligent robots typically support multiple human-computer interaction methods, such as touch-based and voice-based interaction.
[0003] In actual applications, after receiving the dialogue content input by the user, the intelligent robot usually first determines the semantics of the dialogue content, and then obtains the response content corresponding to the dialogue content based on the semantics, thereby realizing human-computer interaction. Summary of the Invention
[0004] Embodiments of the present invention provide a model training method, device, electronic device, and storage medium to ensure the fluency of human-computer dialogue.
[0005] An embodiment of the present invention provides a model training method, including:
[0006] Get the first question and answer set;
[0007] Filtering out redundant statements in the first question and answer set to obtain a second question and answer set, wherein any question and answer pair in the second question and answer set includes a target question statement and a target answer statement;
[0008] Determining a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set;
[0009] A language model is trained based on the second question and answer set and the third question and answer set.
[0010] An embodiment of the present invention provides a model training device, comprising:
[0011] An acquisition module, used to acquire a first question and answer set;
[0012] a filtering module, configured to filter out redundant statements in the first question and answer set to obtain a second question and answer set, wherein any question and answer pair in the second question and answer set includes a target question statement and a target answer statement;
[0013] A set determination module, configured to determine a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set;
[0014] A training module is used to train a language model based on the second question and answer set and the third question and answer set.
[0015] An embodiment of the present invention provides an electronic device, comprising: a processor and a memory; wherein the memory is configured to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the following are implemented:
[0016] Get the first question and answer set;
[0017] Filtering out redundant statements in the first question and answer set to obtain a second question and answer set, wherein any question and answer pair in the second question and answer set includes a target question statement and a target answer statement;
[0018] Determining a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set;
[0019] A language model is trained based on the second question and answer set and the third question and answer set.
[0020] An embodiment of the present invention provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to perform at least the following actions:
[0021] Get the first question and answer set;
[0022] Filtering out redundant statements in the first question and answer set to obtain a second question and answer set, wherein any question and answer pair in the second question and answer set includes a target question statement and a target answer statement;
[0023] Determining a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set;
[0024] A language model is trained based on the second question and answer set and the third question and answer set.
[0025] The model training method provided by the present invention obtains a first question and answer set, filters out redundant sentences in the first question and answer set, and obtains a second question and answer set. Any question and answer pair in the second question and answer set can be composed of a target question sentence and a target answer sentence, and the two are semantically related and can be considered as positive samples. At this time, the target question sentence and the non-target answer sentence in the second question and answer set can also be used to form a third question and answer set. The question and answer pairs in the third question and answer set are not semantically related and can be considered as negative samples. Finally, the language model training is implemented based on the second question and answer set and the third question and answer set.
[0026] In actual applications, in the first question and answer set, there may be multiple question statements and answer statements with the same or similar semantics. Therefore, it is easy for a question statement to be semantically associated with multiple answer statements, or for an answer statement to be semantically associated with multiple question statements. The situation where the question statement and the answer statement are not semantically corresponding one to one will affect the training effect of the language model. In this solution, by filtering out redundant statements, it is possible to ensure that the question statement and the answer statement in the second question and answer set are strictly semantically corresponding one to one, and that the question statement and the answer statement in the third question and answer set are not semantically corresponding, making the division of positive and negative samples more accurate and unique. Using accurately divided positive and negative samples for training can enable the language model to learn the intrinsic connection between question statements and answer statements. This connection includes both semantic association and semantic non-association, thereby ensuring the model training effect and further ensuring the fluency of human-computer dialogue. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0028] Figure 1 A schematic diagram of the structure of a dialogue system provided by an embodiment of the present invention;
[0029] Figure 2 A flowchart of a model training method provided by an embodiment of the present invention;
[0030] Figure 3 A flowchart of another model training method provided by an embodiment of the present invention;
[0031] Figure 4 A schematic diagram of a language model and its training process provided by an embodiment of the present invention;
[0032] Figure 5a A schematic diagram of the application of the model training method provided by an embodiment of the present invention in a banking scenario;
[0033] Figure 5b Another schematic diagram of applying the model training method provided by an embodiment of the present invention in a banking scenario;
[0034] Figure 5c Another schematic diagram of applying the model training method provided by an embodiment of the present invention in a banking scenario;
[0035] Figure 6A schematic structural diagram of a model training device provided by an embodiment of the present invention;
[0036] Figure 7 For Figure 6 A schematic structural diagram of an electronic device corresponding to the model training device provided in the illustrated embodiment. DETAILED DESCRIPTION
[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0038] The terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. As used in the embodiments of the present invention and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms. Unless the context clearly indicates otherwise, "a plurality" generally includes at least two.
[0039] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.
[0040] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.
[0041] Before describing the model training method provided by the embodiment of the present invention, an exemplary description of the actual use of the language model is provided:
[0042] As mentioned in the background technology, intelligent robots used in different scenarios can provide human-computer interaction functions, that is, human-computer dialogue functions. This dialogue function can usually be implemented by a dialogue system configured in the intelligent robot. The specific structure of the dialogue system can be as follows: Figure 1 shown.
[0043] The workflow of a dialogue system can be described as follows: an intelligent robot receives a question from a user and inputs it into the dialogue system. The language model in the dialogue system converts the question into a sentence vector. The retrieval model in the dialogue system then calculates the similarity between the sentence vector corresponding to the question and the sentence vectors in the knowledge base. The knowledge base contains pre-collected sentence vectors corresponding to different sentences. The ranking model in the system ranks the sentence vectors in the knowledge base according to their similarity and determines the sentence vector with the highest similarity in the knowledge base as the target sentence vector. Finally, the dialogue system outputs the textual representation of this target sentence vector, which is the response sentence.
[0044] In practical applications, the aforementioned dialogue system containing language models can optionally be deployed on intelligent robots such as service robots and autonomous vending robots. The dialogue system can also be deployed in human-computer dialogue plug-ins (or human-computer dialogue interfaces or human-computer dialogue function modules) integrated into systems such as online shopping and public service systems. The dialogue system can also be deployed on intelligent terminals such as smart home appliances and smart wearable devices. Broadly speaking, the dialogue system containing language models can be deployed on any device or system that supports human-computer dialogue.
[0045] As can be seen from the working process of the above-mentioned dialogue system, the language model is a core component of the system. Whether the language model can convert question sentences into accurate sentence vectors will have a direct impact on the fluency of human-computer dialogue. Therefore, to ensure the fluency of human-computer dialogue, the model training method provided by the present invention can be used to train the language model so that the language model can output accurate sentence vectors.
[0046] In a human-computer dialogue scenario, the training samples used to train the language model can be pre-collected question-answer pairs. Based on the training samples composed of question-answer pairs, the overall concept of the model training method provided by various embodiments of the present invention can be summarized as a two-stage training method combined with contrastive learning.
[0047] Specifically, a first training sample is first obtained (i.e., the first question-and-answer set in the following embodiment), and then a second training sample is generated based on the first training sample (i.e., the fourth question-and-answer set in the following embodiment). The first training sample is used as a positive sample, and the second training sample is used as a negative sample for the first stage of training to obtain a language model (i.e., the first language model in the following embodiment).
[0048] Then, redundant sentences in the first training sample are filtered out to obtain a third training sample (i.e., the second question-answer set in the following embodiments). Next, a fourth training sample (i.e., the third question-answer set in the following embodiments) is generated based on the third training sample. The third training sample is used as a positive sample, and the fourth training sample is used as a negative sample for the second stage of training. This means that the language model obtained after the first stage of training is further trained a second time, thereby obtaining the second language model in the following embodiments.
[0049] For the specific implementation of the above two-stage training process, please refer to the description in the following embodiments.
[0050] Based on the above description, some embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the absence of conflicts between the embodiments, the following embodiments and the features in the embodiments may be combined with each other. In addition, the step timings in the following method embodiments are only examples and are not strictly limiting.
[0051] Figure 2 This is a flow chart of a model training method provided by an embodiment of the present invention. The model training method provided by an embodiment of the present invention can be executed by a training device. It is understandable that the training device can be implemented as software, or a combination of software and hardware, specifically a server. Figure 2 As shown, the method includes the following steps:
[0052] 101. Obtain the first question and answer set.
[0053] The user can collect a first question-answer set through the Internet. The first question-answer set consists of multiple question-answer pairs, and the question statement and the answer statement in each question-answer pair are semantically related. Therefore, the question-answer pairs in the first question-answer set can be considered as positive samples.
[0054] In order to ensure the training effect of the language model, negative samples can also be generated based on the above positive samples. Assume that the question-answer pair 1 in the first question-answer set consists of question statement Q1 and answer statement A1. At this time, the answer statements in the first question-answer set except answer statement A1 can be re-formed into question-answer pairs with the question statement Q1, and the re-formed question-answer pairs are used as negative samples. Among them, question-answer pair 1 can be any question-answer pair in the first question-answer set. This negative sample is also Figure 3 The fourth question and answer set in the illustrated embodiment.
[0055] In practical applications, the above positive and negative samples can be used directly to train the language model. The combined use of positive and negative samples enables the language model to learn the semantic relationship between each question statement and each answer statement in the first question and answer set, which is to achieve contrastive learning, thus ensuring the training effect of the model.
[0056] It should be noted that, ideally, the question statements and answer statements in the negative samples are not semantically related. However, in actual conversations, especially in casual chat scenarios, the semantic association between question statements and answer statements is very flexible, which can easily lead to a question statement in the first question and answer set being semantically associated with multiple answer statements, or a response statement in the first question and answer pair being semantically associated with multiple question statements. This makes it impossible for the question statements and answer statements in the first question and answer set to be semantically one-to-one corresponding. Therefore, in the negative samples obtained in the above manner, it is very likely that the question statements and answer statements are semantically associated. Such negative samples are obviously invalid and will affect the training effect of the language model. In order to avoid the above problems, the following sentence filtering steps can be further performed.
[0057] 102. Filter out redundant statements in the first question and answer set to obtain a second question and answer set, where any question and answer pair in the second question and answer set includes a target question statement and a target answer statement.
[0058] The redundant statements in the first question and answer set may be redundant question statements and / or redundant answer statements. At this time, the first similarity between each question statement in the first question and answer set may be calculated first, and the second similarity between each answer statement in the first question and answer set may be calculated. Then, based on the first similarity, a plurality of similar question statements in the first question and answer set may be determined. Optionally, any one of the plurality of similar question statements may be randomly retained, and other question statements may be filtered out. Similarly, the answer statements may also be filtered out based on the second similarity, which means that the first step of filtering is achieved.
[0059] Furthermore, after the above filtering, it is very likely that the question statement or the answer statement in a question and answer pair will be filtered out separately. Therefore, after the first filtering step, the remaining statements in the first question and answer set can also be filtered out in the second step, that is, the question statements or answer statements whose identifiers cannot be matched are filtered out according to the question and answer relationship identifier, thereby completing the second filtering step and obtaining the second question and answer set. Among them, the question and answer relationship identifiers of each statement in the first question and answer set are preset, and the question statement and the answer statement in a question and answer pair have the same question and answer relationship identifier, and the same question and answer relationship identifier also indicates that the two statements have a preset question and answer relationship.
[0060] The above filtering process actually filters out semantically similar or identical sentences based on similarity, and then filters out isolated sentences that cannot constitute a question-answer pair based on sentence identifiers. The second question-answer set obtained after filtering is also composed of multiple question-answer pairs, and they can all be used as positive samples. Assuming that any question-answer pair in the second question-answer set consists of a target sentence and a target answer sentence, then in the second question-answer set, the target question sentence is only semantically associated with the target answer sentence, and has no semantic association with the non-target answer sentence, which means that in the second question-answer set, a strict one-to-one semantic correspondence between the question sentence and the answer sentence is achieved.
[0061] 103. Determine a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set.
[0062] Next, similar to the description in step 101, a third question and answer set can be further generated based on the second question and answer set. The question and answer pairs contained in the third question and answer set can be composed of target sentences and non-target answer sentences in the second question and answer set. And through the filtering process in step 102, the question sentence and answer sentence contained in any question and answer pair in the third question and answer set are semantically unrelated, and any question and answer pair in the third question and answer set is a valid negative sample.
[0063] 104. Train a language model based on the second question and answer set and the third question and answer set.
[0064] Finally, the second Q&A set can be used as positive samples, and the third Q&A set as negative samples for language model training. This combined use of positive and negative samples allows the language model to learn the semantic relationship between each question and answer in the second Q&A set, effectively implementing contrastive learning and ensuring effective model training.
[0065] In this embodiment, a first set of questions and answers is obtained, and redundant statements in the first set of questions and answers are filtered out to obtain a second set of questions and answers. The target question statements and non-target answer statements in the second set of questions and answers are then combined to form a third set of questions and answers. Finally, the second set of questions and answers is used as positive samples, and the third set of questions and answers is used as negative samples for language model training.
[0066] In this method, redundant sentence filtering ensures a one-to-one semantic correspondence between the question and answer in the second Q&A set, while mismatching the question and answer in the third Q&A set. This results in a more accurate delineation of positive and negative samples. Using accurately delineated positive and negative samples for training enables the language model to learn the semantic connection between question and answer, ensuring effective model training and further enhancing the fluency of human-computer dialogue.
[0067] The description of the above embodiment has mentioned that the overall idea of the model training method can be a two-stage training combined with contrastive learning. Figure 2 The embodiment shown describes the training process of the second stage. Figure 2 Based on the embodiment shown, Figure 3 A flowchart of another model training method provided by an embodiment of the present invention, such as Figure 3 As shown, the method may include the following steps:
[0068] 201. Obtain a first question and answer set, where any question and answer set in the first question and answer set includes a target question statement and a target answer statement.
[0069] 202. Determine a fourth question and answer set based on the target question sentence and the non-target answer sentence in the first question and answer set.
[0070] 203. Perform model training based on the first question and answer set and the fourth question and answer set to obtain a first language model.
[0071] For the first question and answer set obtained, any question and answer pair therein is composed of a target question statement and a target answer statement, and the two are semantically related. At this time, the target question statement in the first question and answer set and the non-answer statement in the first question and answer set can also be reconstructed into multiple question and answer pairs, and these reconstructed question and answer pairs form a fourth question and answer set. The question statements and answer statements in the question and answer pairs contained in the fourth question and answer set are not semantically related. The first question and answer set can be considered as a positive sample, and the fourth question and answer set can be considered as a negative sample. At this time, the positive and negative samples can be used to perform the first stage of model training to obtain the first language model.
[0072] The above steps 201 to 203 can be understood in conjunction with the description of step 101.
[0073] 204. Filter out redundant statements in the first question and answer set to obtain a second question and answer set, where any question and answer pair in the second question and answer set includes a target question statement and a target answer statement.
[0074] Next, redundant statements can be filtered out. Figure 2 In the embodiment shown, the filtering of sentences can be achieved using the first similarity and the second similarity, and the similarity can be calculated in the following way:
[0075] The first question and answer set is input into the first language model obtained after the first stage of training, so that the question sentences and answer sentences in the first question and answer set are converted into corresponding sentence vectors by the first language model. Then, the first similarity between the sentence vectors corresponding to the question sentences is calculated. Optionally, the first similarity can be expressed as a distance value or dot product between the sentence vectors. The smaller the distance value or the smaller the product, the higher the similarity. The second similarity between the sentence vectors corresponding to the answer sentences can also be specifically expressed as a distance value or dot product of the sentence vectors.
[0076] Optionally, because question sentences and answer sentences are semantically different, in practical applications, sentence vector conversion can typically be performed by a first conversion network in the first language model to convert question sentences in the first question and answer set, and by a second conversion network in the first language model to convert answer sentences in the first question and answer set. Optionally, the first conversion network and the second conversion network can be convolutional neural networks.
[0077] After the first step of filtering the sentences using the first and second conversion networks is completed, a second step of filtering can be further performed based on the question-answer relationship identifiers of the sentences. The specific process of the second step of filtering can be found in the relevant description of step 102 above and will not be repeated here.
[0078] After the above two steps of filtering, at least one target question-answer pair can remain in the first question-answer set. Each target question-answer pair has a preset question-answer relationship, that is, the question statement and answer statement in each target question-answer pair have the same question-answer relationship identifier, and the two are semantically related. Figure 2 In the illustrated embodiment, the second question-answer set is directly composed of at least one target question-answer pair to complete the training of the language model.
[0079] However, given that the question-answer relationship identifiers for each sentence in the target question-answer pair are manually set during the sentence collection process, there may be some deviation. This means that the semantic relationship between the question and the answer in the target question-answer pair may not be accurate, and the pre-defined question-answer relationship between the question and the answer may not hold. Therefore, directly using the target question-answer pair as positive samples for model training will obviously affect the model training effect.
[0080] To avoid the above problems, the preset question-answer relationship of the target question-answer pair can optionally be verified with the help of the first language model: the target question-answer pair is input into the first language model, and the classification network in the first language model is used to classify whether the preset question-answer relationship of the target question-answer pair is established. According to the confidence of the classification result output by the classification network, if the confidence is lower than the preset threshold, it indicates that the preset question-answer relationship of the target question-answer pair is not established, and it is filtered out. The remaining question-answer pairs after this filtering process are used as the second question-answer set. This process can also be considered as the third step of filtering the sentence.
[0081] It should also be noted that to ensure the effectiveness of model training, the number of question-answer pairs in the second question-answer set must be greater than or equal to a preset number. Therefore, in actual applications, the number of question-answer pairs in the first question-answer set can be N times that in the second question-answer set, where N ≥ 5.
[0082] 205. Determine a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set.
[0083] 206. Train the first language model based on the second question and answer set and the third question and answer set to obtain a second language model.
[0084] Finally, the second question and answer set is used as a positive sample, and the third answer set is used as a negative sample. The second stage training is performed on the basis of the first language model obtained after step 203 to obtain a second language model. The execution process of the above steps 205 to 206 is similar to the corresponding steps of the above embodiment, which can be seen in FIG. Figure 2 The relevant descriptions in the illustrated embodiments will not be repeated here.
[0085] In this embodiment, the first question and answer set is first used as a positive sample, and the fourth sample set is used as a negative sample to perform the first stage of comparative training to obtain a first language model. The combined use of positive and negative samples can enable the language model to learn the semantic association relationship between each question statement and each answer statement in the first question and answer set. Then, the first question and answer set is filtered out of redundant statements based on the semantics and question and answer relationship identifiers to obtain a second question and answer set, which is used as a positive sample, and the third question and answer set is used as a negative sample to perform the second stage of comparative training to obtain a second language model. The combined use of positive and negative samples can also enable the language model to learn the semantic association relationship between each question statement and each answer statement in the second question and answer set. The two-stage comparative learning can ensure the training effect of the model.
[0086] Figure 3 The structure of the language model in the embodiment shown and the training process of the first training stage can be as follows: Figure 4 This language model can also be applied to Figure 2In the illustrated embodiment, during the first stage of training, the first conversion network, the second conversion network, and the classification network included in the first language model can be trained together when executing step 203 .
[0087] Optionally, the first conversion network and the second conversion network can be trained simultaneously based on the first question and answer set and the fourth question and answer set. It should be noted that the network parameters of the first conversion network and the second conversion network are independent and not shared.
[0088] For the training of the two conversion networks, assuming that there are a total of N question statements and N answer statements in the first question and answer set and the fourth question and answer set, the first conversion network can output the sentence vectors of the N question statements, and the second conversion network can output the sentence vectors of the N answer statements.
[0089] Calculate the similarities between the sentence vectors corresponding to the N question statements and the sentence vectors corresponding to the N question-answer statements respectively, thereby obtaining N*N similarities. At this time, the network parameters of the first conversion network and the second conversion network can be adjusted according to the N*N similarities. Among them, the principle of adjusting the parameters is to make the similarity between the question statement Q1 and the answer statement A1 greater than the similarity between the question statement Q1 and the other answer statements. Among them, the question statement Q1 and the answer statement A1 constitute a positive sample, and the two can be a question-answer pair in the first question-answer set. The question statement Q1 and the other answer statements constitute a negative sample, and the two are a question-answer pair in the fourth question-answer set.
[0090] For training the classification network, since the sentences in the first and fourth question-answer sets are pre-assigned question-answer relationship identifiers, which reflect the pre-assigned question-answer relationship between the question and the answer, the pre-assigned question-answer relationship can be used as supervisory information to train the classification network. The network parameters of the classification network are then adjusted based on the difference between the pre-assigned question-answer relationship and the predicted question-answer relationship output by the classification network.
[0091] The training process of the above conversion network and classification network can occur in the first training stage, such as Figure 3 In the illustrated embodiment, the conversion network and classification network obtained after the first training phase can be used to further implement the second phase of training. Furthermore, during the second phase of training, the network parameters of the classification network are fixed, and only the first conversion network and the second conversion network are further trained.
[0092] Specifically, the second and third question-and-answer sets are input into the first language model, with the first conversion network in the first language model outputting sentence vectors for the question sentences, and the second conversion network outputting sentence vectors for the answer sentences. Based on the similarity between the sentence vectors for the question sentences and the answer sentences, the network parameters of the first and second conversion networks are adjusted to produce the second language model. The principles for adjusting network parameters are the same as in the first stage of training, and can be found in the description above.
[0093] In summary of the above embodiments, one method is to directly filter out redundant sentences after obtaining the first question and answer set, thereby obtaining a second question and answer set as a positive sample, and then generate a third question and answer set as a negative sample based on the second question and answer set. After sentence filtering, it can be guaranteed that the question sentences and answer sentences in the second question and answer set are strictly one-to-one corresponding in semantics, and it can also be guaranteed that the question sentences and answer sentences in the third question and answer set are semantically unrelated, ensuring the validity of the negative sample. Finally, the language model is trained by contrastive learning using positive and negative samples to ensure the training effect. The above process is also called Figure 2 The embodiment shown provides the method.
[0094] To ensure efficient and effective model training, another approach is to obtain a first set of questions and answers as positive samples and then generate a fourth set of questions and answers as negative samples. The first and fourth sets of questions and answers are then used to perform the first phase of model training using contrastive learning to obtain a first language model comprising a first conversion network, a second conversion network, and a classification network.
[0095] Based on the first language model, two conversion networks are used to convert the question sentences and answer sentences in the first question and answer set into sentence vectors respectively, and the first step of filtering out redundant sentences is achieved based on the sentence vectors. The second step of filtering out sentences is achieved based on the question-answer relationship identifier of the sentence, and then the third step of filtering out sentences is achieved based on the classification network in the first language model, thereby obtaining a second question and answer set as a positive sample. A third question and answer set is generated as a negative sample based on the second question and answer set. Finally, the second stage of training of the two conversion networks in the first language model is carried out based on the second and third question and answer sets to obtain a second language model. In particular, during the second stage of training, the classification network is fixed and does not participate in the training.
[0096] It should be noted that the classification network in the language model is used to verify whether the preset question-answer relationship of the question-answer pair is established, and the second conversion network is used to convert the sentence vector of the response sentence. Therefore, in the actual use of the language model, the first conversion network in the language model is used to convert the sentence vector of the question sentence input by the user. The second conversion network and classification network are not used, so these two networks can be removed.
[0097] For ease of understanding, the specific implementation of the model training method provided above is exemplified in combination with the following application scenarios.
[0098] Taking a bank in a public service scenario as an example, the first question-answer set obtained may include the following question-answer pairs:
[0099] Question Q1: What do you like? Answer A1: I like watching movies
[0100] Question Q2: Do you like watching movies? Answer A2: I like watching movies
[0101] Question statement Q3: What financial products are recommended? Answer statement A3: M financial products have high returns, I recommend buying them
[0102] Question Q4: What's the weather like today? Answer A2: Blue sky and white clouds
[0103] And the question-answer relationship identifier set between Q1 and A1 in the first question-answer set can be C1, the question-answer relationship identifier set between Q2 and A2 can be C2, and so on.
[0104] Based on the above-mentioned first question and answer set, a fourth question and answer set is generated, which includes: question and answer pairs consisting of Q1 and A2~A4 respectively, question and answer pairs consisting of Q2 and A1, A3, and A4 respectively, question and answer pairs consisting of Q3 and A1, A2, and A4 respectively, and question and answer pairs consisting of Q4 and A1~A3 respectively.
[0105] The first stage of model training is carried out using the above two question-answer sets. The first stage of training can be as follows: Figure 5a As shown. At this point, the similarities between the four sentence vectors output by the first conversion network and the second conversion network can be calculated, and the network parameters of the two conversion networks can be adjusted based on the 16 similarity values, so that the first conversion network and the second conversion network in the first language model have a certain sentence vector conversion capability. At the same time, the network parameters of the classification network can be adjusted based on the predicted classification results output by the classification network in the first language model, so that the classification network can accurately distinguish whether the question sentence and the answer sentence in a question-answer pair are semantically related.
[0106] Then, the second stage of training can be started: the sentence vectors output by the first and second conversion networks can be used to randomly filter out the answer sentence A1 in the first question and answer set, and then Q1 can be filtered out according to the question and answer relationship identifier of the sentence. Optionally, the classification network in the first language model can also be used to verify the preset question and answer relationship of the remaining three question and answer pairs. If the classification model outputs that the preset question and answer relationship between Q4 and A4 does not hold, it will also be filtered out. The above filtering process can be as follows Figure 5b shown
[0107] After the above filtering, Q2 and A2 as well as Q3 and A3 in the first question and answer set will be used as positive samples to form the second question and answer set. Q2 and A3 as well as Q3 and A2 will be used as negative samples to form the third question and answer set, and the first language model will be trained in the second stage. In the second training process, the classification network needs to be fixed, and the parameters of the conversion network are adjusted according to the similarity between the two sentence vectors output by the two conversion networks in the first language model to obtain the second language model. This process can be combined with Figure 5c shown.
[0108] If the traditional training method is used to train the language model, then in a shopping mall scenario, when the user inputs the question sentence "Who are you?", the dialogue system containing the language model will output the response sentence "Who are you talking about?" for the user. However, after using the embodiment of the present invention, based on the second language model, the response sentence output by the dialogue system is "I am a robot, please give me your advice." The response sentence is more logical and the dialogue is smooth.
[0109] In addition, taking the shopping mall scenario as an example, the first question and answer set obtained may include the following question and answer pairs:
[0110] Question Q1: What are some good movies? Answer A1: I don’t like watching movies
[0111] Question statement Q2: Does store M have any discount activities? Answer statement A2: Store M has a 20% discount on all items today.
[0112] Question Q3: What clothes are on sale? Answer A3: Store M has a 20% discount on all items today. Question Q4: What's the weather like today? Answer A2: Blue sky and white clouds
[0113] Similar to the above scenario, a fourth question-answer set can also be obtained, and the first stage of training can be completed using the two question-answer sets to obtain the first language model.
[0114] Sentence A2 can be randomly filtered out according to the similarity, sentence Q2 can be filtered out according to the question-answer relationship identifier, and sentences Q4 and A4 can be filtered out according to the classification results output by the classification network.
[0115] Thus, the remaining sentences Q1 and A1 and Q3 and A3 constitute the second question and answer set, and then Q1 and A3 and Q3 and A1 constitute the third question and answer set, and the second stage of training is carried out to obtain the second language model.
[0116] If the traditional training method is used to train the language model, then in a shopping mall scenario, when the user inputs the question sentence "Tell a joke", the dialogue system containing the language model will output the response sentence "Let's go in and talk?". However, after using the embodiment of the present invention, based on the second language model, the response sentence output by the dialogue system is "While the sun is shining on you, you are also basking in the sun."
[0117] The training process in this scenario is similar to Figures 5a to 5c The scenarios shown are similar, so the training diagram is not shown.
[0118] The following describes in detail the model training device of one or more embodiments of the present invention. Those skilled in the art will appreciate that these model training devices can be constructed by configuring commercially available hardware components through the steps taught in this solution.
[0119] Figure 6 A structural diagram of a model training device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the device includes:
[0120] The acquisition module 11 is used to acquire a first question and answer set.
[0121] The filtering module 12 is used to filter out redundant sentences in the first question and answer set to obtain a second question and answer set, wherein any question and answer pair in the second question and answer set includes a target question sentence and a target answer sentence.
[0122] The set determination module 13 is used to determine a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set.
[0123] The training module 14 is configured to train a language model based on the second question and answer set and the third question and answer set.
[0124] Optionally, any question and answer set in the first question and answer set includes a target question sentence and a target answer sentence;
[0125] The set determination module 13 is further configured to determine a fourth question and answer set based on the target question sentences and non-target answer sentences in the first question and answer set.
[0126] The training module 14 is further used to perform model training based on the first question and answer set and the fourth question and answer set to obtain a first language model; and to train the first language model based on the second question and answer set and the third question and answer set to obtain a second language model.
[0127] Optionally, the filtering module 12 is specifically configured to:
[0128] According to the first similarity between the question statements in the first question and answer set, redundant question statements in the first question and answer set are filtered out; according to the second similarity between the answer statements in the first question and answer set, redundant answer statements in the first question and answer set are filtered out; and according to the question and answer relationship identifiers of the remaining statements in the first question and answer set, the remaining statements are collected and filtered out to obtain a second question and answer set.
[0129] Optionally, the device further comprises:
[0130] Input module 21, configured to input the first question and answer set into the first language model, so that the first language model converts the question sentences and answer sentences in the first question and answer set into sentence vectors respectively.
[0131] The similarity determination module 22 is configured to determine the first similarity based on the sentence vectors corresponding to the question sentences; and to determine the second similarity based on the sentence vectors corresponding to the response sentences.
[0132] Optionally, the first language model is configured to perform vector conversion on the question statement by a first conversion network in the first language model; and to perform vector conversion on the response statement by a second conversion network in the first language model.
[0133] Optionally, the filtering module 12 is specifically used to: determine a target question-answer pair with a preset question-answer relationship in the remaining sentences based on the question-answer relationship identifier; input the target question-answer pair into the first language model, so that the classification network in the first language model classifies whether the preset question-answer relationship of the target question-answer pair is established; and filter out the target question-answer pairs for which the preset question-answer relationship is not established based on the confidence of the classification result output by the classification network.
[0134] Optionally, the training module 14 is specifically configured to perform model training based on the first question and answer set and the fourth question and answer set to obtain a first conversion network and a second conversion network in the first language model.
[0135] Optionally, the training module 14 is further configured to: input the second question-answer set and the third question-answer set into the first language model, so that the first conversion network in the first language model outputs a sentence vector corresponding to the question sentence, and the second conversion network outputs a sentence vector corresponding to the answer sentence;
[0136] According to the similarity between the sentence vector corresponding to the question sentence and the sentence vector corresponding to the answer sentence, the network parameters of the first conversion network and the second conversion network are adjusted to obtain the second language model.
[0137] Optionally, the training module 14 is further specifically used to: perform model training according to a preset question-answer relationship between question sentences and answer sentences in the first question-answer set to obtain a classification network in the first language model.
[0138] Figure 6 The device shown can perform the aforementioned Figures 1 to 4 The model training method provided in the embodiment shown in the figure is not described in detail in this embodiment. Figures 1 to 4 The relevant description of the illustrated embodiment will not be repeated here.
[0139] The above describes the internal functions and structure of the model training device. In a possible design, the structure of the model training device can be implemented as an electronic device, such as Figure 7 As shown, the electronic device may include: a processor 31 and a memory 32. The memory 32 is used to store the data that supports the electronic device to execute the aforementioned Figures 1 to 4 The program of the model training method provided in the illustrated embodiment, the processor 31 is configured to execute the program stored in the memory 32.
[0140] The program includes one or more computer instructions, wherein the one or more computer instructions, when executed by the processor 31, can implement the following steps:
[0141] Optionally, the processor 31 is further configured to execute the aforementioned Figures 1 to 4 All or part of the steps in the illustrated embodiments.
[0142] Get the first question and answer set;
[0143] Filtering out redundant statements in the first question and answer set to obtain a second question and answer set, wherein any question and answer pair in the second question and answer set includes a target question statement and a target answer statement;
[0144] Determining a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set;
[0145] A language model is trained based on the second question and answer set and the third question and answer set.
[0146] The structure of the electronic device may further include a communication interface 33 for communicating with other devices or a communication network.
[0147] In addition, an embodiment of the present invention provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to perform at least the following actions:
[0148] Get the first question and answer set;
[0149] Filtering out redundant statements in the first question and answer set to obtain a second question and answer set, wherein any question and answer pair in the second question and answer set includes a target question statement and a target answer statement;
[0150] Determining a third question and answer set based on the target question sentences and non-target answer sentences in the second question and answer set;
[0151] A language model is trained based on the second question and answer set and the third question and answer set.
[0152] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.
[0153] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by adding a necessary general-purpose hardware platform, or of course, can be implemented by combining hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the existing technology, can be embodied in the form of a computer product.
[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A model training method, characterized in that: include: Acquire a first question and answer set, wherein the first question and answer set includes semantically similar target question sentences and redundant question sentences, and semantically similar target answer sentences and redundant answer sentences; filtering out redundant question statements in the first question and answer set according to a first similarity between question statements in the first question and answer set; filtering out redundant answer statements in the first question and answer set according to a second similarity between answer statements in the first question and answer set; According to the question-answer relationship identifiers of the remaining sentences in the first question-answer set, the remaining sentences are filtered out to obtain a second question-answer set, wherein any question-answer pair in the second question-answer set includes a target question sentence and a target answer sentence that are semantically one-to-one associated; Determining the target question sentences and non-target answer sentences that have no semantic association in the second question and answer set as a third question and answer set; A language model is trained based on the second question and answer set and the third question and answer set.
2. The method according to claim 1, characterized in that Any question and answer set in the first question and answer set includes a target question sentence and a target answer sentence; Before filtering out the remaining sentences according to the question-answer relationship identifiers of the remaining sentences in the first question-answer set, the method further includes: Determine a fourth question and answer set according to the target question sentence and the non-target answer sentence in the first question and answer set; Performing model training according to the first question and answer set and the fourth question and answer set to obtain a first language model; The step of training a language model according to the second question and answer set and the third question and answer set includes: The first language model is trained according to the second question and answer set and the third question and answer set to obtain a second language model.
3. The method according to claim 2, characterized in that The method further comprises: Inputting the first question and answer set into the first language model, so that the first language model converts question sentences and answer sentences in the first question and answer set into sentence vectors respectively; Determining the first similarity according to the sentence vectors corresponding to the question sentences; The second similarity is determined according to the sentence vectors corresponding to each of the response sentences.
4. The method according to claim 3, characterized in that The first language model converting the question sentences and the answer sentences in the first question and answer set into sentence vectors respectively includes: Performing vector conversion on the question sentence by a first conversion network in the first language model; The response sentence is vectorized by a second conversion network in the first language model.
5. The method according to claim 3 or 4, characterized in that: The filtering out the remaining sentences according to the question-answer relationship identifiers of the remaining sentences in the first question-answer set includes: Determining, according to the question-answer relationship identifier, a target question-answer pair having a preset question-answer relationship in the remaining sentences; Inputting the target question-answer pair into the first language model, so that a classification network in the first language model classifies whether a preset question-answer relationship of the target question-answer pair is established; According to the confidence of the classification result output by the classification network, the target question-answer pairs for which the preset question-answer relationship does not hold are filtered out.
6. The method according to claim 4, characterized in that The performing model training according to the first question and answer set and the fourth question and answer set to obtain a first language model includes: Model training is performed according to the first question and answer set and the fourth question and answer set to obtain a first conversion network and a second conversion network in the first language model.
7. The method according to claim 4, characterized in that The training of the first language model according to the second question and answer set and the third question and answer set to obtain a second language model includes: Inputting the second question-answer set and the third question-answer set into the first language model, so that the first conversion network in the first language model outputs a sentence vector corresponding to the question sentence, and the second conversion network outputs a sentence vector corresponding to the answer sentence; According to the similarity between the sentence vector corresponding to the question sentence and the sentence vector corresponding to the answer sentence, the network parameters of the first conversion network and the second conversion network are adjusted to obtain the second language model.
8. The method according to claim 5, characterized in that Before filtering out the remaining sentences according to the question-answer relationship identifiers of the remaining sentences in the first question-answer set, the method further includes: Model training is performed according to a preset question-answer relationship between question sentences and answer sentences in the first question-answer set to obtain a classification network in the first language model.
9. A model training device, characterized in that: include: An acquisition module, configured to acquire a first question and answer set, wherein the first question and answer set includes a semantically similar target question sentence and a redundant question sentence, and includes a semantically similar target answer sentence and a redundant answer sentence; A filtering module, configured to filter out redundant question statements in the first question and answer set according to a first similarity between question statements in the first question and answer set; filter out redundant answer statements in the first question and answer set according to a second similarity between answer statements in the first question and answer set; and filter out the remaining statements in the first question and answer set according to question and answer relationship identifiers of the remaining statements to obtain a second question and answer set, wherein any question and answer pair in the second question and answer set includes a target question statement and a target answer statement that are semantically one-to-one associated; A set determination module, configured to determine the target question sentences and non-target answer sentences that have no semantic association in the second question and answer set as a third question and answer set; A training module is used to train a language model based on the second question and answer set and the third question and answer set.
10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement: Acquire a first question and answer set, wherein the first question and answer set includes semantically similar target question sentences and redundant question sentences, and semantically similar target answer sentences and redundant answer sentences; filtering out redundant question statements in the first question and answer set according to a first similarity between question statements in the first question and answer set; filtering out redundant answer statements in the first question and answer set according to a second similarity between answer statements in the first question and answer set; According to the question-answer relationship identifiers of the remaining sentences in the first question-answer set, the remaining sentences are filtered out to obtain a second question-answer set, wherein any question-answer pair in the second question-answer set includes a target question sentence and a target answer sentence that are semantically one-to-one associated; Determining the target question sentences and non-target answer sentences that have no semantic association in the second question and answer set as a third question and answer set; A language model is trained based on the second question and answer set and the third question and answer set.
11. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to perform at least the following actions: Acquire a first question and answer set, wherein the first question and answer set includes semantically similar target question sentences and redundant question sentences, and semantically similar target answer sentences and redundant answer sentences; filtering out redundant question statements in the first question and answer set according to a first similarity between question statements in the first question and answer set; filtering out redundant answer statements in the first question and answer set according to a second similarity between answer statements in the first question and answer set; According to the question-answer relationship identifiers of the remaining sentences in the first question-answer set, the remaining sentences are filtered out to obtain a second question-answer set, wherein any question-answer pair in the second question-answer set includes a target question sentence and a target answer sentence that are semantically one-to-one associated; Determining the target question sentences and non-target answer sentences that have no semantic association in the second question and answer set as a third question and answer set; A language model is trained based on the second question and answer set and the third question and answer set.