Training sample expansion method and device, computer storage medium and electronic device

Abstract

The present disclosure relates to the technical field of data processing, and provides a training sample expansion method, a training sample expansion device, a computer storage medium and an electronic device. The training sample expansion method comprises: collecting corpus data, and extracting a candidate question from the corpus data; identifying, by using an updated domain detection model and based on first identification logic, an association relationship between the candidate question and a plurality of preset question labels, to obtain a first question label associated with the candidate question; identifying, by using the updated domain detection model and based on second identification logic, an association relationship between the candidate question and the plurality of preset question labels, to obtain a second question label associated with the candidate question; and if the first question label is consistent with the second question label, expanding a training sample corresponding to the updated domain detection model by using the candidate question. The present disclosure can improve the data quality of the training sample, thereby improving the question and answer quality of the intelligent question and answer model.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a training sample expansion method, a training sample expansion device, a computer storage medium, and an electronic device. Background Technology

[0002] Intelligent question-answering models use a question-and-answer format to accurately pinpoint the knowledge questions that website users need. By interacting with website users, they provide personalized information services. The effectiveness of intelligent question-answering models depends on the quality of the training data. Generally speaking, the higher the quality of the training data and the fewer the error data, the better the question-answering performance of the intelligent question-answering model.

[0003] Currently, the training samples for intelligent question-answering models are typically built and expanded by manually collecting knowledge points and questions under each knowledge point. However, this approach leads to a decline in the quality of the training sample data, thus affecting the question-answering performance of the intelligent question-answering model.

[0004] Therefore, there is an urgent need in this field to develop a new method and apparatus for expanding training samples.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure. Summary of the Invention

[0006] The purpose of this disclosure is to provide a training sample expansion method, a training sample expansion device, a computer storage medium, and an electronic device, thereby overcoming, to at least a certain extent, the technical problems in the related art that affect the question-answering performance of intelligent question-answering models due to low training sample quality.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part by practice of this disclosure.

[0008] According to a first aspect of this disclosure, a method for expanding training samples is provided, comprising: collecting corpus data and extracting candidate questions from the corpus data; identifying the association between the candidate questions and multiple preset question labels using an updated domain detection model based on a first recognition logic to obtain a first question label associated with the candidate questions; and identifying the association between the candidate questions and multiple preset question labels using the updated domain detection model based on a second recognition logic to obtain a second question label associated with the candidate questions; if the first question label and the second question label are consistent, expanding the training samples corresponding to the updated domain detection model using the candidate questions.

[0009] In an exemplary embodiment of this disclosure, the preset question tag includes the knowledge point tag or the question intent tag.

[0010] In an exemplary embodiment of this disclosure, the step of identifying the association between the candidate questions and multiple preset question labels based on the updated domain detection model and the first identification logic to obtain the first question label associated with the candidate questions includes: extracting the latent feature vectors corresponding to the candidate questions through the updated domain detection model; obtaining the latent feature vectors corresponding to each preset question label; calculating the first vector distance between the latent feature vectors corresponding to the candidate questions and the latent feature vectors corresponding to each preset question label; and determining the first question label associated with the candidate questions based on the multiple first vector distances.

[0011] In an exemplary embodiment of this disclosure, the updated domain detection model includes n network layers, where n is an integer greater than 1; the step of extracting the latent feature vector corresponding to the candidate problem using the updated domain detection model includes: pooling the output features of each of the n network layers to obtain the output vector corresponding to each network layer; calculating the similarity between the output vector corresponding to the nth network layer and the output vector corresponding to each network layer to obtain n similarities; using the n similarities as weighting coefficients for the output vectors corresponding to the n network layers, and weighting the n output vectors corresponding to the n network layers to obtain the latent feature vector corresponding to the candidate problem.

[0012] In an exemplary embodiment of this disclosure, determining the first question label associated with the candidate question based on a plurality of first vector distances includes: filtering out the smallest first vector distance from the plurality of first vector distances, and determining the preset question label corresponding to the smallest first vector distance as the candidate first question label associated with the candidate question; finding a specified distance threshold corresponding to the candidate first question label; if the smallest first vector distance is greater than the specified distance threshold, determining that there is no association between the candidate question and the candidate first question label; if the smallest first vector distance is not greater than the specified distance threshold, determining the candidate first question label as the first question label associated with the candidate question.

[0013] In an exemplary embodiment of this disclosure, after determining that there is no correlation between the sample question and the candidate first question label, the method further includes: discarding the candidate question.

[0014] In an exemplary embodiment of this disclosure, the step of identifying the association between the candidate question and multiple preset question labels based on the updated domain detection model using a second identification logic to obtain a second question label associated with the candidate question includes: extracting a CLS vector from the latent feature vector corresponding to the candidate question, wherein the CLS vector is used to characterize the correlation between the candidate question and each preset question label; and determining the second question label associated with the candidate question from the multiple preset question labels based on the CLS vector.

[0015] In an exemplary embodiment of this disclosure, the updated domain detection model is trained in the following manner: obtaining initial training samples, the initial training samples including multiple preset question labels, each preset question label being associated with multiple sample questions; iteratively training the domain detection model to be trained based on the initial training samples to obtain an initial domain detection model; using the initial domain detection model to clean the initial training samples to obtain cleaned training samples; using the cleaned training samples as the training samples, and iteratively training the initial domain detection model based on the training samples to obtain the updated domain detection model.

[0016] In an exemplary embodiment of this disclosure, the step of iteratively training the domain detection model to be trained based on the initial training samples to obtain an initial domain detection model includes: inputting the initial training samples into the domain detection model to be trained to obtain a predicted question label associated with each sample question output by the domain detection model to be trained; determining a loss value of the domain detection model to be trained based on the degree of difference between the predicted question label and a preset question label associated with the sample question, and the degree of difference between any two sample questions among the plurality of sample questions; and iteratively training the domain detection model to be trained based on the loss value to obtain the initial domain detection model.

[0017] In an exemplary embodiment of this disclosure, determining the loss value of the domain detection model to be trained based on the degree of difference between the predicted question label and the preset question label associated with the sample question, and the degree of difference between any two sample questions among the plurality of sample questions, includes: calculating a first type of loss value based on the degree of difference between the predicted question label and the preset question label associated with the sample question; and calculating a second type of loss value based on the degree of difference between any two sample questions among the plurality of sample questions; and determining the loss value of the domain detection model to be trained based on the first type of loss value and the second type of loss value.

[0018] In an exemplary embodiment of this disclosure, the step of using the initial domain detection model to clean the initial training samples to obtain cleaned training samples includes: using the initial domain detection model to obtain the latent feature vector corresponding to each sample question; determining the latent feature vector corresponding to each preset question label based on the latent feature vector corresponding to the target sample question associated with each preset question label; determining the third question label associated with each sample question based on the second vector distance between the latent feature vector corresponding to the sample question and the latent feature vector corresponding to each preset question label; and cleaning the initial training samples based on the consistency comparison result between the third question label and the preset question label associated with the target sample question to obtain cleaned training samples.

[0019] In an exemplary embodiment of this disclosure, the initial domain detection model includes n network layers, where n is an integer greater than 1; the step of obtaining the latent feature vector corresponding to each sample question using the initial domain detection model includes: pooling the output features of each of the n network layers to obtain the output vector corresponding to each network layer; calculating the similarity between the output vector corresponding to the nth network layer and the output vector corresponding to each network layer to obtain n similarities; using the n similarities as weighting coefficients for the output vectors corresponding to the n network layers, and weighting the n output vectors corresponding to the n network layers to obtain the latent feature vector corresponding to each sample question.

[0020] In an exemplary embodiment of this disclosure, determining the latent feature vector corresponding to each preset question label based on the latent feature vector corresponding to the target sample question associated with each preset question label includes: averaging the latent feature vectors corresponding to the target sample questions associated with each preset question label to obtain the latent feature vector corresponding to each preset question label.

[0021] In an exemplary embodiment of this disclosure, determining the third question label associated with each sample question based on the second vector distance between the latent feature vector corresponding to the sample question and the latent feature vector corresponding to each preset question label includes: filtering out the second vector distance with the smallest value from a plurality of second vector distances, and determining the preset question label corresponding to the second vector distance with the smallest value as the candidate third question label associated with the sample question; calculating a specified distance threshold corresponding to the candidate third question label, and storing the correspondence between the candidate third question label and the specified distance threshold; if the second vector distance with the smallest value is greater than the specified distance threshold, determining that there is no association between the sample question and the candidate third question label; if the second vector distance with the smallest value is not greater than the specified distance threshold, determining the candidate third question label as the third question label associated with the sample question.

[0022] In an exemplary embodiment of this disclosure, calculating the specified distance threshold corresponding to the candidate third question label includes: calculating multiple third vector distances between the latent feature vector corresponding to the candidate third question label and the latent feature vectors corresponding to other preset question labels; and calculating multiple fourth vector distances between the latent feature vectors corresponding to multiple sample questions associated with the candidate third question label and the latent feature vector corresponding to the candidate third question label; and determining the specified distance threshold corresponding to the candidate third question label based on the multiple third vector distances and the multiple fourth vector distances.

[0023] In an exemplary embodiment of this disclosure, determining the specified distance threshold corresponding to the candidate third question label based on the plurality of third vector distances and the plurality of fourth vector distances includes: filtering the third vector distance with the smallest value from the plurality of third vector distances, and filtering the fourth vector distance with the largest value from the plurality of fourth vector distances; obtaining the target association value of the third vector distance with the smallest value; and determining the larger value between the target association value and the fourth vector distance with the largest value as the specified distance threshold corresponding to the candidate third question label.

[0024] In an exemplary embodiment of this disclosure, the step of cleaning the initial training samples based on the consistency comparison result between the third question label and the preset question label associated with the sample question to obtain cleaned training samples includes: if the third question label is inconsistent with the preset question label associated with the sample question, then the sample question is removed from the initial training samples to obtain the cleaned training samples; if the third question label is consistent with the preset question label associated with the sample question, then a fourth question label associated with the sample question is identified based on the latent feature vector corresponding to each sample question; and the initial training samples are cleaned based on the consistency comparison result between the third question label and the fourth question label to obtain cleaned training samples.

[0025] In an exemplary embodiment of this disclosure, the step of identifying the fourth question label associated with the sample question based on the latent feature vector corresponding to each sample question includes: extracting a CLS vector from the latent feature vector corresponding to the sample question, wherein the CLS vector is used to characterize the correlation between the sample question and each of the preset question labels; and determining the fourth question label associated with the sample question from the plurality of preset question labels based on the CLS vector.

[0026] According to a second aspect of this disclosure, a training sample expansion device is provided, comprising: a data acquisition module for acquiring corpus data and extracting candidate questions from the corpus data; a first identification module for identifying the association between the candidate questions and multiple preset question labels based on a first identification logic using an updated domain detection model, thereby obtaining a first question label associated with the candidate questions; and a second identification module for identifying the association between the candidate questions and multiple preset question labels based on a second identification logic using the updated domain detection model, thereby obtaining a second question label associated with the candidate questions; and a sample expansion module for expanding the training samples corresponding to the updated domain detection model using the candidate questions if the first question label and the second question label are consistent.

[0027] According to a third aspect of this disclosure, a computer storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the training sample augmentation method described in the first aspect above.

[0028] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the training sample augmentation method described in the first aspect by executing the executable instructions.

[0029] As can be seen from the above technical solutions, the training sample augmentation method, training sample augmentation device, computer storage medium, and electronic device in the exemplary embodiments of this disclosure have at least the following advantages and positive effects:

[0030] In some embodiments of this disclosure, the technical solutions provide a novel approach to expanding training samples by collecting corpus data and extracting candidate questions from it. This allows for expansion not only by processing the training samples themselves but also by expanding the training samples based on the collected corpus data. Furthermore, the updated domain detection model identifies the association between candidate questions and multiple preset question labels using a first recognition logic to obtain a first question label associated with the candidate questions. Additionally, the updated domain detection model identifies the association between candidate questions and multiple preset question labels using a second recognition logic to obtain a second question label associated with the candidate questions. This approach accurately determines the association between candidate questions and multiple preset question labels through two recognition logics. On the other hand, if the first question label is consistent with the second question label, the training samples corresponding to the updated domain detection model can be expanded using alternative questions. This allows for a clear understanding of the relationship between the alternative questions and multiple preset question labels. Then, the training samples can be expanded based on the alternative questions. This solves the problem of confusion or erroneous data caused by manually dividing the relationship during sample expansion in related technologies, improves the data quality of the expanded training samples, and further enhances the question-answering effect of the intelligent question-answering system.

[0031] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0033] Figure 1 A flowchart illustrating the training sample augmentation method in an embodiment of this disclosure is shown.

[0034] Figure 2 This diagram illustrates the process of training an updated domain detection model in an embodiment of the present disclosure.

[0035] Figure 3This illustration shows a flowchart of how to iteratively train the domain detection model to be trained based on the initial training samples in an embodiment of this disclosure to obtain the initial domain detection model;

[0036] Figure 4 This illustration shows a flowchart of how to determine the loss value of the domain detection model to be trained based on the degree of difference between the predicted question label and the preset question label associated with the sample question in an embodiment of this disclosure.

[0037] Figure 5 This illustration shows a flowchart of how to use an initial domain detection model to clean the initial training samples and obtain the cleaned training samples in an embodiment of this disclosure.

[0038] Figure 6 This diagram illustrates how the initial domain detection model is used to obtain the latent feature vector corresponding to each sample question in an embodiment of this disclosure.

[0039] Figure 7 This diagram illustrates how the initial domain detection model is used to obtain the latent feature vector corresponding to each sample question in an embodiment of this disclosure.

[0040] Figure 8 This illustration shows a flowchart of how to determine the third question label associated with each sample question based on the second vector distance between the latent feature vector corresponding to each sample question and the latent feature vector corresponding to each preset question label in an embodiment of this disclosure;

[0041] Figure 9 This diagram illustrates how candidate third question labels associated with sample questions are determined in an embodiment of this disclosure;

[0042] Figure 10 This diagram illustrates how a specified distance threshold corresponding to a preset problem label A is determined in an embodiment of this disclosure.

[0043] Figure 11 This illustration shows a flowchart of how, in an embodiment of this disclosure, the initial training samples are cleaned based on the consistency comparison results between the third question label and the preset question label associated with the sample question, resulting in cleaned training samples.

[0044] Figure 12 This illustration shows a flowchart of how, in an embodiment of the present disclosure, the updated domain detection model identifies the association between candidate questions and multiple preset question labels based on the first identification logic, and obtains the first question label associated with the candidate question.

[0045] Figure 13 This illustration shows a flowchart of how to determine the first question label associated with a candidate question based on multiple first vector distances in an embodiment of this disclosure;

[0046] Figure 14 This diagram shows an overall schematic of the training sample augmentation method in an embodiment of this disclosure.

[0047] Figure 15 This diagram illustrates the structure of the training sample augmentation device in an exemplary embodiment of this disclosure.

[0048] Figure 16 A schematic diagram of the structure of an electronic device in an exemplary embodiment of this disclosure is shown. Detailed Implementation

[0049] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0050] The terms “a,” “an,” “the,” and “the” are used in this specification to indicate the presence of one or more elements / components / etc.; the terms “including” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first” and “second” are used only as markings and are not a limitation on the number of objects.

[0051] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0052] The intelligent question-answering system consists of two core modules: knowledge base response and intent response. The number of questions in each core module and the degree of confusion between the questions directly determine the question-answering effect of that core module.

[0053] The aforementioned knowledge base consists of multiple knowledge points, each with a corresponding answer and multiple questions. When answering questions in the knowledge base, the user's question is matched against the questions in the knowledge base, and the question with the highest matching degree is selected, returning the answer corresponding to that knowledge point. With the matching system remaining constant, the higher the richness of questions under a knowledge point and the lower the confusion (no semantic overlap with questions from other knowledge points), the better the response. However, because the questions under each knowledge point are usually collected manually, as the number of knowledge points increases, the boundaries for manually distinguishing the meaning of knowledge points become increasingly blurred, and the questions under a knowledge point gradually become more easily confused.

[0054] Intent-based response uses a set of intent corpora as training data (the training data typically consists of multiple intents, with multiple training corpora under each intent). An offline classification model is trained and then deployed to the system. When a user's question enters the intent module, the model classifies and identifies the user's question, determines the user's intent, and then provides a response. In this process, the intent model plays a crucial role, and its accuracy directly affects the response performance. The model's accuracy depends on the quality of the training data; higher quality training data with fewer errors results in better model performance. Therefore, only when the intents corresponding to the corpora are correct can the trained model achieve good intent recognition performance. However, current training data is generally manually labeled, which is costly and prone to errors.

[0055] Currently, common data augmentation schemes are based on expanding existing data. For example, they can use generative methods to generate a new question with the same semantics as the existing question, or use substitution methods to replace words in the existing question with synonyms to generate a new question with the same semantics.

[0056] While the above solutions can enrich the expression of questions, they cannot guarantee fluency and may result in text that does not conform to colloquial expressions. Furthermore, the problems of confusion and mislabeling in the existing data remain unresolved; in fact, expanding the data may further increase the number of confused and mislabeled samples, leading to lower data quality in the training samples and preventing significant improvement in the question-answering performance of the intelligent question-answering model.

[0057] In the embodiments of this disclosure, a training sample expansion method is first provided, which at least to some extent overcomes the defects in related technologies that affect the question-answering effect of intelligent question-answering models due to the low quality of training samples.

[0058] Figure 1The diagram shows a flowchart of a training sample augmentation method in an embodiment of this disclosure. The execution entity of the training sample augmentation method can be a server that performs augmentation processing on the training samples.

[0059] refer to Figure 1 The training sample augmentation method according to an embodiment of the present disclosure includes the following steps:

[0060] Step S110: Collect corpus data and extract alternative questions from the corpus data;

[0061] Step S120: Using the updated domain detection model, the association between candidate questions and multiple preset question labels is identified based on the first recognition logic, thereby obtaining the first question label associated with the candidate question; and,

[0062] Step S130: Based on the second recognition logic, the updated domain detection model identifies the association between candidate questions and multiple preset question labels, and obtains the second question label associated with the candidate question;

[0063] Step S140: If the first question label is consistent with the second question label, expand the training samples corresponding to the updated domain detection model using alternative questions.

[0064] exist Figure 1 The technical solution provided in the illustrated embodiment offers a novel approach to expanding training samples by collecting corpus data and extracting candidate questions from it. This allows for expansion beyond simply processing the training samples themselves; the training samples can also be expanded based on the collected corpus data. Furthermore, the updated domain detection model identifies the association between candidate questions and multiple preset question labels using a first recognition logic to obtain the first question label associated with the candidate question. Additionally, the updated domain detection model identifies the association between candidate questions and multiple preset question labels using a second recognition logic to obtain the second question label associated with the candidate question. This allows for accurate determination of the association between candidate questions and multiple preset question labels through these two recognition logics. On the other hand, if the first question label is consistent with the second question label, the training samples corresponding to the updated domain detection model can be expanded using alternative questions. This allows for a clear understanding of the relationship between the alternative questions and multiple preset question labels. Then, the training samples can be expanded based on the alternative questions. This solves the problem of confusion or erroneous data caused by manually dividing the relationship during sample expansion in related technologies, improves the data quality of the expanded training samples, and further enhances the question-answering effect of the intelligent question-answering system.

[0065] The following are Figure 1 The specific implementation process of each step in the process will be explained in detail:

[0066] Before step S110, it should be noted that this disclosure can first train an updated domain detection model. The function of the updated domain detection model is to identify the first question label and the second question label associated with the question through two recognition logics.

[0067] refer to Figure 2 , Figure 2 This diagram illustrates the process of training an updated domain detection model in an embodiment of the present disclosure, including steps S201-S204:

[0068] In step S201, the initial training samples are obtained.

[0069] In this step, the initial training samples can be data contained in existing knowledge bases used for intelligent question answering, or they can be existing training corpora used for intent response.

[0070] The initial training samples mentioned above include multiple preset question labels, and each preset question label is associated with multiple sample questions.

[0071] For example, in the form of knowledge base responses, the initial training samples can contain multiple knowledge point tags (i.e., preset question tags), and each knowledge point tag is associated with multiple sample questions. For example, the training samples contain knowledge point tags A, B, and C, and knowledge point tag A is associated with sample questions A1, A2, and A3, knowledge point tag B is associated with sample questions B1, B2, B3, and B4, and knowledge point tag C is associated with sample questions C1 and C2.

[0072] Regarding the form of intent response, the initial training samples can contain multiple question intent labels (i.e., preset question labels). Each question intent label is associated with multiple sample questions. For example, the training samples contain question intent labels A, B, and C. Question intent label A is associated with sample questions A1, A2, and A3; question intent label B is associated with sample questions B1, B2, B3, and B4; and question intent label C is associated with sample questions C1 and C2.

[0073] After obtaining the initial training samples, you can proceed to step S202 to iteratively train the domain detection model to be trained based on the initial training samples to obtain the initial domain detection model.

[0074] In this step, refer to Figure 3 , Figure 3This embodiment of the present disclosure illustrates how to iteratively train the domain detection model to be trained based on initial training samples to obtain the initial domain detection model, including steps S301-S303:

[0075] In step S301, the initial training samples are input into the domain detection model to be trained to obtain the predicted question label associated with each sample question output by the domain detection model to be trained.

[0076] In this step, the initial training samples can be input into the domain detection model to be trained. Based on the output of the domain detection model, the predicted question label associated with each sample question is obtained. The domain detection model to be trained can be a BERT (Bidirectional Encoder Representations from Transformers) model, which can be set according to the actual situation; this disclosure does not impose any special limitations on it.

[0077] For example, the predicted question label associated with the sample question can be the knowledge point label to which the sample question belongs, or the questioning intent label corresponding to the sample question.

[0078] In step S302, the loss value of the domain detection model to be trained is determined based on the degree of difference between the predicted question label and the preset question label associated with the sample question, and the degree of difference between any two sample questions among the multiple sample questions.

[0079] In this step, refer to Figure 4 , Figure 4 This illustration shows a flowchart of how to determine the loss value of the domain detection model to be trained based on the degree of difference between the predicted question label and the preset question label associated with the sample question in an embodiment of this disclosure, including steps S401-S403:

[0080] In step S401, the first type of loss value is calculated based on the degree of difference between the predicted question label and the preset question label associated with the sample question.

[0081] In this step, the first type of loss value can be the cross-entropy loss value (CE), which represents the difference between the true sample label and the predicted sample label.

[0082] In step S402, the second type of loss value is calculated based on the degree of difference between any two sample problems among the multiple sample problems.

[0083] In this step, the second type of loss value can be the contrastive loss (CL). Contrastive loss is a binary classification loss function, which is usually used to measure the similarity between two samples. It determines whether the two samples belong to the same category based on the similarity. If they belong to the same category, the contrastive loss is 0. If they belong to different categories, the contrastive loss is a positive number.

[0084] In step S403, the loss value of the domain detection model to be trained is determined based on the first type of loss value and the second type of loss value.

[0085] In this step, the cross-entropy loss value and the contrastive loss value can be summed, and the summed value can be used as the loss value of the domain detection model to be trained.

[0086] Next, refer to Figure 3 In step S303, the domain detection model to be trained is iteratively trained according to the loss value to obtain the initial domain detection model.

[0087] In this step, the domain detection model to be trained can be iteratively trained based on the initial training samples mentioned above until the loss value of the domain detection model to be trained tends to converge, thus obtaining the initial domain detection model.

[0088] After obtaining the initial domain detection model, since the initial domain detection model is trained based on the initial training samples, and the data in the initial training samples is manually labeled, there may be some erroneous or confusing data that mixes up the questions with their respective labels, affecting the question answering quality of the intelligent question answering model. Therefore, the initial detection model can be used to clean the initial training samples to improve the sample quality.

[0089] Next, refer to Figure 2 In step S203, the initial training samples are cleaned using the initial neighborhood detection model to obtain cleaned training samples.

[0090] In this step, refer to Figure 5 , Figure 5 This illustration shows a flowchart of how to use an initial domain detection model to clean the initial training samples and obtain cleaned training samples in an embodiment of this disclosure, including steps S501-S503:

[0091] In step S501, the latent feature vector corresponding to each sample problem is obtained using the initial neighborhood detection model.

[0092] In this step, the initial domain detection model described above can contain multiple network layers, thus, referring to Figure 6 , Figure 6This embodiment of the present disclosure illustrates how to obtain the latent feature vector corresponding to each sample question using an initial domain detection model, including steps S601-S603:

[0093] In step S601, the output features of each network layer in the multiple network layers are pooled to obtain the output vector corresponding to each network layer.

[0094] In this step, each network layer of the initial detection model encodes the sample problem, resulting in a corresponding output feature for each network layer. Then, mean pooling is performed on the output features of each network layer to obtain the output vector h for each layer. j (j = 1, 2, 3...n).

[0095] In step S602, the similarity between the output vector corresponding to the multiple network layers and the output vector corresponding to each network layer is calculated to obtain multiple similarity values.

[0096] In this step, the aforementioned similarity can be cosine similarity. Therefore, after obtaining the output vector corresponding to each network layer, the output vector h corresponding to the multiple network layers can be calculated based on the following formula 1. n The output vector h corresponding to each network layer j Cosine similarity s between j :

[0097]

[0098] It should be noted that after obtaining the above cosine similarity s j Then, the cosine similarity s above can be calculated based on the following formula 2. j Normalization is performed:

[0099]

[0100] Where, α j denoted by cosine similarity after normalization, and e represents the natural logarithm.

[0101] In step S603, multiple similarities are used as weighting coefficients for the output vectors corresponding to multiple network layers. The multiple output vectors corresponding to multiple network layers are weighted to obtain the latent feature vector corresponding to each sample problem.

[0102] In this step, after obtaining the aforementioned similarities, these similarities can be used as weighting coefficients for the output vectors corresponding to multiple network layers. Then, the multiple output vectors are weighted according to these weighting coefficients to obtain the latent feature vector corresponding to each sample problem. (i is a positive integer from 1 to the total number of the above sample questions). Specifically, refer to Formula 3 below:

[0103]

[0104] refer to Figure 7 , Figure 7 This illustration shows how the initial domain detection model is used to obtain the latent feature vector corresponding to each sample question in an embodiment of this disclosure, such as... Figure 7 As shown:

[0105] First, multiple sample questions included under each preset question label (for example, sample questions under the "Refund" label could be: "I don't want it anymore, how do I refuse a refund?"; "How do I get a refund after refusing delivery?"; sample questions under the "Return" label could be: "Give me the return address?"; "Do you have a return address?") can be input into the initial domain detection model mentioned above. Thus, the multiple network layers (Layer 1 to Layer 2) of the initial domain detection model are then... n Each network layer can output its corresponding output features. For the output features corresponding to each network layer, mean pooling can be performed to obtain the latent feature vectors corresponding to multiple sample problems.

[0106] After obtaining the latent feature vectors corresponding to multiple sample questions in the training samples, we can proceed to step S502, where we determine the latent feature vector corresponding to each preset question label based on the latent feature vectors corresponding to the target sample questions associated with each preset question label.

[0107] In this step, the latent feature vector corresponding to the target sample question associated with each preset question label can be determined. Then, the latent feature vectors corresponding to multiple sample questions are averaged to obtain the latent feature vector corresponding to each preset question label. For example, referring to the relevant explanation of step S201 above, taking knowledge point label A as an example, when determining the latent feature vector corresponding to the knowledge point label A, the values ​​of each dimension of the latent feature vectors corresponding to sample question A1, sample question A2, and sample question A3 can be averaged to obtain the latent feature vector corresponding to knowledge point label A.

[0108] After calculating the latent feature vector corresponding to each preset question label, it can be stored so that the data can be quickly retrieved during subsequent model applications.

[0109] In step S503, the third question label associated with each sample question is determined based on the second vector distance between the latent feature vector corresponding to the sample question and the latent feature vector corresponding to each preset question label.

[0110] In this step, the above-mentioned second vector distance can be the Mahalanobis distance (from a mathematical perspective, the Mahalanobis distance represents the covariance distance of data, and it is an effective method for calculating the similarity between two unknown sample sets), Euclidean distance, Manhattan distance, Chebyshev distance, etc., which can be set according to the actual situation, and the present disclosure does not make special limitations on this.

[0111] Reference Figure 8 , Figure 8 shows a flowchart of how to determine the third problem label associated with each sample problem according to the second vector distance between the implicit feature vector corresponding to each sample problem and the implicit feature vector corresponding to each preset problem label in an embodiment of the present disclosure, including step S801-step S804:

[0112] In step S801, the second vector distance with the smallest value is selected from multiple second vector distances, and the preset problem label corresponding to the second vector distance with the smallest value is determined as the candidate third problem label associated with the sample problem.

[0113] In this step, after calculating multiple second vector distances, the above-mentioned multiple second vector distances can be numerically compared to select the second vector distance with the smallest value from the above-mentioned multiple second vector distances. Furthermore, the preset problem label corresponding to the second vector distance with the smallest value can be determined as the candidate third problem label associated with the sample problem.

[0114] Exemplarily, reference Figure 9 , Figure 9 shows a schematic diagram of how to determine the candidate third problem label associated with a sample problem in an embodiment of the present disclosure. As Figure 9 shown:

[0115] Assume that the second vector distance between the implicit feature vector X corresponding to the sample problem and the preset problem label A is l1, the second vector distance between the implicit feature vector X corresponding to the sample problem and the preset problem label B is l2, and the second vector distance between the implicit feature vector X corresponding to the sample problem and the preset problem label C is l3, and l1 < l2 < l3. Then, the preset problem label A corresponding to the second vector distance l1 with the smallest value can be determined as the candidate third problem label corresponding to the above sample problem.

[0116] Then refer to Figure 8 , in step S802, calculate the specified distance threshold corresponding to the candidate third problem label, and store the corresponding relationship between the candidate third problem label and the specified distance threshold.

[0117] In this step, firstly, multiple third vector distances (which can be Mahalanobis distances) can be calculated between the latent feature vectors corresponding to the candidate third question labels and the latent feature vectors corresponding to other preset question labels. These third vector distances are used to measure the correlation between each candidate third question label and other preset question labels, i.e., inter-class distances.

[0118] Secondly, multiple fourth vector distances (or Mahalanobis distances) can be calculated between the latent feature vectors corresponding to the multiple sample questions associated with the candidate third question label and the latent feature vectors corresponding to the candidate third question label. These fourth vector distances are used to measure the correlation between the candidate third question label and its associated multiple sample questions, i.e., intra-class distances.

[0119] Furthermore, the smallest third vector distance (i.e., the minimum inter-class distance) can be selected from the aforementioned multiple third vector distances, and its target association value can be obtained. For example, the target association value can be the median of the smallest third vector distances (for instance, assuming the smallest second vector distance is r, then the target association value can be...). Furthermore, the largest fourth vector distance is selected from the aforementioned multiple fourth vector distances, i.e., the maximum intra-class distance. Then, the larger of the aforementioned target association value and the largest fourth vector distance can be determined as the specified distance threshold corresponding to the candidate third question label.

[0120] For example, refer to Figure 10 , Figure 10 This illustration shows how a specified distance threshold corresponding to a preset problem label A is determined in an embodiment of this disclosure, as shown below. Figure 10 As shown:

[0121] Refer to the above Figure 9 The relevant explanation is that after determining the preset question label A as the candidate third question label corresponding to the above sample question, multiple third vector distances can be calculated between the latent feature vector of preset question label A and the latent feature vectors of other preset question labels. The third vector distance with the smallest value is then selected from these multiple third vector distances. For example: Figure 10 As shown in l4, further, multiple fourth vector distances are calculated between the latent feature vector of the candidate third problem label and the latent feature vectors of the multiple sample problems it contains. The fourth vector distance with the largest value is then selected from these multiple fourth vector distances. For example... Figure 10 The l5 shown, further, will Determine the above target correlation values, and The larger value between l5 and l6 is determined as the specified distance threshold.

[0122] It should be noted that when iterating through the above sample questions, the specified distance threshold corresponding to each preset question label can be calculated and stored for later use, so as to quickly find the specified distance threshold corresponding to each preset question label later.

[0123] In step S803, if the distance between the second vector with the smallest value is greater than a specified distance threshold, it is determined that there is no correlation between the sample question and the candidate third question label.

[0124] In this step, the numerical relationship between the second vector distance with the smallest value and the specified distance threshold can be compared. If the second vector distance with the smallest value is greater than the specified distance threshold, it can be determined that there is no correlation between the sample problem and the candidate third problem label. At this time, the sample problem can be removed from the training samples.

[0125] In step S804, if the distance of the second vector with the smallest value is not greater than the specified distance threshold, the candidate third question label is determined as the third question label associated with the sample question.

[0126] In this step, if the distance of the second vector with the smallest value is not greater than the specified distance threshold, then the candidate third question label can be determined as the third question label associated with the sample question.

[0127] After identifying the third question label associated with the sample question, you can then refer to... Figure 5 In step S504, the initial training samples are cleaned based on the consistency comparison results between the third question label and the preset question label associated with the sample question, and the cleaned training samples are obtained.

[0128] In this step, you can refer to Figure 11 , Figure 11 This embodiment of the present disclosure illustrates how to perform data cleaning on the initial training samples based on the consistency comparison results between the third question label and the preset question label associated with the sample question, to obtain the cleaned training samples, including steps S1101-S1103:

[0129] In step S1101, if the third question label is inconsistent with the preset question label associated with the sample question, the sample question is removed from the initial training samples to obtain cleaned training samples.

[0130] In this step, the third question label can be compared with the preset question label associated with the sample question. If they are inconsistent, the sample question can be directly removed from the initial training samples to obtain cleaned training samples. For example, assuming that the third question label corresponding to sample question A1 determined in step S804 is knowledge point label B, while the preset question label corresponding to sample question A1 is knowledge point label A, it can be determined that they are inconsistent. In this case, it indicates that sample question A1 may be misclassified, and therefore, sample question A1 can be removed from the initial training samples.

[0131] In step S1102, if the third question label is consistent with the preset question label associated with the sample question, then the fourth question label associated with the sample question is identified based on the latent feature vector corresponding to each sample question.

[0132] In this step, if the third question label is consistent with the preset question label associated with the sample question, a CLS vector can be extracted from the latent feature vector corresponding to the sample question. The CLS vector is used to characterize the correlation between the sample question and each preset question label. For example, the CLS vector can characterize the probability distribution of the sample question belonging to each preset question label. Therefore, the probability distribution can be normalized first, and then the preset question label with the highest probability can be selected as the fourth question label associated with the sample question.

[0133] In step S1103, the initial training samples are cleaned based on the consistency comparison results between the third question label and the fourth question label to obtain cleaned training samples.

[0134] In this step, the third and fourth question labels can be compared for consistency. If the third and fourth question labels match, the sample question can be retained in the initial training samples. If the third and fourth question labels do not match, the sample question can be removed from the initial training samples to perform data cleaning, resulting in cleaned training samples.

[0135] After obtaining the cleaned training samples, then refer to Figure 2 In step S204, the cleaned training samples are used as training samples, and the initial domain detection model is iteratively trained based on the training samples to obtain the updated domain detection model.

[0136] In this step, the cleaned training samples can be used as training samples to iteratively train the initial domain detection model to obtain an updated domain detection model, thereby improving the model quality.

[0137] Optionally, after obtaining the updated domain detection model, steps S203-S204 can be repeated multiple times to update the updated domain detection model multiple times. This can be set according to the actual situation, and this disclosure does not impose any special limitations on it.

[0138] By performing multiple data cleaning processes on the initial training samples and iterative updates on the initial domain detection model, the data quality of the training samples can be continuously improved, thereby enhancing the model's response quality.

[0139] After obtaining the updated domain detection model, you can then refer to... Figure 1 In step S110, corpus data is collected, and alternative questions are extracted from the corpus data.

[0140] In this step, the corpus data can be log data collected online, that is, the question data entered by users during the use of the intelligent response system. This question data can be data without question labels. By collecting corpus data, this disclosure can expand the training samples with new online data. Compared with the existing technology that only expands based on existing data, this disclosure can add new data to the training samples, thereby improving the generalization ability of the model.

[0141] After collecting the corpus data, we can proceed to step S120, where the updated domain detection model identifies the association between candidate questions and multiple preset question labels based on the first recognition logic, and obtains the first question label associated with the candidate questions.

[0142] In this step, you can refer to Figure 12 , Figure 12 This embodiment of the present disclosure illustrates how an updated domain detection model identifies the association between candidate questions and multiple preset question labels based on a first identification logic, thereby obtaining the first question label associated with a candidate question. The flowchart includes steps S1201-S1204.

[0143] In step S1201, the latent feature vectors corresponding to the candidate questions are extracted using the updated domain detection model.

[0144] In this step, the updated domain detection model can also contain n network layers. Therefore, referring to the explanations of steps S601-S603 above, the output features of each of the n network layers can be pooled to obtain the output vector corresponding to each network layer. Then, the cosine similarity between the output vector corresponding to the nth network layer and the output vector corresponding to each network layer is calculated to obtain n cosine similarities. After that, the n similarities can be used as weighting coefficients for the output vectors corresponding to the n network layers to perform weighted processing on the n output vectors corresponding to the n network layers to obtain the latent feature vectors corresponding to the candidate problem.

[0145] In step S1202, the latent feature vector corresponding to each preset question label is obtained.

[0146] In this step, since the latent feature vector corresponding to each preset question label has been calculated and stored in step S502 above, this step can directly call the pre-stored data to obtain the latent feature vector corresponding to each preset question label.

[0147] In step S1203, the first vector distance between the latent feature vector corresponding to the candidate question and the latent feature vector corresponding to each preset question label is calculated.

[0148] In this step, the first vector distance between the latent feature vector corresponding to the candidate question and the latent feature vector corresponding to each preset question label can be calculated. For example, the first vector distance can be Mahalanobis distance.

[0149] In step S1204, the first question label associated with the candidate question is determined based on multiple first vector distances.

[0150] In this step, refer to Figure 13 , Figure 13 This illustration shows a flowchart of how to determine the first question label associated with a candidate question based on multiple first vector distances in an embodiment of this disclosure, including steps S1301-S1304:

[0151] In step S1301, the first vector distance with the smallest value is selected from multiple first vector distances, and the preset question label corresponding to the first vector distance with the smallest value is determined as the candidate first question label associated with the candidate question.

[0152] In this step, the smallest first vector distance can be selected from multiple first vector distances mentioned above, and the preset question label corresponding to the smallest first vector distance can be determined as the candidate first question label associated with the candidate question.

[0153] In step S1302, a specified distance threshold corresponding to the candidate first problem label is found.

[0154] In this step, since the specified distance threshold corresponding to each preset question label has been calculated and stored in step S802 above, in this step, the specified distance threshold corresponding to the above candidate first question label can be directly searched.

[0155] In step S1303, if the distance between the first vector with the smallest value is greater than a specified distance threshold, it is determined that there is no correlation between the candidate question and the candidate first question label.

[0156] In this step, if the distance between the first vector with the smallest value is greater than the specified distance threshold, it is determined that there is no correlation between the candidate problem and the candidate first problem label. Therefore, the candidate problem can be directly discarded without using it to expand the training samples.

[0157] In step S1304, if the distance of the first vector with the smallest value is not greater than a specified distance threshold, the candidate first question label is determined as the first question label associated with the candidate question.

[0158] In this step, if the distance of the first vector with the smallest value is not greater than the specified distance threshold, then the first issue label can be determined as the first issue label associated with the filing issue.

[0159] Next, refer to Figure 1 In step S130, the updated domain detection model identifies the association between candidate questions and multiple preset question labels based on the second identification logic, and obtains the second question label associated with the candidate question.

[0160] In this step, since the latent feature vector corresponding to each candidate question has been extracted in step S1201, and the latent feature vector contains the CLS vector, the updated domain detection model can extract the CLS vector (the CLS vector is used to characterize the correlation between the candidate question and each preset question label) from the latent feature vector corresponding to the above candidate questions. Based on the CLS vector, the second question label associated with the candidate question is determined from multiple preset question labels.

[0161] In step S140, if the first question label is consistent with the second question label, the training samples corresponding to the updated domain detection model are expanded using alternative questions.

[0162] In this step, if the first problem label determined in step S1304 is consistent with the second problem label determined in step S130, the training samples corresponding to the updated domain detection model can be expanded using alternative problems (for example, the alternative problems can be classified under the first problem label). Otherwise, the alternative problems are discarded, and the process jumps to the next alternative problem, and it is determined whether to expand the training samples corresponding to the updated domain detection model using the next alternative problem.

[0163] refer to Figure 14 , Figure 14 This diagram illustrates the overall schematic of the training sample augmentation method in an embodiment of this disclosure, as shown below. Figure 14 As shown:

[0164] First, initial training samples can be obtained;

[0165] Secondly, an initial domain detection model is trained using the initial training samples. The initial training samples are then cleaned using the initial domain detection model. Finally, the initial domain detection model is iteratively trained using the cleaned initial training samples to obtain an updated domain detection model.

[0166] Next, the updated domain detection model is used to filter the online log data;

[0167] Finally, the updated domain detection model outputs augmented data to expand the training samples.

[0168] Overall, on the one hand, this disclosure trains an initial domain detection model, then cleans the initial training samples using the initial domain detection model, and uses the cleaned initial training samples to iteratively train the initial domain detection model to obtain an updated domain detection model. After multiple iterations, a domain detection model with better detection performance can be obtained. On this basis, errors and scrambled data in the initial training samples can be filtered out, improving the data quality of the initial training samples, thereby ensuring the question-answering performance of the intelligent question-answering system.

[0169] On the other hand, this disclosure utilizes an updated domain detection model based on two recognition logics to identify the association between candidate questions in online log data and preset question labels in the training samples. This allows candidate questions to be assigned to corresponding knowledge points or questioning intentions, obtaining the first and second question labels associated with the candidate questions. By comparing and verifying the first and second question labels, the association between candidate questions and multiple preset question labels can be accurately clarified. Then, the training samples can be expanded based on the candidate questions. This solves the problem of confusion or erroneous data caused by manually dividing associations during sample expansion in related technologies, improves the data quality of the expanded training samples, and further enhances the question-answering effect of the intelligent question-answering system.

[0170] This disclosure also provides a training sample augmentation device. Figure 15 This diagram illustrates the structure of the training sample augmentation device in an exemplary embodiment of this disclosure; as shown below. Figure 15 As shown, the training sample expansion device 1500 may include a data acquisition module 1510, a first recognition module 1520, a second recognition module 1530, and a sample expansion module 1540. Wherein:

[0171] Data acquisition module 1510 is used to acquire corpus data and extract alternative questions from the corpus data;

[0172] The first identification module 1520 is used to identify the association between the candidate questions and multiple preset question labels based on the first identification logic using the updated domain detection model, and to obtain the first question label associated with the candidate questions;

[0173] The second identification module 1530 is used to identify the association between the candidate questions and multiple preset question labels based on the second identification logic through the updated domain detection model, and obtain the second question label associated with the candidate questions;

[0174] The sample expansion module 1540 is used to expand the training samples corresponding to the updated domain detection model using the alternative questions if the first question label is consistent with the second question label.

[0175] In an exemplary embodiment of this disclosure, the preset question tag includes the knowledge point tag or the question intent tag.

[0176] In an exemplary embodiment of this disclosure, the first identification module 1520 identifies the association between the candidate questions and multiple preset question labels based on the updated domain detection model and the first identification logic, and obtains the first question label associated with the candidate questions, including: extracting the latent feature vectors corresponding to the candidate questions through the updated domain detection model; obtaining the latent feature vectors corresponding to each preset question label; calculating the first vector distance between the latent feature vectors corresponding to the candidate questions and the latent feature vectors corresponding to each preset question label; and determining the first question label associated with the candidate questions based on the multiple first vector distances.

[0177] In an exemplary embodiment of this disclosure, the updated domain detection model includes n network layers, where n is an integer greater than 1; the first identification module 1520 extracts the latent feature vector corresponding to the candidate question through the updated domain detection model, including: pooling the output features of each of the n network layers to obtain the output vector corresponding to each network layer; calculating the similarity between the output vector corresponding to the nth network layer and the output vector corresponding to each network layer to obtain n similarities; using the n similarities as weighting coefficients for the output vectors corresponding to the n network layers, and weighting the n output vectors corresponding to the n network layers to obtain the latent feature vector corresponding to the candidate question.

[0178] In an exemplary embodiment of this disclosure, the first identification module 1520 determines the first question label associated with the candidate question based on a plurality of first vector distances, including: filtering out the first vector distance with the smallest value from the plurality of first vector distances, and determining the preset question label corresponding to the first vector distance with the smallest value as the candidate first question label associated with the candidate question; searching for a specified distance threshold corresponding to the candidate first question label; if the first vector distance with the smallest value is greater than the specified distance threshold, determining that there is no association between the candidate question and the candidate first question label; if the first vector distance with the smallest value is not greater than the specified distance threshold, determining the candidate first question label as the first question label associated with the candidate question.

[0179] In an exemplary embodiment of this disclosure, after determining that there is no correlation between the sample question and the candidate first question label, the first identification module 1520 is configured to:

[0180] Discard the aforementioned alternative questions.

[0181] In an exemplary embodiment of this disclosure, the second identification module 1530 identifies the association between the candidate question and multiple preset question labels based on the updated domain detection model and the second identification logic, and obtains the second question label associated with the candidate question, including: extracting a CLS vector from the latent feature vector corresponding to the candidate question, wherein the CLS vector is used to characterize the correlation between the candidate question and each preset question label; and determining the second question label associated with the candidate question from the multiple preset question labels based on the CLS vector.

[0182] In an exemplary embodiment of this disclosure, the sample augmentation module 1540 is configured to:

[0183] Obtain initial training samples, which include multiple preset question labels, each of which is associated with multiple sample questions; iteratively train the domain detection model to be trained based on the initial training samples to obtain an initial domain detection model; use the initial domain detection model to clean the initial training samples to obtain cleaned training samples; use the cleaned training samples as the training samples, and iteratively train the initial domain detection model based on the training samples to obtain the updated domain detection model.

[0184] In an exemplary embodiment of this disclosure, the sample expansion module 1540 iteratively trains the domain detection model to be trained based on the initial training samples to obtain an initial domain detection model, including: inputting the initial training samples into the domain detection model to be trained to obtain a predicted question label associated with each sample question output by the domain detection model to be trained; determining a loss value of the domain detection model to be trained based on the degree of difference between the predicted question label and the preset question label associated with the sample question, and the degree of difference between any two sample questions among the plurality of sample questions; and iteratively training the domain detection model to be trained based on the loss value to obtain the initial domain detection model.

[0185] In an exemplary embodiment of this disclosure, the sample expansion module 1540 determines the loss value of the domain detection model to be trained based on the degree of difference between the predicted question label and the preset question label associated with the sample question, and the degree of difference between any two sample questions among the plurality of sample questions. This includes: calculating a first type of loss value based on the degree of difference between the predicted question label and the preset question label associated with the sample question; calculating a second type of loss value based on the degree of difference between any two sample questions among the plurality of sample questions; and determining the loss value of the domain detection model to be trained based on the first type of loss value and the second type of loss value.

[0186] In an exemplary embodiment of this disclosure, the sample expansion module 1540 uses the initial domain detection model to clean the initial training samples to obtain cleaned training samples, including: obtaining the latent feature vector corresponding to each sample question using the initial domain detection model; determining the latent feature vector corresponding to each preset question label based on the latent feature vector corresponding to the target sample question associated with each preset question label; determining the third question label associated with each sample question based on the second vector distance between the latent feature vector corresponding to the sample question and the latent feature vector corresponding to each preset question label; and cleaning the initial training samples based on the consistency comparison result between the third question label and the preset question label associated with the target sample question to obtain cleaned training samples.

[0187] In an exemplary embodiment of this disclosure, the initial domain detection model includes n network layers, where n is an integer greater than 1; the sample expansion module 1540 uses the initial domain detection model to obtain the latent feature vector corresponding to each sample question, including: pooling the output features of each of the n network layers to obtain the output vector corresponding to each network layer; calculating the similarity between the output vector corresponding to the nth network layer and the output vector corresponding to each network layer to obtain n similarities; using the n similarities as weighting coefficients for the output vectors corresponding to the n network layers, and weighting the n output vectors corresponding to the n network layers to obtain the latent feature vector corresponding to each sample question.

[0188] In an exemplary embodiment of this disclosure, the sample expansion module 1540 determines the latent feature vector corresponding to each preset question label based on the latent feature vector corresponding to the target sample question associated with each preset question label, including: performing mean calculation on the latent feature vector corresponding to the target sample question associated with each preset question label to obtain the latent feature vector corresponding to each preset question label.

[0189] In an exemplary embodiment of this disclosure, the sample expansion module 1540 determines a third question label associated with each sample question based on a second vector distance between the latent feature vector corresponding to the sample question and the latent feature vector corresponding to each preset question label. This includes: filtering out the second vector distance with the smallest value from a plurality of second vector distances, and determining the preset question label corresponding to the second vector distance with the smallest value as a candidate third question label associated with the sample question; calculating a specified distance threshold corresponding to the candidate third question label, and storing the correspondence between the candidate third question label and the specified distance threshold; if the second vector distance with the smallest value is greater than the specified distance threshold, determining that there is no association between the sample question and the candidate third question label; if the second vector distance with the smallest value is not greater than the specified distance threshold, determining the candidate third question label as the third question label associated with the sample question.

[0190] In an exemplary embodiment of this disclosure, the sample expansion module 1540 calculates a specified distance threshold corresponding to the candidate third question label, including: calculating multiple third vector distances between the latent feature vector corresponding to the candidate third question label and the latent feature vectors corresponding to other preset question labels; and calculating multiple fourth vector distances between the latent feature vectors corresponding to multiple sample questions associated with the candidate third question label and the latent feature vector corresponding to the candidate third question label; and determining the specified distance threshold corresponding to the candidate third question label based on the multiple third vector distances and the multiple fourth vector distances.

[0191] In an exemplary embodiment of this disclosure, the sample expansion module 1540 determines a specified distance threshold corresponding to the candidate third question label based on the plurality of third vector distances and the plurality of fourth vector distances, including: filtering the third vector distance with the smallest value from the plurality of third vector distances, and filtering the fourth vector distance with the largest value from the plurality of fourth vector distances; obtaining the target association value of the third vector distance with the smallest value; and determining the larger value between the target association value and the fourth vector distance with the largest value as the specified distance threshold corresponding to the candidate third question label.

[0192] In an exemplary embodiment of this disclosure, the sample expansion module 1540 performs data cleaning on the initial training samples based on the consistency comparison result between the third question label and the preset question label associated with the sample question to obtain cleaned training samples. This includes: if the third question label is inconsistent with the preset question label associated with the sample question, then the sample question is removed from the initial training samples to obtain the cleaned training samples; if the third question label is consistent with the preset question label associated with the sample question, then a fourth question label associated with the sample question is identified based on the latent feature vector corresponding to each sample question; and the initial training samples are cleaned based on the consistency comparison result between the third question label and the fourth question label to obtain cleaned training samples.

[0193] In an exemplary embodiment of this disclosure, the sample expansion module 1540 identifies a fourth question label associated with each sample question based on the latent feature vector corresponding to each sample question, including: extracting a CLS vector from the latent feature vector corresponding to the sample question, wherein the CLS vector is used to characterize the correlation between the sample question and each preset question label; and determining the fourth question label associated with the sample question from the plurality of preset question labels based on the CLS vector.

[0194] The specific details of each module in the above-mentioned training sample expansion device have been described in detail in the corresponding training sample expansion method, so they will not be repeated here.

[0195] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0196] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0197] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0198] This application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device.

[0199] Computer-readable storage media can be, for example—but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0200] A computer-readable storage medium can be sent, propagated, or transmitted for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0201] A computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0202] Furthermore, this disclosure also provides an electronic device capable of implementing the above-described method.

[0203] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0204] The following reference Figure 16 To describe an electronic device 1600 according to such an embodiment of the present disclosure. Figure 16 The electronic device 1600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0205] like Figure 16 As shown, the electronic device 1600 is manifested in the form of a general-purpose computing device. The components of the electronic device 1600 may include, but are not limited to: at least one processing unit 1610, at least one storage unit 1620, a bus 1630 connecting different system components (including storage unit 1620 and processing unit 1610), and a display unit 1640.

[0206] The storage unit stores program code that can be executed by the processing unit 1610, causing the processing unit 1610 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1610 can perform actions such as... Figure 1 As shown: Step S110, collect corpus data and extract candidate questions from the corpus data; Step S120, use the updated domain detection model to identify the association between the candidate questions and multiple preset question labels based on the first recognition logic, and obtain the first question label associated with the candidate questions; Step S130, use the updated domain detection model to identify the association between the candidate questions and multiple preset question labels based on the second recognition logic, and obtain the second question label associated with the candidate questions; Step S140, if the first question label and the second question label are consistent, use the candidate questions to expand the training samples corresponding to the updated domain detection model.

[0207] Storage unit 1620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 16201 and / or cache memory 16202, and may further include read-only memory (ROM) 16203.

[0208] Storage unit 1620 may also include a program / utility 16204 having a set (at least one) program module 16205, such program module 16205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0209] Bus 1630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0210] Electronic device 1600 can also communicate with one or more external devices 1700 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1600, and / or any device that enables electronic device 1600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1650. Furthermore, electronic device 1600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1660. As shown, network adapter 1660 communicates with other modules of electronic device 1600 via bus 1630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0211] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A method for expanding training samples, characterized in that, include: Collect corpus data and extract candidate questions from the corpus data; The updated domain detection model identifies the association between the candidate questions and multiple preset question labels based on the first identification logic, and obtains the first question label associated with the candidate questions; The step of identifying the association between the candidate questions and multiple preset question labels using the updated domain detection model based on the first recognition logic, and obtaining the first question label associated with the candidate questions, includes: extracting latent feature vectors corresponding to the candidate questions using the updated domain detection model; obtaining latent feature vectors corresponding to each preset question label; calculating a first vector distance between the latent feature vectors corresponding to the candidate questions and the latent feature vectors corresponding to each preset question label; and determining the first question label associated with the candidate questions based on multiple first vector distances; and... The updated domain detection model identifies the association between the candidate questions and multiple preset question labels based on the second recognition logic, thereby obtaining a second question label associated with the candidate questions. This process includes: extracting a CLS vector from the latent feature vectors corresponding to the candidate questions; the probability distribution output by the CLS vector after processing by a classification layer is used to characterize the correlation between the candidate questions and each preset question label; and determining the second question label associated with the candidate questions from the multiple preset question labels based on the CLS vector. If the first question label is the same as the second question label, the training samples corresponding to the updated domain detection model are expanded using the alternative questions.

2. The method according to claim 1, characterized in that, The preset question tags include knowledge point tags or question intent tags.

3. The method according to claim 1, characterized in that, The updated domain detection model includes n network layers, where n is an integer greater than 1; The step of extracting the latent feature vectors corresponding to the candidate questions using the updated domain detection model includes: Pooling is performed on the output features of each of the n network layers to obtain the output vector corresponding to each network layer. Calculate the similarity between the output vector corresponding to the nth network layer and the output vector corresponding to each of the network layers to obtain n similarity values; The n similarities are used as weighting coefficients for the output vectors corresponding to the n network layers. The n output vectors corresponding to the n network layers are weighted to obtain the latent feature vectors corresponding to the candidate problem.

4. The method according to claim 1, characterized in that, The step of determining the first question label associated with the candidate questions based on multiple first vector distances includes: The first vector distance with the smallest value is selected from multiple first vector distances, and the preset question label corresponding to the first vector distance with the smallest value is determined as the candidate first question label associated with the candidate question; Find the specified distance threshold corresponding to the candidate first question label; If the distance between the smallest first vector is greater than the specified distance threshold, it is determined that there is no correlation between the candidate question and the candidate first question label. If the distance of the first vector with the smallest value is not greater than the specified distance threshold, the candidate first question label is determined as the first question label associated with the alternative question.

5. The method according to claim 4, characterized in that, After determining that there is no correlation between the alternative questions and the candidate first question label, the method further includes: Discard the aforementioned alternative questions.

6. The method according to claim 1 or 2, characterized in that, The updated domain detection model is trained in the following way: Obtain initial training samples, which include multiple preset question labels, and each preset question label is associated with multiple sample questions; The initial domain detection model is iteratively trained based on the initial training samples to obtain the initial domain detection model. The initial training samples are cleaned using the initial domain detection model to obtain cleaned training samples. The cleaned training samples are used as the training samples, and the initial domain detection model is iteratively trained based on the training samples to obtain the updated domain detection model.

7. The method according to claim 6, characterized in that, The step of iteratively training the domain detection model to be trained based on the initial training samples to obtain the initial domain detection model includes: The initial training samples are input into the domain detection model to be trained to obtain the predicted question label associated with each sample question output by the domain detection model to be trained; The loss value of the domain detection model to be trained is determined based on the degree of difference between the predicted question label and the preset question label associated with the sample question, and the degree of difference between any two sample questions among the plurality of sample questions. Based on the loss value, the domain detection model to be trained is iteratively trained to obtain the initial domain detection model.

8. The method according to claim 7, characterized in that, The step of determining the loss value of the domain detection model to be trained based on the degree of difference between the predicted question label and the preset question label associated with the sample question, and the degree of difference between any two sample questions among the plurality of sample questions, includes: Based on the degree of difference between the predicted question label and the preset question label associated with the sample question, a first type of loss value is calculated; and, The second type of loss value is calculated based on the degree of difference between any two sample problems among the plurality of sample problems; The loss value of the domain detection model to be trained is determined based on the first type of loss value and the second type of loss value.

9. The method according to claim 6, characterized in that, The step of cleaning the initial training samples using the initial domain detection model to obtain cleaned training samples includes: The initial domain detection model is used to obtain the latent feature vector corresponding to each sample question. Based on the latent feature vector corresponding to the target sample question associated with each preset question label, determine the latent feature vector corresponding to each preset question label; Based on the second vector distance between the latent feature vector corresponding to the sample question and the latent feature vector corresponding to each preset question label, the third question label associated with each sample question is determined. Based on the consistency comparison results between the third question label and the preset question label associated with the target sample question, the initial training sample is cleaned to obtain the cleaned training sample.

10. The method according to claim 9, characterized in that, The initial domain detection model comprises n network layers, where n is an integer greater than 1; The step of obtaining the latent feature vector corresponding to each sample question using the initial domain detection model includes: Pooling is performed on the output features of each of the n network layers to obtain the output vector corresponding to each network layer. Calculate the similarity between the output vector corresponding to the nth network layer and the output vector corresponding to each of the network layers to obtain n similarity values; The n similarities are used as weighting coefficients for the output vectors corresponding to the n network layers. The n output vectors corresponding to the n network layers are weighted to obtain the latent feature vector corresponding to each sample problem.

11. The method according to claim 9, characterized in that, The step of determining the latent feature vector corresponding to each preset question label based on the latent feature vector corresponding to the target sample question associated with each preset question label includes: The latent feature vectors corresponding to the target sample questions associated with each preset question label are averaged to obtain the latent feature vectors corresponding to each preset question label.

12. The method according to claim 9, characterized in that, The step of determining the third question label associated with each sample question based on the second vector distance between the latent feature vector corresponding to the sample question and the latent feature vector corresponding to each preset question label includes: The second vector distance with the smallest value is selected from multiple second vector distances, and the preset question label corresponding to the second vector distance with the smallest value is determined as the candidate third question label associated with the sample question; Calculate the specified distance threshold corresponding to the candidate third question label, and store the correspondence between the candidate third question label and the specified distance threshold; If the distance between the second vector with the smallest value is greater than the specified distance threshold, it is determined that there is no correlation between the sample question and the candidate third question label. If the distance of the second vector with the smallest value is not greater than the specified distance threshold, the candidate third question label is determined as the third question label associated with the sample question.

13. The method according to claim 12, characterized in that, The calculation of the specified distance threshold corresponding to the candidate third question label includes: Calculate the distances between multiple third vectors between the latent feature vectors corresponding to the candidate third question labels and the latent feature vectors corresponding to other preset question labels; In addition, calculate multiple fourth vector distances between the latent feature vectors corresponding to the multiple sample questions associated with the candidate third question label and the latent feature vectors corresponding to the candidate third question label; Based on the plurality of third vector distances and the plurality of fourth vector distances, a specified distance threshold corresponding to the candidate third question label is determined.

14. The method according to claim 13, characterized in that, The step of determining the specified distance threshold corresponding to the candidate third question label based on the plurality of third vector distances and the plurality of fourth vector distances includes: Filter the third vector distance with the smallest value from the plurality of third vector distances, and filter the fourth vector distance with the largest value from the plurality of fourth vector distances; Obtain the target association value with the smallest third vector distance; The larger value between the target association value and the fourth vector distance with the largest value is determined as the specified distance threshold corresponding to the candidate third question label.

15. The method according to claim 9, characterized in that, The step of cleaning the initial training samples based on the consistency comparison results between the third question label and the preset question label associated with the sample question to obtain cleaned training samples includes: If the third question label is inconsistent with the preset question label associated with the sample question, the sample question is removed from the initial training samples to obtain the cleaned training samples. If the third question label is consistent with the preset question label associated with the sample question, then the fourth question label associated with the sample question is identified based on the latent feature vector corresponding to each sample question. Based on the consistency comparison results between the third question label and the fourth question label, the initial training samples are cleaned to obtain cleaned training samples.

16. The method according to claim 15, characterized in that, The step of identifying the fourth question label associated with each sample question based on the latent feature vector corresponding to each sample question includes: Extract the CLS vector from the latent feature vector corresponding to the sample question. The CLS vector is used to characterize the correlation between the sample question and each preset question label. Based on the CLS vector, determine the fourth question label associated with the sample question from the plurality of preset question labels.

17. A training sample augmentation device, characterized in that, include: The data acquisition module is used to collect corpus data and extract alternative questions from the corpus data; The first identification module is used to identify the association between the candidate questions and multiple preset question labels based on the first identification logic using the updated domain detection model, and to obtain the first question label associated with the candidate questions; The first identification module is used to extract the latent feature vectors corresponding to the candidate questions through the updated domain detection model; obtain the latent feature vectors corresponding to each preset question label; calculate the first vector distance between the latent feature vectors corresponding to the candidate questions and the latent feature vectors corresponding to each preset question label; and determine the first question label associated with the candidate questions based on multiple first vector distances. as well as, The second identification module is used to identify the association between the candidate questions and multiple preset question labels based on the second identification logic using the updated domain detection model, and to obtain the second question label associated with the candidate questions; The second identification module is used to extract CLS vectors from the latent feature vectors corresponding to the candidate questions. The probability distribution output by the CLS vectors after processing by the classification layer is used to characterize the correlation between the candidate questions and each preset question label. Based on the CLS vectors, the second question label associated with the candidate questions is determined from the plurality of preset question labels. The sample expansion module is used to expand the training samples corresponding to the updated domain detection model using the alternative questions if the first question label is consistent with the second question label.

18. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training sample augmentation method according to any one of claims 1 to 16.

19. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the training sample augmentation method according to any one of claims 1 to 16 by executing the executable instructions.

Images