Sample statement processing methods, devices, computer equipment, and storage media

Abstract

This application relates to a sample statement processing method, apparatus, computer device, and storage medium. The method includes: acquiring a target statement with an incorrect intent label; inputting the target statement with the incorrect intent label into a trained similarity sample detection model; comparing the similarity of the target statement with the original sample statements with intent labels based on the similarity sample detection model to detect incorrectly labeled sample statements similar to the target statement from a sample statement library; performing sample statement correction processing on the sample statement library based on the incorrectly labeled sample statements to update the sample statement library; using the updated sample statement library to train an intent classification model; the intent classification model is used to identify the intent of user input statements in a chatbot dialogue scenario and instruct the chatbot to respond based on the identified intent. This method can improve the efficiency of updating the sample statement library.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a sample statement processing method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, chatbot technology has emerged. Chatbots can be used for customer service, capturing keywords entered by users on the chat page and then searching for the most suitable response from a sample database. Using chatbots can not only improve service quality but also reduce service costs.

[0003] Clearly, the most important factor for chatbot applications is the accuracy of the sample library. In some cases, the sample library may contain confusing samples. Traditional methods rely heavily on manual labor, which is too time-consuming and labor-intensive, resulting in low efficiency in updating the sample library. Summary of the Invention

[0004] Therefore, it is necessary to provide a sample statement processing method, apparatus, computer equipment, and storage medium to improve the efficiency of sample library updates, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a sample statement processing method. The method includes:

[0006] Retrieve the target statement with an incorrect intent tag;

[0007] The target statement labeled with the erroneous intent tag is input into a trained similar sample detection model; the similar sample detection model is trained based on a set of original sample statements labeled with intent tags in the sample statement library;

[0008] Based on the similarity sample detection model, the target statement marked with the erroneous intent label is compared with each of the original sample statements marked with intent labels to detect erroneously labeled sample statements similar to the target statement from the sample statement library.

[0009] The sample statement library is updated by performing sample statement correction processing based on the erroneously marked sample statements. The updated sample statement library is used to train an intent classification model. The intent classification model is used to identify the intent of user input statements in a chatbot dialogue scenario and instruct the chatbot to respond based on the identified intent.

[0010] In one embodiment, the method further includes a training step for a similar sample detection model; the training step for the similar sample detection model includes:

[0011] Select basic sample statements from the set of original sample statements;

[0012] From the set of original sample statements, obtain the original sample statements that meet the preset similarity conditions with the basic sample statements, and use them as the similar sample statements corresponding to the basic sample statements.

[0013] From the similar sample statements corresponding to each basic sample statement, filter out similar sample statements that have the same intent label as the basic sample statement, and take the basic sample statement and the filtered similar sample statements as similar sample pairs;

[0014] The similar sample detection model is trained based on the similar sample pairs.

[0015] In one embodiment, the step of performing sample statement correction processing on the sample statement library based on the error-marked sample statements to update the sample statement library includes:

[0016] If a revision operation is detected for the error-marked sample statement, then the intent tag for revising the error-marked sample statement in the sample statement library is triggered;

[0017] If a deletion operation is detected for the error-marked sample statement, the error-marked sample statement is deleted from the sample statement library.

[0018] In one embodiment, after the step of triggering the revision of the intent tag of the error-tagged sample statement in the sample statement library if a revision operation for the error-tagged sample statement is detected, the method further includes:

[0019] Identify historical statements similar to the target statement from the chatbot's historical conversation records with users;

[0020] For the historical statements, add the revised intent tags corresponding to the error-marked sample statements;

[0021] The historical statements with the revised intent tags added will be added as sample statements to the sample statement library.

[0022] In one embodiment, the target statement is a user-input statement generated in the chatbot's historical conversation with the user;

[0023] The method further includes:

[0024] Determine the first context statement corresponding to the target statement in its target historical session;

[0025] Identify a second context statement that is similar to the first context statement from the session records of non-target historical sessions;

[0026] From the session records of the non-target historical sessions, determine the reference statement located between the second context statements and marked with the correct intent label;

[0027] If the reference statement and the target statement meet a preset similarity condition, then the correct intent tag of the reference statement is added to the target statement, and the target statement with the correct intent tag is added to the sample statement library.

[0028] In one embodiment, the step of comparing the target statement tagged with an erroneous intent label with each of the original sample statements tagged with intent labels based on the similarity sample detection model, so as to detect erroneously tagged sample statements similar to the target statement from the sample statement library, includes:

[0029] The intent tags marked for each of the original sample statements are compared with the erroneous intent tags marked for the target statement;

[0030] The original sample statement corresponding to the intent label that is the same as the erroneous intent label is taken as the target original sample statement;

[0031] Based on the similarity sample detection model, the target statement is compared with each of the target original sample statements to detect erroneously marked sample statements similar to the target statement from the sample statement library.

[0032] Secondly, this application also provides a sample statement processing apparatus. The apparatus includes:

[0033] The acquisition module is used to acquire target statements with incorrect intent marking.

[0034] The detection module is used to input the target statement labeled with an incorrect intent tag into a trained similar sample detection model; the similar sample detection model is trained based on a set of original sample statements labeled with intent tags in the sample statement library; based on the similar sample detection model, the target statement labeled with an incorrect intent tag is compared with each of the original sample statements labeled with intent tags to detect incorrectly labeled sample statements similar to the target statement from the sample statement library;

[0035] The correction module is used to perform sample statement correction processing on the sample statement library based on the erroneously marked sample statements, so as to update the sample statement library; the updated sample statement library is used to train an intent classification model; the intent classification model is used to identify the intent of the user input statement in the robot dialogue scenario and instruct the robot to respond based on the identified intent.

[0036] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the methods described in the embodiments of this application.

[0037] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, causes the processor to perform the steps of the methods described in the embodiments of this application.

[0038] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the steps of the methods described in the embodiments of this application.

[0039] The aforementioned sample statement processing method, apparatus, computer equipment, and storage medium acquire target statements with incorrect intent labels; input the target statements with incorrect intent labels into a trained similarity sample detection model; based on the similarity sample detection model, compare the similarity of the target statements with each original sample statement with intent labels to detect incorrectly labeled sample statements similar to the target statements from the sample statement library. This method improves efficiency compared to manually detecting potentially erroneous sample statements. The sample statement library is then updated based on the incorrectly labeled sample statements, enabling rapid updates. The updated sample statement library is used to train an intent classification model; this model is used to identify the intent of user input statements in robot dialogue scenarios, thereby automatically updating the sample statement library and improving its update efficiency. Attached Figure Description

[0040] Figure 1 This is an application environment diagram of a sample statement processing method in one embodiment;

[0041] Figure 2 This is a flowchart illustrating a sample statement processing method in one embodiment;

[0042] Figure 3a This is a schematic diagram of the structure of a similar sample detection model in one embodiment;

[0043] Figure 3b This is a schematic diagram of the basic network structure in one embodiment;

[0044] Figure 4 This is a flowchart illustrating the sample statement processing method in another embodiment;

[0045] Figure 5 This is a structural block diagram of a sample statement processing device in one embodiment;

[0046] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0047] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] The sample statement processing method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 can obtain target statements with incorrect intent labels through terminal 102; server 104 can input the target statements with incorrect intent labels into a trained similarity sample detection model, and based on the similarity sample detection model, compare the similarity of the target statements with each original sample statement with intent labels to detect incorrectly labeled sample statements similar to the target statements from the sample statement library; server 104 can perform sample statement correction processing on the sample statement library based on the incorrectly labeled sample statements through terminal 102 to update the sample statement library. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0050] In one embodiment, such as Figure 2 As shown, a sample statement processing method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0051] S202, Obtain the target statement with incorrect intent label, and input the target statement with incorrect intent label into the trained similar sample detection model.

[0052] Intent tags are labels that identify the intent of a statement. In the context of a chatbot's conversation with a user, the user's intent can be identified based on the intent tags of the matched user input statements. An intent tagging error occurs when the intent of the statement itself does not correspond to its intent tag. In other words, an incorrect intent tag is an intent tag that does not correspond to the statement's actual intent. The similarity sample detection model is trained on a collection of original sample statements labeled with intent tags from a sample statement library. The sample statement library stores sample statements. It is a collection of all sample statements. Original sample statements are those labeled with intent tags in the sample statement library before correction. A sample statement is a statement combined with its associated intent tag to form a sample.

[0053] Specifically, the server can obtain the target statement with the incorrect intent label through the terminal and input the target statement with the incorrect intent label into the trained similar sample detection model.

[0054] In one embodiment, the terminal can obtain the target statement intended to mark an error and send the target statement to the server.

[0055] In one embodiment, the terminal can store a trained similar sample detection model. The terminal can obtain a target statement with an incorrect intent label and directly input the target statement with the incorrect intent label into the trained similar sample detection model.

[0056] In one embodiment, the instructor can use the terminal to determine statements with incorrect intent labeling and mark those statements as incorrect. It can be understood that the terminal can use statements marked as having incorrect intent labeling by the instructor as target statements.

[0057] Among them, machine trainers are essentially artificial intelligence trainers. They are professionals in enterprises who understand both business operations and the principles of artificial intelligence. They are able to combine the characteristics of artificial intelligence with actual business scenarios, independently deploy and configure intelligent products, and conduct targeted training, improvement, and optimization of certain functions of intelligent products, so that artificial intelligence can play a specific business value in the application process.

[0058] In one embodiment, the target statement may include at least one of sample statements from a sample statement library and statements from historical session records.

[0059] In one embodiment, the similarity sample detection model may include at least one of a triplet network model, a Siamese neural network model, and a sentence-transformer-based bidirectional encoder representation (Sentence-BERT) model.

[0060] S204, based on the similarity sample detection model, compares the similarity between the target statement marked with the erroneous intent label and each original sample statement marked with the intent label, so as to detect the erroneously labeled sample statements similar to the target statement from the sample statement library.

[0061] Among them, the similar sample detection model refers to the model used to detect similar sample sentences.

[0062] Specifically, the server can use a similarity sample detection model to compare the similarity of the target statement tagged with an erroneous intent label with each original sample statement tagged with an intent label, in order to detect erroneously tagged sample statements similar to the target statement from the sample statement library. The server can then obtain the erroneously tagged sample statements and send them to the terminal.

[0063] In one embodiment, the terminal can store a similar sample detection model. After comparing the similarity of the target statement marked with the erroneous intent label with each original sample statement marked with the intent label using the similar sample detection model, the terminal can directly detect the erroneously labeled sample statements that are similar to the target statement from the sample statement library.

[0064] In one embodiment, the similarity sample detection model has preset detection conditions for the similarity comparison results. It can be understood that if the similarity comparison result between the target statement and the original sample statement meets the preset detection conditions, then the original sample statement is an incorrectly labeled sample statement similar to the target statement.

[0065] The preset detection conditions may include at least one of the following: the similarity between the target statement and the original sample statement is greater than or equal to a similarity threshold, and the similarity ranking is within the top preset position. It can be understood that if the similarity between the target statement and the original sample statement is greater than or equal to the similarity threshold, then the original sample statement is an incorrectly labeled sample statement similar to the target statement. If, after sorting the similarity between the target statement and the original sample statement in descending order, the similarity ranking is within the top preset position, then the original sample statement is an incorrectly labeled sample statement similar to the target statement. It can be understood that the top preset position refers to the number of statements ranked highest after sorting the similarity in descending order. For example, if the top preset position is 20, then the original sample statements whose similarity to the target statement is within the top 20 are incorrectly labeled sample statements similar to the target statement.

[0066] S206, Based on the error-marked sample statements, perform sample statement correction processing on the sample statement library to update the sample statement library.

[0067] The updated sample statement library is used to train the intent classification model. The intent classification model is used to identify the intent of user input statements in chatbot dialogue scenarios and instruct the chatbot to respond based on the identified intent.

[0068] Specifically, the terminal can perform sample statement correction processing on the sample statement library based on the error-marked sample statements, and the server can update the sample statement library in response to the sample statement correction processing.

[0069] In one embodiment, the server can remove error-marked sample statements from the sample statement library.

[0070] In one embodiment, the server can revise the intent label of an error-tagged sample statement from a sample statement library.

[0071] In one embodiment, the server can retrieve statements similar to the target statement from historical session records based on the target statement, and update the sample statement library with the statement correctly labeled with the intent tag.

[0072] The above sample statement processing method involves: acquiring target statements with incorrect intent labels; inputting the target statements with incorrect intent labels into a trained similarity sample detection model; comparing the similarity of the target statements with the original sample statements with intent labels based on the similarity detection model to detect incorrectly labeled sample statements similar to the target statements from the sample statement library. This method is more efficient than manually detecting potentially erroneous sample statements. The sample statement library is then updated based on the incorrectly labeled sample statements, enabling rapid updates. The updated sample statement library is used to train an intent classification model. This intent classification model is used to identify the intent of user input statements in robot dialogue scenarios, thus automatically updating the sample statement library, improving update efficiency, and ensuring timely training of the intent classification model based on the updated sample statement library, thereby improving the classification accuracy of the intent classification model.

[0073] In one embodiment, the method further includes a training step for a similar sample detection model; the training step for the similar sample detection model includes: selecting basic sample statements from a set of original sample statements; obtaining original sample statements from the set of original sample statements that satisfy preset similarity conditions with the basic sample statements, and using them as similar sample statements corresponding to the basic sample statements; filtering similar sample statements with the same intent label as the basic sample statements from the similar sample statements corresponding to each basic sample statement, and using the basic sample statements and the filtered similar sample statements as similar sample pairs; and training the similar sample detection model based on the similar sample pairs.

[0074] The base sample statement is used to construct similar sample pairs. Essentially, the base sample statement is a reference statement; only by obtaining original sample statements that are similar to the base sample statement and have the same intent label can similar sample pairs be constructed. Preset similarity conditions are used to determine similar sample statements to the base sample statement from the set of original sample statements. A similar sample statement is a sample statement that is similar to the base sample statement.

[0075] Specifically, the server can select base sample statements from the set of original sample statements, and use a text similarity algorithm to obtain original sample statements from the set that meet preset similarity conditions with the base sample statements. These are then used as similar sample statements corresponding to the base sample statements. The server compares the intent of the base sample statements and the similar sample statements, filters out similar sample statements with the same intent tag as the base sample statements, and sets the base sample statements and the filtered similar sample statements as similar sample pairs. The server can then train a similar sample detection model based on these similar sample pairs.

[0076] In one embodiment, the server can select all the original sample statements from the sample statement library as the base sample statements.

[0077] In one embodiment, the text similarity algorithm can be the BM25 algorithm. The server can obtain basic sample sentences and, using the BM25 algorithm, search the sample sentence library for original sample sentences that meet preset similarity conditions with the basic sample sentences. These are then used as the similar sample sentences corresponding to the basic sample sentences. For example, the preset similarity condition can be that the similarity value output by the BM25 algorithm is within the top preset position. It can be understood that the larger the similarity value output by the BM25 algorithm, the higher the ranking, and the more similar the similar sample sentences are to the basic sample sentences. When the current preset position is 15, the server can obtain the 15 original sample sentences most similar to the basic sample sentences, i.e., the 15 similar sample sentences.

[0078] The BM25 algorithm is an algorithm for calculating the similarity between sentences and documents. It can segment the input sentence into words, calculate the similarity between each word in the sentence and the document, perform weighted summation, and finally obtain the similarity score between the sentence and the document.

[0079] In one embodiment, when comparing the intent tags of similar sample statements and base sample statements, the server may assign a value of 1 for statements with the same intent tag and a value of 0 for statements with different intent tags, thus outputting the intent tag comparison result. It can be understood that base sample statements with an output of 1 and similar sample statements with the same intent tag can be used as similar sample pairs for training a similar sample detection model.

[0080] In one embodiment, Figure 3aThis is a schematic diagram of the similarity detection model. The similarity detection model is a Siamese neural network model. It consists of two base networks. The server can input two sample statements into the two base networks respectively. After processing by the two base networks, a loss function is used to evaluate the similarity, and finally, cosine similarity is used to compare the similarity between the two input sample statements. In other words, the server can use the similar sample statements and the base sample statements from a similar sample pair as inputs to the two base networks to train the similarity detection model.

[0081] The Siamese neural network model is based on a coupled architecture of two artificial neural networks. It takes two samples as input and outputs their representations embedded in a high-dimensional space to compare the similarity between the two samples. In a narrow sense, a Siamese neural network consists of two neural networks with identical structures and shared weights. In a broader sense, or "pseudo-Siamese network," it can be composed of any two neural networks. The loss function of the similarity sample detection model is the Contrastive Loss function, and the optimizer is the RMSpro (Root Mean Square (Backpropagation) optimizer. This means that when training the similarity sample detection model, the server can use the optimizer based on gradients to minimize the loss function.

[0082] In one embodiment, the server can use the target statement and each sample statement in the sample statement library as... Figure 3a The similarity sample detection model takes two base networks as inputs and evaluates their similarity using the Contrastive Loss function to detect sample statements similar to the target statement. After detecting sample statements similar to the target statement using the similarity sample detection model, the server can then perform cosine similarity statistics. After two similarity evaluations, the incorrectly labeled sample statements are obtained.

[0083] Figure 3b This is a schematic diagram of the basic network structure. It can be understood that a similar sample detection model can include a basic network. Figure 3b The basic network in the middle corresponds to Figure 3a The basic network in the diagram consists of inputs, an embedding layer, a flattening layer, three fully connected layers, a dense and dropout layer, and outputs.

[0084] When a sample sentence is input, the embedding layer maps it from the semantic space to the vector space, while preserving the original semantic relationships of the samples in the vector space as much as possible; for example, two semantically similar words are also positioned close together in the vector space. The flattening layer can reduce the multidimensional input processed by the embedding layer to one dimension. The fully connected (Dense) layer maps distributed feature representations to the sample label space, essentially a linear transformation from one feature space to another. Dropout allows the activation values ​​of a neuron to stop working with a certain probability during forward propagation, making the model more generalizable because it does not rely too much on certain local features.

[0085] In this embodiment, basic sample statements are selected from the set of original sample statements; from the set of original sample statements, original sample statements that meet preset similarity conditions with the basic sample statements are obtained as similar sample statements corresponding to the basic sample statements; from the similar sample statements corresponding to each basic sample statement, similar sample statements with the same intent tag as the basic sample statements are selected, and the basic sample statements and the selected similar sample statements are used as similar sample pairs; based on the similar sample pairs, a similar sample detection model is trained. The similar sample detection model trained in this way is adapted to the sample statement library. When searching for obfuscated samples in the sample statement library in the future, it can quickly and accurately locate obfuscated samples similar to the target statement based on the target statement, thereby reducing labor costs.

[0086] In one embodiment, updating the sample statement library by correcting sample statements based on erroneously marked sample statements includes: if a revision operation for an erroneously marked sample statement is detected, triggering the revision of the intent label of the erroneously marked sample statement in the sample statement library; if a deletion operation for an erroneously marked sample statement is detected, triggering the deletion of the erroneously marked sample statement from the sample statement library.

[0087] The "revision" operation refers to revising the intent label of an incorrectly marked sample statement from the sample statement library. The "deletion" operation refers to deleting the intent label of an incorrectly marked sample statement from the sample statement library.

[0088] Specifically, if the server detects that a trainer has modified an error-marked sample statement via a terminal, the server can receive the modification information sent by the terminal and modify the intent tag of the error-marked sample statement in the sample statement library according to the modification information. If the server detects that a trainer has deleted an error-marked sample statement via a terminal, the server can receive the deletion information sent by the terminal and delete the error-marked sample statement from the sample statement library according to the deletion information.

[0089] The revision information includes details of how the instructor revised the intent tags of the erroneously labeled sample statements via the terminal. For example, it specifies which erroneously labeled sample statement's intent tag was revised and what the revisions were. The deletion information includes details of how the instructor deleted erroneously labeled sample statements via the terminal. For example, it specifies which erroneously labeled sample statement was deleted.

[0090] In one embodiment, the instructor can batch revise the intent tags of erroneously labeled sample statements on the terminal, and batch delete erroneously labeled sample statements.

[0091] In this embodiment, if a revision operation for an incorrectly labeled sample statement is detected, the intent label of the incorrectly labeled sample statement in the sample statement library is revised; if a deletion operation for an incorrectly labeled sample statement is detected, the incorrectly labeled sample statement is deleted from the sample statement library. Revising the intent label of the incorrectly labeled sample statement and deleting the incorrectly labeled sample statement can achieve timely updates to the sample statement library, improve the quality of the sample statement library, and thus improve the recognition accuracy of the intent classification model trained based on the sample statement library.

[0092] In one embodiment, after triggering the revision intent tag of the erroneously marked sample statement in the sample statement library if a revision operation for the erroneously marked sample statement is detected, the method further includes: identifying historical statements similar to the target statement from the historical conversation records between the chatbot and the user; adding the revised intent tag corresponding to the erroneously marked sample statement to the historical statement; and adding the historical statement with the revised intent tag to the sample statement library as a sample statement.

[0093] Here, "historical conversation records" refers to the historical conversation records between the chatbot and the user within the conversation system. In other words, a chatbot can be applied to a conversation system to chat with users. "Historical statements" refers to the statements recorded in the historical conversation records.

[0094] Specifically, the server can obtain historical conversation records between the chatbot and the user from the conversation system, and use a similar sample detection model to identify historical statements similar to the target statement from the historical conversation records. The server can add revised intent tags corresponding to the error-marking sample statements to the historical statements, and add the historical statements with the revised intent tags to the sample statement library as sample statements.

[0095] In this embodiment, historical statements similar to the target statement are identified from the chatbot's historical conversation records with the user; for the historical statements, revised intent tags corresponding to the erroneously marked sample statements are added; the historical statements with the revised intent tags are added as sample statements to the sample statement library, the erroneously marked sample statements are revised, and similar samples are efficiently added to improve the richness of the sample statement library, thereby improving the recognition and generalization ability of the intent classification model.

[0096] In one embodiment, the target statement is a user input statement generated in the historical conversation between the chatbot and the user; the method further includes: determining a first context statement corresponding to the target statement in its target historical conversation; identifying a second context statement similar to the first context statement from the conversation records of non-target historical conversations; determining a reference statement located between the second context statements and marked with a correct intent label from the conversation records of non-target historical conversations; if the reference statement and the target statement meet a preset similarity condition, adding the correct intent label of the reference statement to the target statement, and adding the target statement with the correct intent label to the sample statement library.

[0097] Here, the first context statement refers to the context statement of the target statement. The second context statement refers to the context statement of the reference statement. The reference statement is the statement corresponding to the target statement, which can be used to mark the target statement with intent tags. It can be understood that in different historical sessions, if the intent of the context statements is the same, then the statements between the context statements are also likely to have the same intent.

[0098] Specifically, the server can determine the identifier of the target historical session to which the target statement belongs, locate the target historical session corresponding to that identifier, and search for the first context statement corresponding to the target statement according to the chronological order of the target statement in the target historical session. The server can compare the relevance between statements in the session records of non-target historical sessions and the first context statement, thereby identifying a second context statement similar to the first context statement. The server can determine the identifier of the historical session mentioned in the second context statement, and from the session records of the non-target historical sessions corresponding to that identifier, determine the reference statement located between the second context statements and marked with the correct intent label according to the chronological order of the second context statements. If the reference statement and the target statement meet a preset similarity condition, the correct intent label of the reference statement is added to the target statement, and the target statement with the added correct intent label is added to the sample statement library.

[0099] In one embodiment, the server can use user input statements whose intent the chatbot cannot recognize as target statements. It is understood that users may make mistakes during conversations with the chatbot, and the chatbot may not accurately identify these errors. If such errors occur multiple times, the server can tag the errors with intent labels by comparing the context of correct statements with the context of incorrect statements, and add the incorrect statements to the sample statement library.

[0100] In one embodiment, the server can use a similarity sample detection model to compare the relevance between statements in the session records of a non-target historical session and statements in the first context.

[0101] In one embodiment, the preset similarity conditions may include at least one of the following: similarity greater than or equal to a similarity threshold and similarity ranking in descending order within the top preset positions.

[0102] In this embodiment, the first context statement corresponding to the target statement in its target historical session is determined; a second context statement similar to the first context statement is identified from the session records of non-target historical sessions; a reference statement located between the second context statements and marked with the correct intent label is determined from the session records of non-target historical sessions; if the reference statement and the target statement meet the preset similarity conditions, the correct intent label of the reference statement is added to the target statement, and the target statement with the correct intent label is added to the sample statement library. This can mark the correct intent label for incorrect statements repeatedly made by the user and add them to the sample statement library, enriching the sample statement library and thereby improving the recognition and generalization ability of the intent classification model.

[0103] In one embodiment, based on a similarity sample detection model, comparing the similarity of a target statement tagged with an erroneous intent label with each original sample statement tagged with an intent label to detect erroneously tagged sample statements similar to the target statement from the sample statement library includes: comparing the intent labels tagged for each original sample statement with the erroneous intent labels tagged for the target statement; taking the original sample statements corresponding to intent labels that are the same as the erroneous intent labels as the target original sample statements; and comparing the similarity of the target statement with each target original sample statement based on a similarity sample detection model to detect erroneously tagged sample statements similar to the target statement from the sample statement library.

[0104] Specifically, the server can use a classification model to compare the intent tags labeled for each original sample statement with the erroneous intent tags labeled for the target statement. The original sample statement corresponding to the intent tag that is the same as the erroneous intent tag is taken as the target original sample statement. The server can also use a similarity sample detection model to compare the similarity between the target statement and each target original sample statement in order to detect erroneously labeled sample statements similar to the target statement from the sample statement library.

[0105] The classification model can include at least one of the following: multi-class classification model, multi-label classification model, and imbalanced classification model.

[0106] In this embodiment, the intent tags marked for each original sample statement are compared with the erroneous intent tags marked for the target statement; the original sample statements corresponding to intent tags that are the same as the erroneous intent tags are taken as the target original sample statements; based on the similarity sample detection model, the target statement is compared with each target original sample statement to detect erroneously marked sample statements that are similar to the target statement from the sample statement library. After filtering out the original sample statements with the same intent tags, the erroneously marked sample statements can be detected based on the similarity sample detection model, which improves the accuracy of detecting erroneously marked sample statements.

[0107] Figure 4 This is a flowchart illustrating the sample statement processing method in another embodiment;

[0108] S402, the machine trainer marks the target statement that has been marked as an error.

[0109] Trainers can use the terminal to identify target statements with incorrect intent tags and mark them as erroneous. The terminal can then send these erroneous statements to the server. For example, if a user says something and the chatbot interprets it as intent A, the trainer will manually verify its accuracy. If incorrect, the user's statement will be tagged. Essentially, tagging marks a statement as erroneous. Alternatively, trainers can tag samples in a sample statement library; in this case, the target statements are those with incorrect intent tags.

[0110] S404 uses a similar sample detection model to recall candidate error-tagged sample statements that are similar to the target statement from the sample statement library.

[0111] The server inputs the received target statement into the similarity sample detection model. The similarity sample detection model compares the similarity between the target statement and sample statements in the sample statement library, and directly outputs sample statements similar to the target statement as candidate error-labeled sample statements. It can be understood that the similarity sample detection model can be a Siamese neural network model. It should be noted that candidate error-labeled sample statements can also be directly used as error-labeled sample statements.

[0112] S406, Calculate the similarity between candidate error-tagged sample statements and target statements in order to filter out error-tagged sample statements from the candidate error-tagged sample statements.

[0113] After detecting candidate error-labeled sample statements output by the similarity sample detection model, the server can again compare the cosine similarity between the candidate error-labeled sample statements and the target statement, and sort them in descending order according to the cosine similarity. The candidate error-labeled sample statements with the highest cosine similarity ranking are then designated as error-labeled samples. For example, if the highest pre-set ranking is 20, and candidate error-labeled sample statements ranked 20th or higher in descending order are considered more similar to the target statement, the server will ultimately send the 20 error-labeled sample statements similar to the target statement to the terminal. In essence, S406 is a further filtering of error-labeled sample statements based on S404.

[0114] S408, the machine trainer determines whether to delete the error-marked sample statement and the target statement.

[0115] The terminal can display incorrectly labeled samples to the training instructor, who then determines whether to delete the target statement and the incorrectly labeled samples. If not deleted, step S410 is executed. If deleted, step S412 is executed. It can be understood that the target statement is essentially a confusing sample. A confusing sample refers to a sample statement labeled with an incorrect intent tag; that is, a misleading sample statement that can cause the chatbot to mistake one thing for another. For example, if sample statement 'a' is associated with intent B in the sample statement library, but it is manually confirmed that sample statement 'a' should be associated with intent A, then sample 'a' is called a confusing sample.

[0116] S410, revise the intent tag, and at the same time, obtain statements similar to the target statement from historical session records through a similar sample detection model, and tag them with intent tags to supplement the sample statement library as sample statements.

[0117] Trainers can revise the intent tags of target statements and error-marked statements via the terminal and store the revised statements in the sample statement library. Simultaneously, the server can use a similarity sample detection model to obtain statements similar to the target statement from historical session records in the customer service session system, mark the revised intent tags of the target statements, and store them in the sample statement library. In essence, the server can directly revise the intent tags of target statements and error-marked statements, eliminating the need for trainers to manually modify them on the terminal.

[0118] S412, delete the erroneous sample statement and the target statement.

[0119] The instructor can delete target statements and error-marked statements on the terminal, and the server can respond to this operation by deleting the target statements and error-marked statements from the sample statement library.

[0120] After performing the above steps to correct the sample statement library, the server can retrain the intent classification model based on the corrected sample statement library.

[0121] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0122] Based on the same inventive concept, this application also provides a sample statement processing apparatus for implementing the sample statement processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more sample statement processing apparatus embodiments provided below can be found in the limitations of the sample statement processing method described above, and will not be repeated here.

[0123] In one embodiment, such as Figure 5 As shown, a sample statement processing device 500 is provided, including: an acquisition module 502, a detection module 504, and a correction module 506, wherein:

[0124] The acquisition module 502 is used to acquire the target statement with an intent-marked error.

[0125] The detection module 504 is used to input the target statement labeled with an erroneous intent tag into a trained similar sample detection model; the similar sample detection model is trained based on a set of original sample statements labeled with intent tags in the sample statement library; based on the similar sample detection model, the target statement labeled with an erroneous intent tag is compared with each of the original sample statements labeled with intent tags to detect erroneously labeled sample statements similar to the target statement from the sample statement library.

[0126] The correction module 506 is used to perform sample statement correction processing on the sample statement library based on the erroneously marked sample statements, so as to update the sample statement library; the updated sample statement library is used to train an intent classification model; the intent classification model is used to identify the intent of the user input statement in the robot dialogue scenario and instruct the chatbot to respond based on the identified intent.

[0127] In one embodiment, the apparatus further includes a training module. The acquisition module 502 is further configured to: select basic sample statements from the set of original sample statements; acquire original sample statements from the set of original sample statements that satisfy preset similarity conditions with the basic sample statements, as similar sample statements corresponding to the basic sample statements; filter similar sample statements with the same intent tag as the basic sample statements from the similar sample statements corresponding to each basic sample statement, and treat the basic sample statements and the filtered similar sample statements as similar sample pairs; the training module is configured to train the similar sample detection model based on the similar sample pairs.

[0128] In one embodiment, the correction module 506 is further configured to, if a revision operation for the error-marked sample statement is detected, trigger the revision of the intent tag of the error-marked sample statement in the sample statement library; and if a deletion operation for the error-marked sample statement is detected, trigger the deletion of the error-marked sample statement from the sample statement library.

[0129] In one embodiment, the correction module 506 is further configured to identify historical statements similar to the target statement from the historical conversation records between the chatbot and the user; add revised intent tags corresponding to the error-marked sample statements to the historical statements; and add the historical statements with the revised intent tags as sample statements to the sample statement library.

[0130] In one embodiment, the target statement is a user input statement generated in the historical conversation between the chatbot and the user; the correction module 506 is further configured to determine the first context statement corresponding to the target statement in the target historical conversation; identify a second context statement similar to the first context statement from the conversation records of the non-target historical conversation; determine a reference statement located between the second context statement and marked with a correct intent tag from the conversation records of the non-target historical conversation; if the reference statement and the target statement meet a preset similarity condition, add the correct intent tag of the reference statement to the target statement, and add the target statement with the added correct intent tag to the sample statement library.

[0131] In one embodiment, the detection module 504 is further configured to compare the intent tags marked for each of the original sample statements with the erroneous intent tags marked for the target statement; take the original sample statements corresponding to intent tags that are the same as the erroneous intent tags as target original sample statements; and perform similarity comparison between the target statement and each of the target original sample statements based on the similarity detection model, so as to detect erroneously marked sample statements similar to the target statement from the sample statement library.

[0132] Each module in the aforementioned sample statement processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0133] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores sample statements. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a sample statement processing method.

[0134] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a sample statement processing method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0135] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0136] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0137] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0138] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0140] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A sample statement processing method, characterized in that, The method includes: Obtain the target statement with an incorrect intent tag; the target statement is marked with an incorrect intent tag; the incorrect intent tag refers to an intent tag that does not correspond to the intent of the statement itself; The target statement is input into a trained similar sample detection model; the similar sample detection model is trained based on a set of original sample statements labeled with intent tags in the sample statement library; The intent tags marked for each of the original sample statements are compared with the erroneous intent tags marked for the target statement; The original sample statement corresponding to the intent label that is the same as the erroneous intent label is taken as the target original sample statement; Based on the similarity sample detection model, the target statement is compared with each of the target original sample statements to detect erroneously labeled sample statements similar to the target statement from the sample statement library; the similarity sample detection model is a Siamese neural network model, which includes two basic networks, each of which is used to process an input statement; The sample statement library is updated by performing sample statement correction processing based on the erroneously marked sample statements. The updated sample statement library is used to train an intent classification model. The intent classification model is used to identify the intent of user input statements in a chatbot dialogue scenario and instruct the chatbot to respond based on the identified intent.

2. The method according to claim 1, characterized in that, The method also includes a training step for a similar sample detection model; The training steps of the similar sample detection model include: Select basic sample statements from the set of original sample statements; From the set of original sample statements, obtain the original sample statements that meet the preset similarity conditions with the basic sample statements, and use them as the similar sample statements corresponding to the basic sample statements. From the similar sample statements corresponding to each basic sample statement, filter out similar sample statements that have the same intent label as the basic sample statement, and take the basic sample statement and the filtered similar sample statements as similar sample pairs; The similar sample detection model is trained based on the similar sample pairs.

3. The method according to claim 1, characterized in that, The step of correcting the sample statement library based on the error-marked sample statements to update the sample statement library includes: If a revision operation is detected for the error-marked sample statement, then the intent tag for revising the error-marked sample statement in the sample statement library is triggered; If a deletion operation is detected for the error-marked sample statement, the error-marked sample statement is deleted from the sample statement library.

4. The method according to claim 3, characterized in that, After the step of triggering the revision of the intent tag of the error-marked sample statement in the sample statement library if a revision operation for the error-marked sample statement is detected, the method further includes: Identify historical statements similar to the target statement from the chatbot's historical conversation records with users; For the historical statements, add the revised intent tags corresponding to the error-marked sample statements; The historical statements with the revised intent tags added will be added as sample statements to the sample statement library.

5. The method according to claim 1, characterized in that, The target statement is a user input statement generated in the chatbot's historical conversations with the user; The method further includes: Determine the first context statement corresponding to the target statement in its target historical session; Identify a second context statement that is similar to the first context statement from the session records of non-target historical sessions; From the session records of the non-target historical sessions, determine the reference statement located between the second context statements and marked with the correct intent label; If the reference statement and the target statement meet a preset similarity condition, then the correct intent tag of the reference statement is added to the target statement, and the target statement with the correct intent tag is added to the sample statement library.

6. A sample statement processing device, characterized in that, The device includes: The acquisition module is used to acquire target statements with incorrect intent tags; the target statements are marked with incorrect intent tags; the incorrect intent tags are intent tags that do not correspond to the intent of the statement itself; A detection module is used to input the target statement into a trained similar sample detection model. The similar sample detection model is trained based on a set of original sample statements labeled with intent tags in a sample statement library. The module compares the intent tags labeled for each of the original sample statements with the erroneous intent tags labeled for the target statement. The original sample statements corresponding to intent tags identical to the erroneous intent tags are taken as target original sample statements. Based on the similar sample detection model, the target statement is compared with each of the target original sample statements to detect erroneously labeled sample statements similar to the target statement from the sample statement library. The similar sample detection model is a Siamese neural network model, comprising two basic networks, each of which processes one input statement. The correction module is used to perform sample statement correction processing on the sample statement library based on the erroneously marked sample statements, so as to update the sample statement library; the updated sample statement library is used to train an intent classification model; the intent classification model is used to identify the intent of the user input statement in the robot dialogue scenario and instruct the robot to respond based on the identified intent.

7. The apparatus according to claim 6, characterized in that, The acquisition module is also used for: Select basic sample statements from the set of original sample statements; From the set of original sample statements, obtain the original sample statements that meet the preset similarity conditions with the basic sample statements, and use them as the similar sample statements corresponding to the basic sample statements. From the similar sample statements corresponding to each basic sample statement, filter out similar sample statements that have the same intent label as the basic sample statement, and take the basic sample statement and the filtered similar sample statements as similar sample pairs; The similar sample detection model is trained based on the similar sample pairs.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images