A method, device and equipment for training a multi-turn dialogue intent recognition model

Abstract

The application is suitable for the technical field of computer application, and provides a training method, device and equipment of a multi-round dialogue intention recognition model, which comprises the following steps: preprocessing sample texts to obtain first sentence information of sample sentences contained in the sample texts; inputting the first sentence information of the sample sentences into a multi-round dialogue intention recognition model for recognition processing to obtain sample recognition results corresponding to the sample texts; calculating a target loss value according to the sample recognition results corresponding to the sample texts, result labels corresponding to the sample texts and a preset loss function; and if the target loss value meets a preset termination condition, outputting a trained multi-round dialogue intention recognition model. The above method divides sentences into finer granularity for recognition in a single-round information, more accurately represents intentions in a dialogue process, increases subsequent obtained information, and improves the accuracy of the result of intention recognition.

Description

Technical Field

[0001] This application belongs to the field of computer application technology, and in particular relates to a training method, apparatus and equipment for a multi-turn dialogue intent recognition model. Background Technology

[0002] In chat systems, a turn of the conversation is generally defined as the period from when one party sends a message to when others send messages. Currently, common methods for dialogue intent recognition use neural network models. However, during training, the samples typically consist of one turn of dialogue and its corresponding semantic meaning. Therefore, the trained dialogue intent recognition model only outputs one semantic meaning per turn of dialogue. During a conversation, a single turn may contain more than one semantic meaning, and these multiple semantic meanings may imply the need for various subsequent responses.

[0003] Dialogue intent recognition models trained using existing model training methods may discard some intent information in dialogues during practical applications, increasing the difficulty of intent recognition. At the same time, the reduced information obtained subsequently leads to a decrease in the accuracy of intent recognition results. Summary of the Invention

[0004] This application provides a training method, apparatus, and device for a multi-turn dialogue intent recognition model, which can solve the above-mentioned technical problems.

[0005] In a first aspect, embodiments of this application provide a training method for a multi-turn dialogue intent recognition model. The method includes: acquiring a training sample set; the training sample set includes multi-turn dialogues and result labels corresponding to sample texts included in the multi-turn dialogues; preprocessing the sample texts to obtain first clause information of sample clauses contained in the sample texts; the first clause information includes the previous round sample text of the sample text, the sample clauses, and the sequential number of the sample clauses; inputting the first clause information of the sample clauses into the multi-turn dialogue intent recognition model for recognition processing to obtain the sample recognition result corresponding to the sample texts; wherein... The multi-turn dialogue intent recognition model includes a sample pre-trained language model, a sample inter-clause language model, a sample autoregressive model, and a sample attention mechanism model. A target loss value is calculated based on the sample recognition result corresponding to the sample text, the result label corresponding to the sample text, and a preset loss function. If the target loss value does not meet a preset termination condition, the multi-turn dialogue intent recognition model is updated based on the target loss value, and the process returns to inputting the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing. If the target loss value meets the preset termination condition, the trained multi-turn dialogue intent recognition model is output.

[0006] Further, the step of inputting the first clause information of the sample clause into a multi-turn dialogue intent recognition model for recognition processing to obtain the sample recognition result corresponding to the sample text includes: inputting the first clause information of the sample clause into a sample pre-trained language model for processing to obtain the first feature information and the first correlation relationship of the sample clause; the first correlation relationship represents the relationship between the sample clause and the previous round sample text; inputting the first feature information and the first correlation relationship of the sample clause into a sample clause inter-language model for processing to obtain the second feature information and the second correlation relationship of the sample text; the second correlation relationship is used to identify the relationship between the sample clauses; obtaining the previous round sample dialogue state information, and inputting the previous round sample dialogue state information, the second feature information and the second correlation relationship into a sample autoregressive model for processing to obtain the current round sample dialogue state information and the third feature information of the sample text; inputting the first correlation relationship, the second correlation relationship and the third feature information into a sample attention mechanism model for processing to obtain the fourth feature information of the sample clause; and determining the sample recognition result corresponding to the sample text based on the third feature information and the fourth feature information.

[0007] Furthermore, the sample recognition result includes the overall sentence intent and the sentence intent; determining the sample recognition result corresponding to the sample text based on the third feature information and the fourth feature information includes: processing the third feature information according to a preset overall sentence task processing strategy to obtain the overall sentence intent of the sample text; and processing the fourth feature information according to a preset sentence task processing strategy to obtain the sentence intent of the sample text.

[0008] Further, the preprocessing of the sample text to obtain the first clause information of the sample clauses contained in the sample text includes: segmenting the sample text into clauses according to a preset clause segmentation strategy to obtain the sample clauses contained in the sample text, and obtaining the sequence number of the sample clauses; determining the first clause information of the sample clauses contained in the sample text based on the previous round of sample text, the sample clauses, and the sequence number of the sample clauses.

[0009] Secondly, embodiments of this application provide a multi-turn dialogue intent recognition method, comprising: acquiring a target text to be recognized and its previous-turn target text; preprocessing the target text to obtain second clause information of a target clause contained in the target text; the second clause information including the previous-turn text of the target text, the text clause, and the sequential number of the text clause; inputting the second clause information into a multi-turn dialogue intent recognition model for recognition processing to obtain an intent recognition result corresponding to the target text; wherein, the multi-turn dialogue intent recognition model is trained using the training method of the multi-turn dialogue intent recognition model described in the first aspect above; and filtering the intent recognition result according to preset filtering conditions to obtain a prediction result corresponding to the target text.

[0010] Further, the multi-turn dialogue intent recognition model includes a target pre-trained language model, a sample clause inter-language model, a target autoregressive model, and a target attention mechanism model; the step of inputting the second clause information into the multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text includes: inputting the second clause information into the target pre-trained language model for processing to obtain the fifth feature information and the third correlation relationship of the target clause; the third correlation relationship represents the relationship between the target clause and the previous round target text; inputting the fifth feature information and the third correlation relationship of the target clause into the target clause inter-language model for processing to obtain... The target text is analyzed using its sixth feature information and fourth correlation relationship. The fourth correlation relationship is used to identify the relationship between the target clauses. The previous round of target dialogue state information is obtained, and the previous round of target dialogue state information, the sixth feature information, and the fourth correlation relationship are input into the target autoregressive model for processing to obtain the current round of target dialogue state information and the seventh feature information of the target text. The third correlation relationship, the fourth correlation relationship, and the seventh feature information are input into the target attention mechanism model for processing to obtain the eighth feature information of the target clause. The intent recognition result corresponding to the target text is determined based on the seventh feature information and the eighth feature information.

[0011] Thirdly, embodiments of this application provide a training apparatus for a multi-turn dialogue intent recognition model, comprising: an acquisition unit for acquiring a training sample set; the training sample set includes multi-turn dialogues and result labels corresponding to sample texts included in the multi-turn dialogues; a first processing unit for preprocessing the sample texts to obtain first clause information of sample clauses contained in the sample texts; the first clause information includes the previous round sample text of the sample text, the sample clauses, and the sequential number of the sample clauses; and a second processing unit for inputting the first clause information of the sample clauses into the multi-turn dialogue intent recognition model for recognition processing to obtain sample recognition results corresponding to the sample texts; wherein, the The multi-turn dialogue intent recognition model includes a sample pre-trained language model, a sample inter-clause language model, a sample autoregressive model, and a sample attention mechanism model; a calculation unit is used to calculate a target loss value based on the sample recognition result corresponding to the sample text, the result label corresponding to the sample text, and a preset loss function; a third processing unit is used to update the multi-turn dialogue intent recognition model based on the target loss value if the target loss value does not meet the preset termination condition, and return to execute the input of the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing; a fourth processing unit is used to output the trained multi-turn dialogue intent recognition model if the target loss value meets the preset termination condition.

[0012] Further, the second processing unit is specifically configured to: input the first clause information of the sample clause into a sample pre-trained language model for processing to obtain the first feature information and the first correlation relationship of the sample clause; the first correlation relationship represents the relationship between the sample clause and the previous round of sample text; input the first feature information and the first correlation relationship of the sample clause into a sample clause inter-language model for processing to obtain the second feature information and the second correlation relationship of the sample text; the second correlation relationship is used to identify the relationship between the sample clauses; obtain the previous round of sample dialogue state information, and input the previous round of sample dialogue state information, the second feature information and the second correlation relationship into a sample autoregressive model for processing to obtain the current round of sample dialogue state information and the third feature information of the sample text; input the first correlation relationship, the second correlation relationship and the third feature information into a sample attention mechanism model for processing to obtain the fourth feature information of the sample clause; and determine the sample recognition result corresponding to the sample text based on the third feature information and the fourth feature information.

[0013] Furthermore, the second processing unit is specifically used to: process the third feature information according to a preset overall sentence task processing strategy to obtain the overall sentence intent of the sample text; and process the fourth feature information according to a preset sentence segmentation task processing strategy to obtain the sentence segmentation intent of the sample text.

[0014] Further, the first processing unit is specifically used to: perform sentence segmentation processing on the sample text according to a preset sentence segmentation strategy to obtain sample clauses contained in the sample text, and obtain the sequence number of the sample clauses; determine the first clause information of the sample clauses contained in the sample text according to the previous round of sample text, the sample clauses and the sequence number of the sample clauses.

[0015] Fourthly, embodiments of this application provide a multi-turn dialogue intent recognition device, comprising: an acquisition unit for acquiring a target text to be recognized and its previous-turn target text; a first processing unit for preprocessing the target text to obtain second clause information of a target clause contained in the target text; the second clause information including the previous-turn text of the target text, the text clause, and the sequential number of the text clause; a second processing unit for inputting the second clause information into a multi-turn dialogue intent recognition model for recognition processing to obtain an intent recognition result corresponding to the target text; wherein the multi-turn dialogue intent recognition model is trained using the training method of the multi-turn dialogue intent recognition model described in the first aspect; and a third processing unit for filtering the intent recognition result according to preset filtering conditions to obtain a prediction result corresponding to the target text.

[0016] Further, the multi-turn dialogue intent recognition model includes a target pre-trained language model, a sample clause inter-language model, a target autoregressive model, and a target attention mechanism model; the second processing unit is specifically used for: inputting the second clause information into the target pre-trained language model for processing to obtain the fifth feature information and the third correlation relationship of the target clause; the third correlation relationship represents the relationship between the target clause and the previous round target text; inputting the fifth feature information and the third correlation relationship of the target clause into the target clause inter-language model for processing to obtain the sixth feature information and the fourth correlation relationship of the target text; the fourth correlation relationship is used to identify the relationship between the target clauses; obtaining the previous round target dialogue state information, and inputting the previous round target dialogue state information, the sixth feature information, and the fourth correlation relationship into the target autoregressive model for processing to obtain the current round target dialogue state information and the seventh feature information of the target text; inputting the third correlation relationship, the fourth correlation relationship, and the seventh feature information into the target attention mechanism model for processing to obtain the eighth feature information of the target clause; and determining the intent recognition result corresponding to the target text based on the seventh feature information and the eighth feature information.

[0017] Fifthly, embodiments of this application provide a training device for a multi-turn dialogue intent recognition model, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method described in the first aspect above.

[0018] In a sixth aspect, embodiments of this application provide a multi-turn dialogue intent recognition device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the second aspect above.

[0019] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0020] Eighthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the second aspect above.

[0021] On one hand, in this embodiment, a training sample set is obtained; the sample text is preprocessed to obtain the first clause information of the sample clause contained in the sample text; the first clause information of the sample clause is input into a multi-turn dialogue intent recognition model for recognition processing to obtain the sample recognition result corresponding to the sample text; wherein, the multi-turn dialogue intent recognition model includes a sample pre-trained language model, a sample clause inter-language model, a sample autoregressive model, and a sample attention mechanism model; a target loss value is calculated based on the sample recognition result corresponding to the sample text, the result label corresponding to the sample text, and a preset loss function; if the target loss value does not meet the preset termination condition, the multi-turn dialogue intent recognition model is updated based on the target loss value, and the process returns to inputting the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing; if the target loss value meets the preset termination condition, the trained multi-turn dialogue intent recognition model is output. In the training method of the aforementioned multi-turn dialogue intent recognition model, sentences are divided into finer granularities for recognition within single-turn information. The previous turn's sample text and sample clauses are used as input to the multi-turn dialogue intent recognition model. Simultaneously, a structure consisting of a sample pre-trained language model, a sample inter-clause language model, a sample autoregressive model, and a sample attention mechanism model is employed to integrate information from multiple turns of dialogue, maximizing the representation of intents during the dialogue process, increasing the amount of information acquired subsequently, and improving the accuracy of intent recognition results. Furthermore, the presence of multiple intents within the same text can perturb intent recognition; finer-grained recognition can reduce the instability caused by such perturbations.

[0022] On one hand, in this embodiment, the target text to be identified and its previous target text are obtained; the target text is preprocessed to obtain the second clause information of the target clause contained in the target text; the second clause information is input into a multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text; wherein, the multi-turn dialogue intent recognition model is trained by the training method of the multi-turn dialogue intent recognition model in the first aspect described above; the intent recognition result is filtered according to preset filtering conditions to obtain the prediction result corresponding to the target text. The multi-turn dialogue intent recognition method described above uses a multi-turn dialogue intent recognition model trained by the training method of the multi-turn dialogue intent recognition model in the first aspect described above. In single-turn information, sentences are divided into finer granularities for recognition. The previous target text and target clause are used as inputs to the multi-turn dialogue intent recognition model. At the same time, a structure consisting of a target pre-trained language model, a target inter-clause language model, a target autoregressive model, and a target attention mechanism model is adopted to integrate information in multi-turn dialogue, to represent the intent in the dialogue process to the greatest extent, to increase the information obtained subsequently, and to improve the accuracy of the intent recognition result. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic flowchart illustrating a training method for a multi-turn dialogue intent recognition model provided in the first embodiment of this application;

[0025] Figure 2 This is a schematic flowchart of step S102 in the training method of a multi-turn dialogue intent recognition model provided in the first embodiment of this application;

[0026] Figure 3 This is a schematic flowchart of step S103 in the training method of a multi-turn dialogue intent recognition model provided in the first embodiment of this application;

[0027] Figure 4 This is a schematic flowchart of step S1035 in the training method of a multi-turn dialogue intent recognition model provided in the first embodiment of this application;

[0028] Figure 5 This is a schematic flowchart of a multi-turn dialogue intent recognition method provided in the second embodiment of this application;

[0029] Figure 6This is a schematic flowchart of step S203 in a multi-turn dialogue intent recognition method provided in the second embodiment of this application;

[0030] Figure 7 This is a schematic diagram of the training device for the multi-turn dialogue intent recognition model provided in the third embodiment of this application;

[0031] Figure 8 This is a schematic diagram of the multi-turn dialogue intent recognition device provided in the fourth embodiment of this application;

[0032] Figure 9 This is a schematic diagram of the training device for the multi-turn dialogue intent recognition model provided in the fifth embodiment of this application;

[0033] Figure 10 This is a schematic diagram of a multi-turn dialogue intent recognition device provided in the sixth embodiment of this application. Detailed Implementation

[0034] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0035] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0036] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0037] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0038] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0039] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0040] In dialogue systems, the purpose of dialogue intent recognition is to enable the machine to understand the input text or classify it into a certain type of task, and to provide the recognition results to subsequent modules to provide a basis for the subsequent processing of the dialogue system.

[0041] Existing dialogue intent recognition models typically identify intents within a single turn of dialogue. The specific techniques involve concatenating the preceding and current dialogue contexts or extracting sentence vectors from each context and feeding them into a language model. The final intent prediction is then obtained by fitting the target task. However, for the method of extracting sentence vectors separately, a large amount of contextual information is generally required for fitting to obtain good results. Therefore, the dialogue concatenation option is currently the most commonly used. In actual use of dialogue systems, some dialogue initiators may express multiple intents within a single turn of dialogue, and these intents may involve multiple responses. This is a deficiency in existing dialogue intent recognition models.

[0042] This phenomenon is generally more pronounced in Chinese. In some Chinese conversations, a single sentence can carry multiple intentions, often separated by commas. For example, in a chat system, a user might type, "I'm so annoyed today, can you find me some interesting news?" Within this single sentence, the user simultaneously expresses an expression, a command, and a question. Existing models that recognize commands on a per-round basis typically only identify one type of instruction. Furthermore, the influence of other instructions can cause instability in the entire recognition model. Even models that can recognize multiple categories on a per-round basis struggle to clearly define which data led to the final recognition result. This increases the amount of data required for model convergence and generalization, and is also susceptible to the semantic influence of other sentences.

[0043] Therefore, this application proposes a training method for a multi-turn dialogue intent recognition model.

[0044] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a training method for a multi-turn dialogue intent recognition model provided in the first embodiment of this application. In this embodiment, the execution subject of the training method for a multi-turn dialogue intent recognition model is a device with training capabilities for a multi-turn dialogue intent recognition model. This training device can be a personal computer, a server, a processor, a microprocessor, etc. This embodiment uses the training device for the multi-turn dialogue intent recognition model (hereinafter referred to as the device) as the execution subject for the following explanation, without specifically limiting the device. Figure 1 The training methods for the multi-turn dialogue intent recognition model shown may include:

[0045] S101: Obtain the training sample set; the training sample set includes multi-turn dialogues and the result labels corresponding to the sample texts included in the multi-turn dialogues.

[0046] Before explaining step S101, let's briefly explain some terms used in this application. Multi-turn dialogue refers to a continuous dialogue based on context, aimed at solving a specific type of task. Generally, a turn of a message from one party to the moment another party begins to send a message is defined as one turn of that message sender.

[0047] The device acquires a training sample set, which includes multi-turn dialogues and the corresponding result labels for the sample text included in each multi-turn dialogue. The sample text included in a multi-turn dialogue refers to the text of a single-turn dialogue. For example, the sample text could be, "I'm so annoyed today, can you find me some interesting news?"

[0048] This application does not restrict the dialogue format for multi-turn conversations. For example, it can be human-computer dialogue, telephone voice, or text communication on the customer service interface.

[0049] This application does not restrict the method of obtaining dialogue text. For example, dialogue text in text form can be obtained, or it can be obtained through speech recognition.

[0050] The result label corresponding to the sample text is the result of intent recognition for the sample text, which can be manually labeled.

[0051] Understandably, the more samples in the training sample set and the more diverse the types of samples, the higher the accuracy of the multi-turn dialogue intent recognition model trained.

[0052] S102: Preprocess the sample text to obtain the first clause information of the sample clause contained in the sample text; the first clause information includes the previous round of sample text of the sample text, the sample clause, and the sequential number of the sample clause.

[0053] The device preprocesses the sample text to obtain the first clause information of the sample clauses contained in the sample text. The purpose of this preprocessing is to divide the sentence into finer granularities, and to eliminate some of the noise caused by input or language habits, or to transform the input into the state required by the deep learning model.

[0054] The device preprocesses the sample text to extract sample clauses. These sample clauses are then combined with the previous round of sample text to obtain the first clause information, which includes the previous round of sample text, the sample clause itself, and its sequential number.

[0055] When training the model, the information of the first clause composed of a single sample clause and the sample text of the previous round is used. This is equivalent to obtaining the meaning representation of each sample clause contained in the sample text. When performing intent recognition, the intent of each sample clause is also recognized separately, which reduces the interference between different sample clauses in the same sample text and increases the stability of recognition.

[0056] In an optional implementation, to obtain more accurate first clause information, thereby further reducing interference between different sample clauses in the same sample text and increasing the stability of recognition, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic flowchart of step S102 in the training method of a multi-turn dialogue intent recognition model provided in the first embodiment of this application. Step S102 includes:

[0057] S1021: The sample text is segmented according to a preset sentence segmentation strategy to obtain the sample clauses contained in the sample text, and the sequential number of the sample clauses is obtained.

[0058] S1022: Determine the first clause information of the sample clause contained in the sample text based on the previous round of sample text, the sample clause, and the sequential number of the sample clause.

[0059] The device pre-stores preset sentence segmentation strategies. It then segments the sample text according to these strategies to obtain sample clauses. The more precise the segmented sample clauses, the more accurate the information obtained from the first clause.

[0060] When processing clauses, they can be divided into different clauses by punctuation marks. Consecutive punctuation marks are treated as a single punctuation mark. For example, in some cases, an ellipsis will be written as three periods.

[0061] In addition, when processing sentences, redundant characters or emoticons can be filtered out and deleted.

[0062] S103: Input the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing to obtain the sample recognition result corresponding to the sample text; wherein, the multi-turn dialogue intent recognition model includes a sample pre-trained language model, a sample inter-clause language model, a sample autoregressive model and a sample attention mechanism model.

[0063] In this embodiment, the multi-turn dialogue intent recognition model includes a sample pre-trained language model, a sample inter-clause language model, a sample autoregressive model, and a sample attention mechanism model.

[0064] The sample pre-trained language model is used to extract the relationship between the sample clause and the previous round of sample text. For example, when the sample clause answers the question in the previous round of sample text, the relationship between the sample clause and the previous round of sample text is considered to be related.

[0065] In this embodiment, the sample pre-trained language model can be pre-trained and then ported to the multi-turn dialogue intent recognition model.

[0066] The language model between sample clauses is used to extract the relationship between different sample clauses. For example, in the sentence "So-and-so's height is not 190cm, but 226cm", the two sample clauses "So-and-so's height is not 190cm" and "but 226cm" together constitute a correction intention, that is, the two sample clauses "So-and-so's height is not 190cm" and "but 226cm" are related.

[0067] Sample autoregressive models are used to extract features from sample text based on historical information. These models are infinitely scalable, allowing for adaptive recording and forgetting of information during the dialogue process. This prevents the model from failing to obtain sufficient information to make a correct judgment when the dialogue response is from an earlier round than the previous one.

[0068] The sample attention mechanism model is set at the end of the previous model and is used to extract the feature information of the sample clauses again.

[0069] The information of the first clause of the sample text is input into a multi-turn dialogue intent recognition model for processing, resulting in the sample recognition result corresponding to the sample text. The first clause information is input into the multi-turn dialogue intent recognition model, and after processing by a sample pre-trained language model, a sample inter-clause language model, a sample autoregressive model, and a sample attention mechanism model, the sample recognition result corresponding to the sample text is obtained.

[0070] Specifically, to explain in detail the internal processing of the multi-turn dialogue intent recognition model, please participate in... Figure 3 , Figure 3 This is a schematic flowchart of step S103 in the training method of a multi-turn dialogue intent recognition model provided in the first embodiment of this application. Step S103 includes:

[0071] S1031: Input the first clause information of the sample clause into the sample pre-trained language model for processing to obtain the first feature information and the first correlation relationship of the sample clause; the first correlation relationship represents the relationship between the sample clause and the previous round of sample text.

[0072] The device inputs the first clause information of the sample clause into the sample pre-trained language model for processing, obtaining the first feature information and the first relevance of the sample clause. The sample pre-trained language model then extracts the features of a single clause and whether it can form an interaction relationship with the previous text.

[0073] The first feature information of the sample clause can be understood as a feature vector, that is, a multi-dimensional floating-point number.

[0074] The first relevance relation represents the relationship between the sample clause and the sample text of the previous round. The first relevance relation can include whether it is related or not.

[0075] In this embodiment, the number of sample pre-trained language models can be one or more. When multiple sample pre-trained language models are set, the first clause information of multiple sample clauses is input into different sample pre-trained language models for processing.

[0076] S1032: Input the first feature information and the first correlation relationship of the sample clause into the language model between sample clauses for processing to obtain the second feature information and the second correlation relationship of the sample text; the second correlation relationship is used to identify the relationship between the sample clauses.

[0077] The device takes the first feature information and the first correlation relationship of the sample clause as input, and processes them into the language model between sample clauses to obtain the second feature information and the second correlation relationship of the sample text.

[0078] Since the second feature information of the sample text is extracted based on the first feature information and the first relevance of the sample clause, the second feature information of the sample text is the full sentence feature extracted based on the clause.

[0079] In this embodiment, multiple first feature information and multiple first correlations of multiple sample clauses are uniformly put into a language model between sample clauses to learn the possible interactions between clauses and obtain the second correlation.

[0080] Taking the example of "So-and-so's height is not 190cm, but 226cm" above, the two sample clauses "So-and-so's height is not 190cm" and "but 226cm" together constitute a corrective intent, which cannot be predicted by a single clause. Furthermore, directly predicting using the entire sentence requires a large amount of data for the model to learn that the interaction between these two clauses causes this intent. Therefore, a language model between sample clauses is set up here, which can directly learn that the two sample clauses "So-and-so's height is not 190cm" and "but 226cm" are related. While ensuring the ability to learn the corresponding knowledge, artificial knowledge is introduced through manually designed structures to reduce the learning cost of the model.

[0081] S1033: Obtain the previous round sample dialogue state information, and input the previous round sample dialogue state information, the second feature information and the second correlation into the sample autoregressive model for processing to obtain the current round sample dialogue state information and the third feature information of the sample text.

[0082] The device acquires the dialogue state information from the previous round of sample dialogues. This information is the output of the autoregressive model during the previous training round. The dialogue state information from the previous round includes historical dialogue information; that is, it includes not only the dialogue from the previous round but also dialogue from earlier rounds.

[0083] The device inputs the previous round of sample dialogue state information, second feature information, and second correlation into the sample autoregressive model for processing, to obtain the current round of sample dialogue state information and the third feature information of the sample text.

[0084] The current sample dialogue state information can be used as input for the sample autoregressive model during the next iteration of training.

[0085] Since the third feature information of the sample text is extracted based on the previous round of sample dialogue state information, the second feature information, and the second relevance relationship, the third feature information of the sample text is the full sentence feature extracted after fusing historical information and clause features.

[0086] S1034: Input the first correlation, the second correlation, and the third feature information into the sample attention mechanism model for processing to obtain the fourth feature information of the sample clause.

[0087] The sample attention mechanism model is used to extract features again from the first, second, and third relevance relationships, resulting in the fourth feature information of the sample clause. Based on this fourth feature information, the results of different dialogue intent tasks can be determined.

[0088] S1035: Determine the sample recognition result corresponding to the sample text based on the third feature information and the fourth feature information.

[0089] The device determines the sample recognition result corresponding to the sample text based on the third and fourth feature information. For intents that may appear at the clause level, the device predicts them using the features at the clause level; for intents that appear in the whole sentence, the device predicts them using the features of the whole sentence.

[0090] Prediction is made using features of the whole sentence. In the prediction of the whole sentence, the overall structure of the model is almost equivalent to the existing model and will not affect the existing capabilities. At the same time, the robustness of the model is enhanced by adding more granular information.

[0091] In an optional implementation, the sample identification results include overall sentence intent and clause intent. See also... Figure 4 , Figure 4 This is a schematic flowchart of step S1035 in the training method of a multi-turn dialogue intent recognition model provided in the first embodiment of this application. Step S1035 includes:

[0092] S10351: Process the third feature information according to the preset total sentence task processing strategy to obtain the total sentence intent of the sample text.

[0093] S10352: The fourth feature information is processed according to the preset sentence segmentation task processing strategy to obtain the sentence segmentation intent of the sample text.

[0094] The device can be pre-configured with task layers, including a general-level task layer and a sentence-level task layer. The general-level task layer has a pre-set general-sentence task processing strategy, allowing the device to process the third feature information according to this strategy to obtain the general-sentence intent of the sample text. Similarly, the sentence-level task layer has a pre-set general-sentence task processing strategy, allowing the device to process the fourth feature information according to this strategy to obtain the sentence-level intent of the sample text.

[0095] In addition, other tasks can be set in the task layer. Existing models often suffer from insufficient data in some task domains when used in real-world scenarios, which often requires manual annotation. The model used in this embodiment can extract deeper contextual semantic information by using the relatively abstract task of dialogue intent as a basis, and can provide semantic support for other tasks, reducing the amount of domain data required in actual use.

[0096] S104: Calculate the target loss value based on the sample recognition result corresponding to the sample text, the result label corresponding to the sample text, and the preset loss function.

[0097] The device stores a preset loss function in advance, which is used to calculate the target loss value, that is, the difference between the sample recognition result corresponding to the sample text and the result label corresponding to the sample text.

[0098] The device calculates the target loss value based on the sample recognition result corresponding to the sample text, the result label corresponding to the sample text, and the preset loss function. The calculated target loss value is used to determine whether to stop training.

[0099] It should be noted that the preset loss function can be the cross-entropy loss function, and there are no restrictions here.

[0100] S105: If the target loss value does not meet the preset termination condition, the multi-turn dialogue intent recognition model is updated according to the target loss value, and the process is returned to input the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing.

[0101] The device has pre-stored preset termination conditions. If the target loss value does not meet the preset termination conditions, the multi-turn dialogue intent recognition model is updated according to the target loss value. The learning parameters of the model are updated, and the process returns to input the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing, and the model is iteratively trained.

[0102] S106: If the target loss value meets the preset termination condition, then output the trained multi-turn dialogue intent recognition model.

[0103] If the target loss value meets the preset termination condition, the trained multi-turn dialogue intent recognition model is output.

[0104] In this embodiment, a training sample set is obtained; the sample text is preprocessed to obtain the first clause information of the sample clauses contained in the sample text; the first clause information of the sample clauses is input into a multi-turn dialogue intent recognition model for recognition processing to obtain the sample recognition result corresponding to the sample text; wherein, the multi-turn dialogue intent recognition model includes a sample pre-trained language model, a sample clause inter-language model, a sample autoregressive model, and a sample attention mechanism model; a target loss value is calculated based on the sample recognition result corresponding to the sample text, the result label corresponding to the sample text, and a preset loss function; if the target loss value does not meet the preset termination condition, the multi-turn dialogue intent recognition model is updated based on the target loss value, and the process returns to inputting the first clause information of the sample clauses into the multi-turn dialogue intent recognition model for recognition processing; if the target loss value meets the preset termination condition, the trained multi-turn dialogue intent recognition model is output. In the training method of the aforementioned multi-turn dialogue intent recognition model, sentences are divided into finer granularities for recognition within single-turn information. The previous turn's sample text and sample clauses are used as input to the multi-turn dialogue intent recognition model. Simultaneously, a structure consisting of a sample pre-trained language model, a sample inter-clause language model, a sample autoregressive model, and a sample attention mechanism model is employed to integrate information from multiple turns of dialogue, maximizing the representation of intents during the dialogue process, increasing the amount of information acquired subsequently, and improving the accuracy of intent recognition results. Furthermore, the presence of multiple intents within the same text can perturb intent recognition; finer-grained recognition can reduce the instability caused by such perturbations.

[0105] Please see Figure 5 , Figure 5 This is a schematic flowchart illustrating a multi-turn dialogue intent recognition method provided in the second embodiment of this application. In this embodiment, the executing entity of the multi-turn dialogue intent recognition method is a device with multi-turn dialogue intent recognition functionality. This device can be a personal computer, a server, a processor, a microprocessor, etc. The following explanation uses a multi-turn dialogue intent recognition device (hereinafter referred to as "device") as the executing entity for the multi-turn dialogue intent recognition method in this application embodiment, without specifically limiting the device. Figure 5 The multi-turn dialogue intent recognition method shown may include:

[0106] S201: Obtain the target text to be identified and its previous target text.

[0107] The device acquires the target text to be identified and the target text of the previous round. The target text to be identified is the message text of one round in a multi-round dialogue.

[0108] S202: Preprocess the target text to obtain the second clause information of the target clause contained in the target text; the second clause information includes the previous text of the target text, the text clause, and the sequential number of the text clause.

[0109] The device preprocesses the target text to obtain the second clause information of the target clause contained in the target text. The second clause information includes the previous text of the target text, the text clause, and the sequential number of the text clause.

[0110] For specific details, please refer to the relevant description of the preprocessing of sample text in S102 of the first embodiment, which will not be repeated here.

[0111] S203: Input the second clause information into the multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text; wherein, the multi-turn dialogue intent recognition model is trained by the training method of the multi-turn dialogue intent recognition model described in the first embodiment.

[0112] The device inputs the second clause information into the multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text; wherein, the multi-turn dialogue intent recognition model is trained by the training method of the multi-turn dialogue intent recognition model in the first embodiment.

[0113] Specifically, the training method and related structure of the multi-turn dialogue intent recognition model can be found in the first embodiment, and the specific details of S203 can be found in the relevant description in S103 of the first embodiment, which will not be repeated here.

[0114] Specifically, the multi-turn dialogue intent recognition model includes a target pre-trained language model, a sample clause inter-language model, a target autoregressive model, and a target attention mechanism model. Please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic flowchart of step S203 in a multi-turn dialogue intent recognition method provided in the second embodiment of this application. Step S203 includes:

[0115] S2031: Input the second clause information into the target pre-trained language model for processing to obtain the fifth feature information and the third correlation of the target clause; the third correlation represents the relationship between the target clause and the previous round of target text.

[0116] S2032: Input the fifth feature information and the third correlation of the target clause into the target clause language model for processing to obtain the sixth feature information and the fourth correlation of the target text; the fourth correlation is used to identify the relationship between the target clauses.

[0117] S2033: Obtain the target dialogue state information of the previous round, and input the target dialogue state information of the previous round, the sixth feature information and the fourth correlation into the target autoregressive model for processing to obtain the target dialogue state information of the current round and the seventh feature information of the target text.

[0118] S2034: Input the third correlation, the fourth correlation, and the seventh feature information into the target attention mechanism model for processing to obtain the eighth feature information of the target clause.

[0119] S2035: Determine the intent recognition result corresponding to the target text based on the seventh feature information and the eighth feature information.

[0120] Specifically, details of S2031 to S2035 can be found in the description of S1031 to S1035 in the first embodiment, and will not be repeated here.

[0121] S204: The intent recognition results are filtered according to preset filtering conditions to obtain the prediction results corresponding to the target text.

[0122] The device stores preset filtering conditions, which are used to filter the intent recognition results after post-processing. These conditions incorporate prior knowledge to prevent the model from predicting impossible outcomes, resulting in the final prediction of the target text. For example, since doubt and commands almost never appear together in the same round of dialogue, the device can restrict the prediction of doubt and commands from being simultaneously predicted in the same round.

[0123] In this embodiment, the target text to be identified and its previous target text are obtained; the target text is preprocessed to obtain the second clause information of the target clause contained in the target text; the second clause information is input into a multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text; wherein, the multi-turn dialogue intent recognition model is trained by the training method of the first aspect of the multi-turn dialogue intent recognition model described above; the intent recognition result is filtered according to preset filtering conditions to obtain the prediction result corresponding to the target text. The multi-turn dialogue intent recognition method described above uses a multi-turn dialogue intent recognition model trained by the training method of the first aspect of the multi-turn dialogue intent recognition model. In single-turn information, sentences are divided into finer granularities for recognition. The previous target text and target clause are used as inputs to the multi-turn dialogue intent recognition model. At the same time, a structure consisting of a target pre-trained language model, a target inter-clause language model, a target autoregressive model, and a target attention mechanism model is adopted to integrate information in multi-turn dialogue, to represent the intent in the dialogue process to the greatest extent, to increase the information obtained subsequently, and to improve the accuracy of the intent recognition result.

[0124] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0125] Please see Figure 7 , Figure 7 This is a schematic diagram of a training apparatus for a multi-turn dialogue intent recognition model provided in the third embodiment of this application. The included units are used for performing... Figures 1-4 The steps in the corresponding embodiments. Please refer to the details. Figures 1-4 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 7 The training device 7 for the multi-turn dialogue intent recognition model includes:

[0126] The acquisition unit 71 is used to acquire a training sample set; the training sample set includes multi-turn dialogues and the result labels corresponding to the sample texts included in the multi-turn dialogues;

[0127] The first processing unit 72 is used to preprocess the sample text to obtain the first clause information of the sample clause contained in the sample text; the first clause information includes the previous round of sample text of the sample text, the sample clause, and the sequential number of the sample clause;

[0128] The second processing unit 73 is used to input the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing to obtain the sample recognition result corresponding to the sample text; wherein, the multi-turn dialogue intent recognition model includes a sample pre-trained language model, a sample clause inter-language model, a sample autoregressive model and a sample attention mechanism model;

[0129] The calculation unit 74 is used to calculate the target loss value based on the sample recognition result corresponding to the sample text, the result label corresponding to the sample text, and the preset loss function;

[0130] The third processing unit 75 is used to update the multi-turn dialogue intent recognition model according to the target loss value if the target loss value does not meet the preset termination condition, and return to execute the input of the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing.

[0131] The fourth processing unit 76 is used to output the trained multi-turn dialogue intent recognition model if the target loss value meets the preset termination condition.

[0132] Furthermore, the second processing unit 73 is specifically used for:

[0133] The first clause information of the sample clause is input into the sample pre-trained language model for processing to obtain the first feature information and the first correlation relationship of the sample clause; the first correlation relationship represents the relationship between the sample clause and the previous round of sample text;

[0134] The first feature information and the first correlation relationship of the sample clauses are input into the language model between sample clauses for processing to obtain the second feature information and the second correlation relationship of the sample text; the second correlation relationship is used to identify the relationship between the sample clauses.

[0135] Obtain the previous round sample dialogue state information, and input the previous round sample dialogue state information, the second feature information, and the second correlation into the sample autoregressive model for processing to obtain the current round sample dialogue state information and the third feature information of the sample text;

[0136] The first correlation, the second correlation, and the third feature information are input into the sample attention mechanism model for processing to obtain the fourth feature information of the sample clause;

[0137] The sample recognition result corresponding to the sample text is determined based on the third feature information and the fourth feature information.

[0138] Furthermore, the second processing unit 73 is specifically used to: process the third feature information according to a preset overall sentence task processing strategy to obtain the overall sentence intent of the sample text; and process the fourth feature information according to a preset sentence segmentation task processing strategy to obtain the sentence segmentation intent of the sample text.

[0139] Further, the first processing unit 72 is specifically used to: perform sentence segmentation processing on the sample text according to a preset sentence segmentation strategy to obtain sample clauses contained in the sample text, and obtain the sequence number of the sample clauses; determine the first clause information of the sample clauses contained in the sample text according to the previous round of sample text, the sample clauses and the sequence number of the sample clauses.

[0140] Please see Figure 8 , Figure 8 This is a schematic diagram of a multi-turn dialogue intent recognition device provided in the fourth embodiment of this application. The included units are used for performing... Figures 5-6 The steps in the corresponding embodiments. Please refer to the details. Figures 5-6 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 8 The multi-turn dialogue intent recognition device 8 includes:

[0141] Acquisition unit 81 is used to acquire the target text to be identified and the target text of the previous round;

[0142] The first processing unit 82 is used to preprocess the target text to obtain the second clause information of the target clause contained in the target text; the second clause information includes the previous text of the target text, the text clause, and the sequential number of the text clause;

[0143] The second processing unit 83 is used to input the second clause information into a multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text; wherein, the multi-turn dialogue intent recognition model is trained by the training method of the multi-turn dialogue intent recognition model described in the first embodiment above;

[0144] The third processing unit 84 is used to filter the intent recognition results according to preset filtering conditions to obtain the prediction result corresponding to the target text.

[0145] Furthermore, the multi-turn dialogue intent recognition model includes a target pre-trained language model, a sample clause inter-language model, a target autoregressive model, and a target attention mechanism model; the second processing unit 83 is specifically used for:

[0146] The second clause information is input into the target pre-trained language model for processing to obtain the fifth feature information and the third correlation relationship of the target clause; the third correlation relationship represents the relationship between the target clause and the previous round of target text.

[0147] The fifth feature information and the third correlation of the target clause are input into the target clause language model for processing to obtain the sixth feature information and the fourth correlation of the target text; the fourth correlation is used to identify the relationship between the target clauses.

[0148] Obtain the target dialogue state information from the previous round, and input the target dialogue state information from the previous round, the sixth feature information, and the fourth correlation into the target autoregressive model for processing to obtain the target dialogue state information from the current round and the seventh feature information of the target text;

[0149] The third correlation, the fourth correlation, and the seventh feature information are input into the target attention mechanism model for processing to obtain the eighth feature information of the target clause;

[0150] The intent recognition result corresponding to the target text is determined based on the seventh feature information and the eighth feature information.

[0151] Please see Figure 9 , Figure 9 This is a schematic diagram of the training device for the multi-turn dialogue intent recognition model provided in the fifth embodiment of this application. Figure 9As shown, the training device 9 for the multi-turn dialogue intent recognition model in this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90, such as a training program for the multi-turn dialogue intent recognition model. When the processor 90 executes the computer program 92, it implements the steps in the above-described training method embodiments for various multi-turn dialogue intent recognition models, for example... Figure 1 Steps S101 to S106 are shown. Alternatively, when the processor 90 executes the computer program 92, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 7 The functions of the acquisition unit 71 to the evaluation unit 76 are shown.

[0152] For example, the computer program 92 can be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 92 in the training device 9 of the multi-turn dialogue intent recognition model. For example, the computer program 92 can be divided into an acquisition unit, a first processing unit, a second processing unit, a calculation unit, a third processing unit, and a fourth processing unit, with the specific functions of each unit as follows:

[0153] An acquisition unit is used to acquire a training sample set; the training sample set includes multi-turn dialogues and the result labels corresponding to the sample texts included in the multi-turn dialogues;

[0154] The first processing unit is used to preprocess the sample text to obtain the first clause information of the sample clause contained in the sample text; the first clause information includes the previous round of sample text of the sample text, the sample clause, and the sequential number of the sample clause;

[0155] The second processing unit is used to input the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing, and obtain the sample recognition result corresponding to the sample text; wherein, the multi-turn dialogue intent recognition model includes a sample pre-trained language model, a sample inter-clause language model, a sample autoregressive model and a sample attention mechanism model;

[0156] The calculation unit is used to calculate the target loss value based on the sample recognition result corresponding to the sample text, the result label corresponding to the sample text, and the preset loss function;

[0157] The third processing unit is used to update the multi-turn dialogue intent recognition model according to the target loss value if the target loss value does not meet the preset termination condition, and return to execute the input of the first clause information of the sample clause into the multi-turn dialogue intent recognition model for recognition processing.

[0158] The fourth processing unit is used to output the trained multi-turn dialogue intent recognition model if the target loss value meets the preset termination condition.

[0159] The training device 9 for the multi-turn dialogue intent recognition model may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will understand that... Figure 9 This is merely an example of the training device 9 for the multi-turn dialogue intent recognition model and does not constitute a limitation on the training device 9 for the multi-turn dialogue intent recognition model. It may include more or fewer components than shown, or combine certain components, or different components. For example, the training device 9 for the multi-turn dialogue intent recognition model may also include input / output devices, network access devices, buses, etc.

[0160] The processor 90 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0161] The memory 91 can be an internal storage unit of the training device 9 for the multi-turn dialogue intent recognition model, such as a hard disk or memory of the training device 9. The memory 91 can also be an external storage device of the training device 9, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the training device 9. Furthermore, the training device 9 for the multi-turn dialogue intent recognition model can include both internal storage units and external storage devices. The memory 91 is used to store the computer program and other programs and data required by the training device 9. The memory 91 can also be used to temporarily store data that has been output or will be output.

[0162] Please see Figure 10 , Figure 10 This is a schematic diagram of a multi-turn dialogue intent recognition device provided in the sixth embodiment of this application. Figure 10 As shown, the multi-turn dialogue intent recognition device 10 of this embodiment includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and executable on the processor 100, such as a multi-turn dialogue intent recognition program. When the processor 100 executes the computer program 102, it implements the steps in the various multi-turn dialogue intent recognition method embodiments described above, for example... Figure 5 The steps S201 to S204 are shown. Alternatively, when the processor 100 executes the computer program 102, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 8 The functions of the acquisition unit 81 to the evaluation unit 84 are shown.

[0163] For example, the computer program 102 can be divided into one or more modules / units, which are stored in the memory 101 and executed by the processor 100 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 102 in the multi-turn dialogue intent recognition device 10. For example, the computer program 102 can be divided into an acquisition unit, a first processing unit, a second processing unit, and a third processing unit, with the specific functions of each unit as follows:

[0164] The acquisition unit is used to acquire the target text to be identified and the target text of the previous round;

[0165] The first processing unit is used to preprocess the target text to obtain the second clause information of the target clause contained in the target text; the second clause information includes the previous text of the target text, the text clause, and the sequential number of the text clause;

[0166] The second processing unit is used to input the second clause information into a multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text; wherein, the multi-turn dialogue intent recognition model is trained by the training method of the multi-turn dialogue intent recognition model described in the first aspect above;

[0167] The third processing unit is used to filter the intent recognition results according to preset filtering conditions to obtain the prediction results corresponding to the target text.

[0168] The multi-turn dialogue intent recognition device 10 may include, but is not limited to, a processor 100 and a memory 101. Those skilled in the art will understand that... Figure 10 This is merely an example of a multi-turn dialogue intent recognition device 10 and does not constitute a limitation on the multi-turn dialogue intent recognition device 10. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the multi-turn dialogue intent recognition device 10 may also include input / output devices, network access devices, buses, etc.

[0169] The processor 100 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0170] The memory 101 can be an internal storage unit of the multi-turn dialogue intent recognition device 10, such as a hard disk or memory of the multi-turn dialogue intent recognition device 10. The memory 101 can also be an external storage device of the multi-turn dialogue intent recognition device 10, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the multi-turn dialogue intent recognition device 10. Furthermore, the multi-turn dialogue intent recognition device 10 can include both its internal storage unit and external storage devices. The memory 101 is used to store the computer program and other programs and data required by the multi-turn dialogue intent recognition device 10. The memory 101 can also be used to temporarily store data that has been output or will be output.

[0171] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0172] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0173] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0174] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0175] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0176] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0177] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0178] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0179] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0180] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for training a multi-turn dialogue intent recognition model, the method comprising: The method comprises: obtaining a training sample set; the training sample set comprises multiple rounds of dialogues and result labels corresponding to sample texts included in the multiple rounds of dialogues; preprocessing the sample texts to obtain first clause information of sample clauses included in the sample texts; the first clause information comprises a last round of sample texts of the sample texts, the sample clauses, and sequence numbers of the sample clauses; inputting the first clause information of the sample clauses into a multi-round dialogue intent recognition model for recognition processing to obtain sample recognition results corresponding to the sample texts; wherein the multi-round dialogue intent recognition model comprises a sample pre-training language model, a sample inter-clause language model, a sample self-recurrent model, and a sample attention mechanism model; calculating a target loss value according to the sample recognition results corresponding to the sample texts, result labels corresponding to the sample texts, and a preset loss function; if the target loss value does not satisfy a preset termination condition, updating the multi-round dialogue intent recognition model according to the target loss value, and returning to perform inputting the first clause information of the sample clauses into the multi-round dialogue intent recognition model for recognition processing; if the target loss value satisfies the preset termination condition, outputting a trained multi-round dialogue intent recognition model. 2.The method of claim 1, wherein, The method comprises: inputting the first clause information of the sample clauses into the sample pre-training language model for processing to obtain first feature information of the sample clauses and first correlation relationships; the first correlation relationships represent relationships between the sample clauses and the last round of sample texts; inputting the first feature information of the sample clauses and the first correlation relationships into the sample inter-clause language model for processing to obtain second feature information of the sample texts and second correlation relationships; the second correlation relationships are used to identify relationships between the sample clauses; obtaining last round sample dialogue state information, and inputting the last round sample dialogue state information, the second feature information, and the second correlation relationships into the sample self-recurrent model for processing to obtain current round sample dialogue state information and third feature information of the sample texts; inputting the first correlation relationships, the second correlation relationships, and the third feature information into the sample attention mechanism model for processing to obtain fourth feature information of the sample clauses; determining the sample recognition results corresponding to the sample texts according to the third feature information and the fourth feature information. 3.The method of claim 2, wherein, The sample recognition results comprise total sentence intents and sub-sentence intents; The method comprises: processing the third feature information according to a preset total sentence task processing strategy to obtain total sentence intents of the sample texts; processing the fourth feature information according to a preset sub-sentence task processing strategy to obtain sub-sentence intents of the sample texts. 4.The method of claim 1, wherein, The method comprises: The sample text is segmented according to a preset sentence segmentation strategy to obtain the sample clauses contained in the sample text, and the sequential number of the sample clauses is obtained. Based on the previous round of sample text, the sample clauses, and the sequential number of the sample clauses, the first clause information of the sample clauses contained in the sample text is determined. 5.A multi-turn dialogue intent recognition method, characterized in that, include: Obtain the target text to be identified and its previous target text; The target text is preprocessed to obtain the second clause information of the target clause contained in the target text; the second clause information includes the previous text of the target text, the target clause, and the sequence number of the target clause; The second clause information is input into a multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text; wherein, the multi-turn dialogue intent recognition model is trained by the training method of the multi-turn dialogue intent recognition model according to any one of claims 1-4; The intent recognition results are filtered according to preset filtering conditions to obtain the prediction results corresponding to the target text. 6.The multi-turn dialogue intent recognition method of claim 5, wherein, The multi-turn dialogue intent recognition model includes a target pre-trained language model, a sample clause inter-language model, a target autoregressive model, and a target attention mechanism model; The step of inputting the second clause information into a multi-turn dialogue intent recognition model for recognition processing to obtain the intent recognition result corresponding to the target text includes: The second clause information is input into the target pre-trained language model for processing to obtain the fifth feature information and the third correlation relationship of the target clause; the third correlation relationship represents the relationship between the target clause and the previous round of target text. The fifth feature information and the third correlation of the target clause are input into the target clause language model for processing to obtain the sixth feature information and the fourth correlation of the target text; the fourth correlation is used to identify the relationship between the target clauses. Obtain the target dialogue state information from the previous round, and input the target dialogue state information from the previous round, the sixth feature information, and the fourth correlation into the target autoregressive model for processing to obtain the target dialogue state information from the current round and the seventh feature information of the target text; The third correlation, the fourth correlation, and the seventh feature information are input into the target attention mechanism model for processing to obtain the eighth feature information of the target clause; The intent recognition result corresponding to the target text is determined based on the seventh feature information and the eighth feature information.

7. A training device of a multi-turn dialogue intent recognition model, comprising: A processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the method as claimed in any one of claims 1 to 4.

8. A multi-turn dialogue intent recognition device, comprising: A processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the method as described in any one of claims 5 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 5 to 6.

Images