Model training, intent recognition method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a model training method, an intent recognition method, a device, an electronic device and a storage medium. The model training method comprises: obtaining a historical multi-turn conversation text set, respectively performing splicing processing on each historical multi-turn conversation text to obtain a historical sample set based on relative position coding; creating a to-be-trained multi-intent understanding model comprising at least one classifier, obtaining a historical sample sub-set corresponding to each classifier from the historical sample set, and the classifier being a multi-label classifier; for each classifier, training the classifier by using the historical sample sub-set corresponding to the classifier, and obtaining the multi-intent understanding model after training of all classifiers is completed. In the embodiments of the present application, the multi-intent understanding model can learn the correlation between the semantic information of the conversation text, the position information before and after the conversation text and the conversation intent category, and at least one classifier is set for analysis, so that the recognition result of the multi-intent understanding model is more accurate and comprehensive.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a model training, intent recognition method, device, electronic device and storage medium. Background Art

[0002] With the increasing maturity of natural language processing technologies, multi-turn conversation systems have been widely used in many scenarios, such as customer service, business consulting, and online shopping. In multi-turn conversation systems based on customer service scenarios, text data contains relevant service and product requirements expressed by users. Therefore, by labeling and classifying these texts, we can understand the user's intentions, help customer service personnel explore the user's potential needs, and promptly optimize products and improve services, thereby more effectively handling business and improving service quality. However, in multi-turn conversation systems in customer service scenarios, user intentions usually involve multiple intentions. Therefore, this task belongs to the multi-label text classification problem. Multi-label text classification is different from multi-category text classification tasks in which each text has only one category label. Multi-label text classification provides more detailed text information and is more meaningful and valuable.

[0003] Currently, multi-label text classification methods are primarily categorized into two approaches: problem transformation and algorithm adaptation. Problem transformation transforms the multi-label classification problem into multiple simple single-label classification problems. However, this approach performs single-label processing and ignores the correlation between labels, resulting in poor accuracy. Algorithm adaptation adapts multi-class classification methods to accommodate multi-label classification. However, this approach typically only considers the correlation between multiple labels within a single text, ignoring the relationships between texts across multiple rounds of conversations. Consequently, its accuracy is poor. Summary of the Invention

[0004] In view of the above problems, the embodiments of the present application propose a model training, intent recognition method, device, electronic device and storage medium to improve the accuracy of intent recognition.

[0005] According to one aspect of an embodiment of the present application, a model training method is provided, the method comprising:

[0006] Obtain a collection of historical multi-round conversation texts, and concatenate each of the historical multi-round conversation texts to obtain a collection of historical samples based on relative position coding;

[0007] Creating a multi-intent understanding model to be trained comprising at least one classifier, and obtaining a subset of historical samples corresponding to each classifier from the historical sample set; the classifier is a multi-label classifier;

[0008] For each classifier, the classifier is trained using a subset of historical samples corresponding to the classifier. After all classifiers are trained, a multi-intent understanding model is obtained.

[0009] Optionally, for any historical multi-round conversation text, performing splicing processing on the historical multi-round conversation text includes:

[0010] Step 1: Initialize i=j=1 and initialize the jth historical concatenation sequence d corresponding to the historical multi-round conversation text. j is empty, initializing the historical samples corresponding to the historical multi-round conversation texts to be empty;

[0011] Step 2: Determine the i-th historical conversation text s in the historical multi-round conversation text i With the d j Is the sum of the lengths of less than or equal to a preset threshold? If so, proceed to step 3; if not, proceed to step 4;

[0012] Step 3, in the d j The tail of the splicing s i , and in the s i Then add the position identifier, set i = i + 1, and execute step 5;

[0013] Step 4, in the d j Add a semantic identifier to the head, add a text delimiter to the tail, and add the d to the tail of the historical sample. j , let j = j + 1, initialize d j If it is empty, go to step 5;

[0014] Step 5: Determine whether i and j are both less than or equal to the total number of historical conversation texts in the historical multi-round conversation texts; if so, execute step 2; if not, obtain the historical samples corresponding to the historical multi-round conversation texts.

[0015] Optionally, obtaining a historical sample subset corresponding to each classifier from the historical sample set includes: dividing the historical sample set into the total number of historical sample auxiliary sets according to the total number of classifiers; for each classifier, selecting one historical sample auxiliary set as the verification sample set corresponding to the classifier, and using the remaining historical sample auxiliary sets as the training sample sets corresponding to the classifier; and using the training sample set and the verification sample set as the historical sample subsets corresponding to the classifier; wherein, the verification sample sets corresponding to different classifiers are different.

[0016] Optionally, the classifier is trained using a subset of historical samples corresponding to the classifier, including: inputting historical samples in the subset of historical samples corresponding to the classifier into the classifier, identifying the historical samples in the classifier, and obtaining the predicted intention category of the historical samples identified by the classifier; calculating the model loss value based on the predicted intention category of the historical samples and the preset actual intention category of the historical samples; and determining that the classifier training is completed when the model loss value meets preset conditions.

[0017] Optionally, the historical sample includes at least one historical splicing sequence, which includes at least one historical conversation text, and a position identifier is added after each historical conversation text; the historical sample is identified in the classifier to obtain the predicted intention category of the historical sample identified by the classifier, including: in the classifier, for each historical splicing sequence in the historical sample in turn, based on the embedding vector corresponding to the position identifier in the historical splicing sequence, obtaining the predicted candidate intention category of the historical splicing sequence; based on the predicted candidate intention category of each historical splicing sequence, determining the predicted intention category of the historical sample.

[0018] Optionally, the training of the classifier using the historical sample subset corresponding to the classifier includes: training the classifier using an exponential moving average operation and an adversarial training operation based on the historical sample subset corresponding to the classifier.

[0019] According to another aspect of an embodiment of the present application, a method for identifying intent is provided, the method comprising:

[0020] Acquire multiple rounds of conversation texts to be analyzed, and perform splicing processing on the multiple rounds of conversation texts to be analyzed to obtain samples to be analyzed based on relative position coding;

[0021] Obtain a pre-trained multi-intent understanding model, wherein the multi-intent understanding model includes at least one classifier, and the classifier is a multi-label classifier; the multi-intent understanding model is trained by the method described in any of the above items;

[0022] The classifiers in the multi-intention understanding model are used to identify the samples to be analyzed respectively, and the intent categories of the multi-round conversation texts to be analyzed are determined based on the identification results of the classifiers.

[0023] Optionally, the sample to be analyzed includes at least one spliced ​​sequence to be analyzed, and the spliced ​​sequence to be analyzed includes at least one conversation text to be analyzed, and a position identifier is added after each conversation text to be analyzed; the sample to be analyzed is identified respectively by each classifier in the multi-intention understanding model, and the intent category of the multi-round conversation text to be analyzed is determined based on the identification result of each classifier, including: for each classifier, the sample to be analyzed is input into the classifier, and in the classifier, for each spliced ​​sequence to be analyzed in the sample to be analyzed, the candidate intent category of the spliced ​​sequence to be analyzed is obtained based on the embedding vector corresponding to the position identifier in the spliced ​​sequence to be analyzed, a voting operation is performed based on the candidate intent categories of the spliced ​​sequence to be analyzed obtained by each classifier to determine the target candidate intent category of the spliced ​​sequence to be analyzed; and the intent category of the multi-round conversation text to be analyzed is determined based on the target candidate intent category of each spliced ​​sequence to be analyzed.

[0024] According to another aspect of an embodiment of the present application, a model training device is provided, comprising:

[0025] The first splicing module is used to obtain a set of historical multi-round conversation texts, and splice each historical multi-round conversation text to obtain a historical sample set based on relative position coding;

[0026] A first acquisition module is configured to create a multi-intent understanding model to be trained comprising at least one classifier, and to obtain a subset of historical samples corresponding to each classifier from the historical sample set; the classifier is a multi-label classifier;

[0027] The training module is used to train each classifier using a subset of historical samples corresponding to the classifier, and obtain a multi-intent understanding model after all classifiers are trained.

[0028] Optionally, the first splicing module includes:

[0029] Initialization unit, used to initialize i=j=1, initialize the jth historical splicing sequence d corresponding to the historical multi-round conversation text j is empty, initializing the historical samples corresponding to the historical multi-round conversation texts to be empty;

[0030] The first judgment unit is used to judge the i-th historical conversation text s in the historical multi-round conversation text i With the d j is less than or equal to a preset threshold; if so, the first splicing unit is called; if not, the second splicing unit is called;

[0031] The first splicing unit is used to jThe tail of the splicing s i , and in the s i Then add the position identifier, set i=i+1, and call the second judgment unit;

[0032] The second splicing unit is used to j Add a semantic identifier to the head, add a text delimiter to the tail, and add the d to the tail of the historical sample. j , let j = j + 1, initialize d j is empty, and the second judgment unit is called;

[0033] The second judgment unit is used to judge whether i and j are both less than or equal to the total number of historical conversation texts in the historical multi-round conversation texts; if so, call the first splicing unit; if not, obtain the historical samples corresponding to the historical multi-round conversation texts.

[0034] Optionally, the first acquisition module includes: a division unit, used to divide the historical sample set into the total number of historical sample auxiliary sets according to the total number of classifiers; a selection unit, used to select one historical sample auxiliary set as the verification sample set corresponding to the classifier for each classifier, and the remaining historical sample auxiliary sets as the training sample set corresponding to the classifier, and the training sample set and the verification sample set as the historical sample subsets corresponding to the classifier; wherein, the verification sample sets corresponding to different classifiers are different.

[0035] Optionally, the training module includes: an identification unit, used to input historical samples in the historical sample subset corresponding to the classifier into the classifier, identify the historical samples in the classifier, and obtain the predicted intention category of the historical samples identified by the classifier; a calculation unit, used to calculate the model loss value based on the predicted intention category of the historical sample and the preset actual intention category of the historical sample; and a determination unit, used to determine that the classifier training is completed when the model loss value meets a preset condition.

[0036] Optionally, the historical sample includes at least one historical splicing sequence, which includes at least one historical conversation text, and a position identifier is added after each historical conversation text; the recognition unit is specifically used to obtain the predicted candidate intention category of the historical splicing sequence based on the embedding vector corresponding to the position identifier in the historical splicing sequence for each historical splicing sequence in the historical sample in the classifier in turn; and determine the predicted intention category of the historical sample based on the predicted candidate intention category of each historical splicing sequence.

[0037] Optionally, the training module is specifically configured to train the classifier based on a subset of historical samples corresponding to the classifier using an exponential moving average operation and an adversarial training operation.

[0038] According to another aspect of an embodiment of the present application, there is provided an apparatus for identifying intention, the apparatus comprising:

[0039] A second splicing module is used to obtain multiple rounds of conversation texts to be analyzed, and splice the multiple rounds of conversation texts to be analyzed to obtain samples to be analyzed based on relative position coding;

[0040] A second acquisition module is configured to acquire a pre-trained multi-intent understanding model, wherein the multi-intent understanding model includes at least one classifier, which is a multi-label classifier; the multi-intent understanding model is trained by the method described in any one of the above methods;

[0041] The recognition module is used to use each classifier in the multi-intention understanding model to respectively identify the samples to be analyzed, and determine the intent category of the multi-round conversation text to be analyzed based on the recognition results of each classifier.

[0042] Optionally, the sample to be analyzed includes at least one spliced ​​sequence to be analyzed, and the spliced ​​sequence to be analyzed includes at least one conversation text to be analyzed, and a position identifier is added after each conversation text to be analyzed; the recognition module is specifically used to input the sample to be analyzed into the classifier for each classifier, and in the classifier, for each spliced ​​sequence to be analyzed in the sample, obtain the candidate intent category of the spliced ​​sequence to be analyzed based on the embedding vector corresponding to the position identifier in the spliced ​​sequence to be analyzed, perform voting based on the candidate intent categories of the spliced ​​sequence to be analyzed obtained by each classifier, and determine the target candidate intent category of the spliced ​​sequence to be analyzed; based on the target candidate intent category of each spliced ​​sequence to be analyzed, determine the intent category of the multiple rounds of conversation texts to be analyzed.

[0043] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; and one or more computer-readable storage media having instructions stored thereon; when the instructions are executed by the one or more processors, the processors execute the method as described in any one of the above items.

[0044] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor is caused to execute any of the methods described above.

[0045] In an embodiment of the present application, on the one hand, by splicing historical multi-round conversation texts to obtain historical samples based on relative position encoding, the position information between the previous and next conversation texts in the multi-round conversation texts can be added to the input sequence of the multi-intention understanding model, so that the multi-intention understanding model can learn the semantic information of the conversation text and the correlation between the previous and next position information of the conversation text and the conversation intent category label, thereby making the analysis of the multi-intention understanding model more accurate; on the other hand, by setting at least one multi-label classifier for the multi-intention understanding model and using the samples corresponding to each classifier to train each classifier, the training process is simpler, and the prediction results of at least one classifier can be used to comprehensively analyze the final intent category subsequently, thereby making the recognition results of the multi-intention understanding model more comprehensive and explainable. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some drawings of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 It is a schematic diagram of an overall processing process of an embodiment of the present application.

[0048] Figure 2 This is a flowchart of the steps of a model training method in an embodiment of the present application.

[0049] Figure 3 This is a flowchart of a splicing processing process in an embodiment of the present application.

[0050] Figure 4 It is a structural diagram of a classifier in an embodiment of the present application.

[0051] Figure 5 This is a flowchart of an adversarial training process according to an embodiment of the present application.

[0052] Figure 6 This is a flowchart of a model training process in an embodiment of the present application.

[0053] Figure 7 This is a flowchart of the steps of an intent recognition method in an embodiment of the present application.

[0054] Figure 8 This is a flowchart of an intent recognition process in an embodiment of the present application.

[0055] Figure 9 This is a schematic diagram of a multi-intent understanding model processing process in an embodiment of the present application.

[0056] Figure 10 This is a structural block diagram of a model training device according to an embodiment of the present application.

[0057] Figure 11 This is a structural block diagram of an intention recognition device according to an embodiment of the present application.

[0058] Figure 12 It is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, rather than all the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0061] Currently, online customer service systems generate massive amounts of text data daily from multiple rounds of conversations between users and customer service personnel. Companies need to leverage this data to understand user needs for related services and products, and to conduct business statistics, product upgrades, and service optimization. Therefore, multi-label text classification is crucial. However, most current methods focus solely on addressing the relevance of multiple labels within a single text, ignoring the relevance of semantic information within multiple rounds of conversations and the contextual location of each text. Furthermore, they cannot effectively address the imbalance of datasets across different categories.

[0062] In response to the above problems, the embodiments of this application propose to train a multi-intent understanding model that includes multiple classifiers, and use text splicing processing to add the position correlation between each conversation text, use model training optimization method to train the model, and use the trained multi-intent understanding model to perform intent recognition, so as to obtain more accurate intent recognition results.

[0063] Reference Figure 1 , which shows a schematic diagram of an overall processing process of an embodiment of the present application.

[0064] like Figure 1 As shown, the overall processing process may include:

[0065] S1: Obtain a historical multi-round conversation text set and a multi-round conversation text to be analyzed, and perform splicing processing on them respectively according to the text sequence splicing strategy (Concat_Strategy) to obtain a historical sample set and a sample to be analyzed.

[0066] The splicing process can solve the correlation problem between text semantic information, text position information and intent category labels.

[0067] S2, based on the historical sample set, uses the model training optimization strategy (Optimization_Strategy) to train a multi-round conversation text multi-intention understanding model based on relative position encoding.

[0068] Model training optimization strategies can address the problem of class-imbalanced training datasets.

[0069] S3, based on the sample to be analyzed, adopts a multi-intent integration strategy (Bagging_Strategy) and uses the trained multi-intent understanding model to predict the intent category of the multi-round conversation text to be analyzed.

[0070] The multi-intent integration strategy can comprehensively analyze the prediction results of multiple classifiers in the multi-intent understanding model, thereby improving the accuracy of the overall prediction results.

[0071] The embodiments of this application are applicable to fields such as business analysis, user demand mining, and user portrait construction. They help enterprises obtain user demand information on various dimensions of product and customer service from multi-round conversation texts, and conduct business volume statistics, product upgrades, and service optimization.

[0072] Reference Figure 2 , shows a step flow chart of a model training method in an embodiment of the present application.

[0073] like Figure 2 As shown, the model training method may include the following steps 201 to 203:

[0074] Step 201: Obtain a set of historical multi-round conversation texts, and perform splicing processing on each historical multi-round conversation text to obtain a set of historical samples based on relative position coding.

[0075] A historical multi-turn conversation text collection contains multiple historical multi-turn conversation texts, each of which contains multiple historical conversation texts. For example, for a historical conversation between a user and customer service representative, multiple user texts from the conversation can be extracted as historical multi-turn conversation texts, with each user text as a historical conversation text.

[0076] In an embodiment of the present application, for each historical multi-round conversation text in the historical multi-round conversation text set, the historical multi-round conversation text is spliced ​​to obtain a historical sample based on relative position coding corresponding to the historical multi-round conversation text, and then the historical samples based on relative position coding corresponding to each historical multi-round conversation text constitute a historical sample set based on relative position coding.

[0077] For any historical multi-round conversation text, the multiple historical conversation texts contained in the historical multi-round conversation text can be sequentially spliced ​​into at least one historical splicing sequence according to the forward greedy algorithm, under the restriction that the length of the historical splicing sequence is less than or equal to a preset threshold.

[0078] In an optional implementation, for any historical multi-round conversation text, the process of splicing the historical multi-round conversation text may include the following steps 1 to 5:

[0079] Step 1: Initialize i=j=1 and initialize the jth historical concatenation sequence d corresponding to the historical multi-round conversation text. j is empty, and the historical samples corresponding to the historical multi-round conversation texts are initialized to be empty.

[0080] Step 2: Determine the i-th historical conversation text s in the historical multi-round conversation text i With the d j Is the sum of the lengths of less than or equal to a preset threshold? If so, proceed to step 3; if not, proceed to step 4.

[0081] The preset threshold is the difference between the preset maximum sequence length and the preset symbol length.

[0082] Step 3, in the d j The tail of the splicing s i , and in the s i Then add the location identifier, set i=i+1, and execute step 5.

[0083] Step 4, in the d j Add a semantic identifier to the head, add a text delimiter to the tail, and add the d to the tail of the historical sample. j , let j = j + 1, initialize d j Leave it blank and proceed to step 5.

[0084] Step 5: Determine whether i and j are both less than or equal to the total number of historical conversation texts in the historical multi-round conversation texts. If so, proceed to step 2; if not, obtain the historical samples corresponding to the historical multi-round conversation texts.

[0085] For example, to obtain a historical multi-round conversation text set: HistConve = {Conv1, Conv2, ..., Conv histID}, HistConvs contains multiple historical multi-round conversation texts, and each historical multi-round conversation text is spliced ​​according to the text sequence splicing strategy (Concat_Strategy), and then a historical sample set based on relative position encoding corresponding to the historical multi-round conversation text set is generated: HistAdjData = {Data1, Data2, ..., Data histID}.

[0086] Reference Figure 3 , which shows a flowchart of a splicing processing process in an embodiment of the present application.

[0087] like Figure 3 As shown, the splicing process may include:

[0088] S1.1, Input: Historical multi-round conversation text Conv ID =[s1, s2, ..., s n ].

[0089] Among them, Conv ID It can refer to the historical multi-round conversation text in the above historical multi-round conversation text set HistConvs, that is, ID = 1, 2, ..., histID. ID Contains n historical conversation texts s i , i=1,2,…,n。

[0090] S1.2, Initialization: Parameters i = j = 1, historical multi-round conversation text Conv ID The corresponding j-th historical splicing sequence d j Empty, historical multi-round conversation text Conv ID Corresponding historical sample Data ID Empty, preset maximum sequence length max_length = 1.

[0091] S1.3, determine whether length(d j +s i )≤(l-3); if so, execute S1.4; otherwise, execute S1.5.

[0092] Among them, length(d j +s i ) means d j Conv with historical multi-round conversation text ID The i-th historical conversation text s in i, 3 represents the preset symbol length, the symbol includes a position identifier, a semantic identifier and a text separator, and the length of each symbol is 1. It can be understood that the symbol and the symbol length can also be set to other forms, and this embodiment does not limit this.

[0093] S1.4, let d j =d j +s i +[MASK], i=i+1, and execute S1.6.

[0094] That is, in d j The tail splicing s i , then in s i Then add a position identifier [MASK] and set i=i+1.

[0095] S1.5, let d j =[CLS]+d j +[SEP], in Data ID Add d at the end j , j=j+1, initialize d j Leave it blank and proceed to S1.6.

[0096] That is, in d j Add a semantic identifier [CLS] to the header of d j Add a text separator [SEP] to the end of j Append to Data ID At the end of , and let j = j + 1, initialize d j Empty.

[0097] S1.6, determine whether i≤n and j≤n are satisfied; if so, execute S1.3; otherwise, execute S1.7.

[0098] Among them, n represents Conv ID The length of the list length(Conv ID )=n, that is, Conv ID The total number of historical conversation texts in .

[0099] S1.7, output Conv ID Corresponding historical sample Data ID =[d1, d2, ..., d q ].

[0100] Data ID Contains q historical splicing sequences d j , 1≤q≤n. d jIt is a historical splicing sequence in historical samples based on relative position coding constructed according to the idea of ​​Prompt method. Its construction template is: j =[CLS]s i [MASK]s i+1 [MASK]…s i+x [MASK][SEP], where [CLS] is a semantic characterizer, [SEP] is a text delimiter, and [MASK] is s i The concatenation strategy Concat_Strategy is based on the forward greedy algorithm (FGA), in d j Under the condition that the length of Conv does not exceed the preset threshold, ID s in i Prompt construction is carried out in sequence to obtain the corresponding Data ID .

[0101] Step 202: Create a multi-intent understanding model to be trained that includes at least one classifier, and obtain a subset of historical samples corresponding to each classifier from the historical sample set.

[0102] In an embodiment of the present application, the multi-intent understanding model to be trained may include at least one classifier. The multi-intent understanding model may use at least one classifier to perform intent recognition separately, and then perform a comprehensive analysis of the intent recognition results of multiple classifiers. During the model training process, in order to simplify the training process, each classifier may be trained separately. The classifier is a multi-label classifier.

[0103] In an optional embodiment, the process of obtaining a historical sample subset corresponding to each classifier from the historical sample set may include: dividing the historical sample set into the total number of historical sample auxiliary sets according to the total number of classifiers; for each classifier, selecting one historical sample auxiliary set as the verification sample set corresponding to the classifier, and using the remaining historical sample auxiliary sets as the training sample sets corresponding to the classifier; and using the training sample set and the verification sample set as the historical sample subsets corresponding to the classifier; wherein, the verification sample sets corresponding to different classifiers are different.

[0104] The above-mentioned K-fold cross-validation method is used for model training to solve the problem of imbalanced training data sets and implement a model training optimization strategy for imbalanced data sets.

[0105] In step 203 , for each classifier, the historical sample subset corresponding to the classifier is used to train the classifier. After all classifiers are trained, a multi-intent understanding model is obtained.

[0106] In an optional implementation, the step of training the classifier using the historical sample subset corresponding to the classifier may include the following steps A1 to A3:

[0107] Step A1: inputting historical samples in the historical sample subset corresponding to the classifier into the classifier, identifying the historical samples in the classifier, and obtaining the predicted intent category of the historical samples identified by the classifier.

[0108] Exemplarily, the process of identifying the historical samples in the classifier and obtaining the predicted intention category of the historical samples identified by the classifier may include: in the classifier, for each historical splicing sequence in the historical sample in turn, based on the embedding vector corresponding to the position identifier in the historical splicing sequence, obtaining the predicted candidate intention category of the historical splicing sequence; and determining the predicted intention category of the historical sample based on the predicted candidate intention category of each historical splicing sequence.

[0109] Exemplarily, after obtaining the predicted candidate intent categories of each historical splicing sequence in the historical sample, the predicted candidate intent categories of all historical splicing sequences in the historical sample can be used as the predicted intent category of the historical sample.

[0110] Reference Figure 4 , shows a structural diagram of a classifier in an embodiment of the present application.

[0111] according to Figure 4 The classifier structure shown above takes a historical concatenation sequence [CLS]s1[MASK]s2[MASK]…s in the historical sample. x [MASK][SEP], input the enhanced Transformer layer with RoPE (Rotary Position Embedding) in the classifier, that is, the RoFormer (Transformer with Rotary Position Encoding) layer; perform word segmentation (Tokenization) on the RoFormer layer to obtain the Token Embedding vector (embedding vector) of each unit position in the historical concatenation sequence; then use only each historical conversation text s i The Token Embedding vector corresponding to the position identifier [MASK] unit position, that is, T [MASK], the subsequent classification calculation is performed through the Linear layer; then the probability of each intent category is calculated through the Softmax layer; finally, the intent category with the highest probability is selected as the predicted intent category through the Argmax layer. It should be noted that for the Softmax layer and the Argmax layer Figure 4 The specific structure can be referred to as follows Figure 9 shown.

[0112] Step A2: Calculate the model loss value based on the predicted intent category of the historical sample and the preset actual intent category of the historical sample.

[0113] Exemplarily, the model loss value may include but is not limited to: a cross entropy loss function, an absolute value loss function, a square loss function, an exponential loss function, and the like.

[0114] Step A3: When the model loss value meets the preset conditions, it is determined that the classifier training is completed.

[0115] For example, if the model loss value is less than a preset loss threshold, the classifier training is determined to be complete. The specific value of the loss threshold can be set based on actual experience and is not limited in this embodiment.

[0116] For example, in the process of training the classifier, in order to improve the recognition ability of the intent understanding model, the classifier can also be trained based on the historical sample subset corresponding to the classifier using the exponential moving average (EMA) operation and adversarial training (AT) operation to optimize the model.

[0117] For example, an EMA strategy can be used to smooth the model weights according to Formula 1 to provide the model with better generalization capabilities. The specific process of the EMA operation can be processed based on actual experience and will not be discussed in detail in this embodiment.

[0118] v t =β·v t-1 +(1-β)·θ t Formula 1

[0119] In formula 1, θ t represents the weight of the model at time t, v t represents the shadow weight of the model at time t, v t-1represents the shadow weight of the model at time t-1, and β represents the preset weighted weight value (for example, β is set to 0.999, of course, other values ​​can also be set).

[0120] For example, when performing an AT operation, any AT operation mode may be selected, and this embodiment does not limit this. AT operations may include, but are not limited to, FGM (Fast Gradient Method), FGSM (Fast Gradient Sign Method), PGD (Projected Gradient Descent), and the like.

[0121] Taking FGM as an example, adversarial learning can be achieved by adding perturbations in the Embedding layer according to Formula 2.

[0122]

[0123] in, represents the distribution of input samples, Represents input, represents the label, θ is the model parameter, is the loss of a single sample, is the perturbation, and S is the perturbation space.

[0124] Reference Figure 5 , shows a schematic diagram of a process of adversarial training in an embodiment of the present application. Figure 5 As shown, the adversarial training process can include: forward (forward) calculation of normal loss (loss), backward propagation (backward) calculation of normal parameter gradient (grad); according to the adversarial training formula and gradient, calculation of the perturbation (that is, the above ), the perturbation calculated is accumulated (attack) on the weights of the embedding layer; according to the accumulated model weights, the new loss (adv_loss) and new gradient (adv_grad) are calculated; if it is not the last step (if PGD), the perturbation is calculated according to the adversarial training formula and the new gradient, the perturbation is accumulated (attack) again, and the next step (K-step) is performed; until the last step, the initial weight of the embedding layer is restored, the original gradient and the new gradient (adv_grad) are accumulated, and the model weights are updated.

[0125] For example, based on the historical multi-round conversation text set HistConvs = {Conv1, Conv2, ..., Conv histID}The corresponding historical sample set based on relative position coding HistAdjData={Data1,Data2,…,Data histID}, through the model training optimization strategy (Optimization_Strategy), the multi-intention understanding model of multi-round conversation text based on relative position encoding is trained: MultiIntRecognizer = {fold1,fold2,…,fold K}, where fold i represents the i-th classifier, K∈N.

[0126] Reference Figure 6 , which shows a flowchart of a model training process in an embodiment of the present application.

[0127] like Figure 6 As shown, the model training process may include:

[0128] S2.1, input: historical sample set HistAdjData = {Data1, Data2, ..., Data histID}.

[0129] S2.2, randomly divide HistAdjDate into K historical sample auxiliary sets: Subsets = {Set1, Set2, ..., Set K}.

[0130] in, X = m / K, where m represents the total number of historical samples in the historical sample set, and K represents the total number of classifiers.

[0131] S2.3, according to the KFoldCV (K-fold cross validation) data set partitioning method, obtain the historical sample subsets used to train and verify K classifiers: KFold-Datasets = {Dataset1, Dataset2, ..., Dataset K}.

[0132] Among them, Dataset k ={Train k ,Valid k}, Train k Represents the training sample set of the kth classifier, Train k =(Subsets-Set k ), Valid k Represents the validation sample set of the kth classifier, Valid k =Set k .

[0133] S2.4, initialization: parameter k = 1, multi-intent understanding model MultiIntRecognizer = {}.

[0134] S2.5, based on Dataset k , through the principle of RoFormer algorithm, using EMA and AT strategies, training to get fold k , and add it to the MultiIntRecognizer, k=k+1.

[0135] S2.6, determine whether k≤K is satisfied. If so, execute S2.5; otherwise, execute S2.7.

[0136] S2.7, output: Multi-intent understanding model MultiIntRecognizer = {fold1,fold2,…,fold K}.

[0137] In the embodiment of the present application, a text sequence splicing strategy and a multi-intent understanding model structure are proposed for the correlation between the position information of the text of multiple rounds of conversations and the intent category labels. The position information between the previous and next texts in multiple rounds of conversations is added to the input sequence of the multi-intent understanding model, and the vector of the position unit in the sequence is used for subsequent classification calculations, so that the model can learn the semantic information of the conversation text and the correlation between the position information of the previous and next texts of the conversation text and the conversation intent category labels. A model training optimization strategy for category-imbalanced data sets is also proposed. The multi-intent understanding model uses the K-fold cross-validation method for model training. It learns as many text features of each category in the training samples as possible through multiple classifiers, enhances the model's expressiveness through adversarial training, and increases the model's robustness and generalization ability through exponential moving average to further improve the effect of multi-intent understanding.

[0138] Reference Figure 7 , shows a step flow chart of an intent recognition method according to an embodiment of the present application.

[0139] like Figure 7 As shown, the intention recognition method may include the following steps:

[0140] Step 701: Acquire multiple rounds of conversation texts to be analyzed, and perform splicing processing on the multiple rounds of conversation texts to be analyzed to obtain samples to be analyzed based on relative position coding.

[0141] The multi-round conversation text to be analyzed includes multiple conversation texts to be analyzed. For example, for a conversation to be analyzed between a user and customer service representative, multiple user texts in the conversation to be analyzed can be extracted as the multi-round conversation text to be analyzed, with each user text as a conversation text to be analyzed.

[0142] For the multiple rounds of conversation texts to be analyzed, a forward greedy algorithm can be used to sequentially splice the multiple conversation texts to be analyzed contained in the multiple rounds of conversation texts to be analyzed into at least one splicing sequence to be analyzed, under the restriction that the length of the splicing sequence to be analyzed is less than or equal to a preset threshold.

[0143] In an optional embodiment, the process of splicing the texts of multiple rounds of conversations to be analyzed to obtain samples to be analyzed based on relative position coding may include the following steps (1) to (5):

[0144] Step (1), initialize i=j=1, initialize the jth splicing sequence d to be analyzed corresponding to the multi-round conversation text to be analyzed j If it is empty, the samples to be analyzed corresponding to the multiple rounds of conversation texts to be analyzed are initialized to be empty.

[0145] Step (2), determine the i-th conversation text to be analyzed s in the multiple rounds of conversation texts to be analyzed i With the d j Is the sum of the lengths of less than or equal to a preset threshold? If so, execute step (3); if not, execute step (4). The preset threshold is the difference between the preset maximum sequence length and the preset symbol length.

[0146] Step (3), in the d j The tail of the splicing s i , and in the s i Then add the position identifier, set i=i+1, and execute step (5).

[0147] Step (4), in d j Add a semantic identifier to the head, add a text delimiter to the tail, and add the d to the tail of the historical sample. j , let j = j + 1, initialize d j is empty and execute step (5).

[0148] Step (5) determines whether i and j are both less than or equal to the total number of conversation texts to be analyzed in the multi-round conversation texts to be analyzed. If so, step (2) is executed; if not, the sample to be analyzed corresponding to the multi-round conversation texts to be analyzed is obtained.

[0149] Step 702: Obtain a pre-trained multi-intent understanding model, where the multi-intent understanding model includes at least one classifier.

[0150] The multi-intent understanding model is trained by the model training method described in any of the above embodiments.

[0151] Step 703: Utilize each classifier in the multi-intent understanding model to respectively identify the sample to be analyzed, and determine the intent category of the multi-round conversation text to be analyzed based on the identification results of each classifier.

[0152] In an optional embodiment, the process of using each classifier in the multi-intent understanding model to respectively identify the sample to be analyzed, and determining the intent category of the multi-round conversation text to be analyzed based on the recognition results of each classifier may include the following steps B1 to B2:

[0153] Step B1: For each classifier, the sample to be analyzed is input into the classifier, and in the classifier, for each spliced ​​sequence to be analyzed in the sample to be analyzed, based on the embedding vector corresponding to the position identifier in the spliced ​​sequence to be analyzed, the candidate intent category of the spliced ​​sequence to be analyzed is obtained, and a voting operation is performed based on the candidate intent categories of the spliced ​​sequence to be analyzed obtained by each classifier to determine the target candidate intent category of the spliced ​​sequence to be analyzed.

[0154] In the embodiment of the present application, any applicable voting method can be used for voting, and this embodiment does not impose any restrictions on this.

[0155] Taking the Plurality Voting (PV) method as an example, the K classifiers can be folded according to the following formula 3: k For s i The multi-intention prediction results of the corresponding [MASK] unit position are fused and analyzed.

[0156]

[0157] Formula 1 indicates that the most common intent categories among the candidate intent categories of the spliced ​​sequence to be analyzed obtained by each classifier are selected. Where t represents the number of intent categories, represents the kth classifier pair s i The prediction result of the corresponding [MASK] unit position on the t-th intent category.

[0158] Step B2: determining the intent category of the multi-round conversation texts to be analyzed based on the target candidate intent category of each of the spliced ​​sequences to be analyzed.

[0159] Illustratively, after obtaining the target candidate intent category of each to-be-analyzed concatenation sequence in the to-be-analyzed sample, the target candidate intent categories of all to-be-analyzed concatenation sequences in the to-be-analyzed sample may be used as the predicted intent category of the multi-round conversation text to be analyzed.

[0160] For example, based on the multi-round conversation text Data to be analyzed newID, use the multi-intent understanding model MultiIntRecognizer for prediction, and get Conv through the multi-intent integration strategy (Bagging_Strategy) newID ConvIntentions = {int1, int2, ..., int e}, e∈N.

[0161] Reference Figure 8 , which shows a flowchart of an intention recognition process in an embodiment of the present application.

[0162] like Figure 8 As shown, the intent recognition process may include:

[0163] S3.1, Input: Multi-round conversation text Data to be analyzed newID =[d1,d2,…,d q ].

[0164] S3.2, initialization: parameter j = 1, the set of intent categories of the multi-round conversation text to be analyzed ConvIntentions = {} (empty).

[0165] S3.3, use the multi-intent understanding model MultiIntRecognizer to analyze the spliced ​​sequence d j Perform intent recognition and obtain each classifier fold k For d j The multi-intent label prediction result set Results j ={fold1(d j ),fold2(d j ),…,fold K (d j )}.

[0166] in,

[0167] S3.4, according to Results j All folds in k The prediction results are given by using the PV method for each l in ClassLabels t Voting, get d j Each s i The conversation intent category label set

[0168] in,

[0169] S3.5, according to Labels jAdd non-repeated l to ConvIntentions t , j=j+1.

[0170] S3.6, determine whether j≤q is satisfied. If so, execute S3.3; otherwise, execute S3.7.

[0171] Among them, q represents the total number of conversation texts to be analyzed in multiple rounds of conversation texts, that is, length (Data newID )=q.

[0172] S3.7, output: intent category set ConvIntentions = {int1:l x ,int2:l y ,…,int e :l z}.

[0173] in, It is fold k For s i The candidate intent category predicted by the corresponding [MASK] unit position, ClassLabels={l1,l2,…,l T} is a set of predefined intent categories, T∈N; It is fold k For s i The corresponding [MASK] unit position is in the conversation intent category l t The prediction result on the prediction output l t Otherwise it is recorded as 0.

[0174] In the embodiment of the present application, intent recognition is performed on the multi-round conversation text to be analyzed based on a pre-trained multi-intent recognition model. The multi-intent integration strategy based on multiple classifiers can solve the problem of comprehensive analysis of the prediction results of multiple classifiers in the multi-intent understanding model, and the output multi-intent understanding results are more comprehensive and explainable.

[0175] Below, based on a specific example, the overall processing process in the embodiment of the present application is explained.

[0176] For example, a multi-round conversation text of an operator's online customer service system is shown in Table 1:

[0177] Table 1

[0178]

[0179]

[0180] In the embodiment of the present application, the overall processing process is as follows:

[0181] S11, obtain the historical multi-round conversation text set HistConvs = {Conv1, Conv2, ..., Conv 9999} and the multi-round conversation text to be analyzed Conv 10000 , and perform splicing according to the text sequence splicing strategy (Concat_Strategy), respectively generating a historical sample set HistAdjData={Data1, Data2, ..., Data 9999}, and the sample data to be analyzed based on relative position encoding corresponding to the multiple rounds of conversation text to be analyzed 10000 .

[0182] To analyze multiple rounds of conversation text Conv 1o000 Taking data as an example, the specific steps of S11 are as follows:

[0183] S11.1, Input: Multi-round conversation text to be analyzed Conv 10000 =[s1, s2, s3, s4, s5, s6, s7].

[0184] S11.2, initialization: parameter i=j=1, the splicing sequence to be analyzed d1 is empty, the multi-round conversation text Conv to be analyzed 10000 Corresponding sample Data to be analyzed 10000 =[], in this embodiment, the maximum sequence length max_length is set to 40.

[0185] S11.3, determine that length(d1+s1)=15 is less than (max_length-3)=37, so execute S11.4.

[0186] S11.4, concatenate s1 and [MASK] at the end of d1 to construct a Prompt, obtaining d1 = "I have received the traffic, but why is there no additional traffic? [MASK]", i = i + 1 = 2, and execute S11.6.

[0187] S11.6, determine if i=2 and j=1 are both less than length(Conv 10000 )=7, so execute S11.3.

[0188] Repeat S11.3, S11.4, and S11.6 until length (d1 + s4) = 47:

[0189] S11.3, determine that length(d1+s4)=47 is greater than 37, so execute S11.5.

[0190] S11.5, add [CLS] and [SEP] at the beginning and end of d1 respectively, and get d1 = "[CLS] I have received the traffic, but why is there no traffic added? [MASK] is the gift you gave me. [MASK] Every and append it to Data 10000 At the end of 10000 ={d1}, j=j+1=2, initialize d2 to empty.

[0191] Repeat steps S11.3 to S11.6 until i=8:

[0192] S11.6, determine that i=8 is greater than 7 and j=3 is less than 7, so execute S11.7.

[0193] S11.7, output Conv 10000 Corresponding samples to be analyzed:

[0194]

[0195] In S12, based on HistAdjData, a multi-round conversation multi-intent understanding model based on relative position encoding is trained through the model training optimization strategy (Optimization_Strategy): MultiIntRecognizer. The specific steps are as follows:

[0196] S12.1, input historical sample set HistAdjData = {Data1, Data2, ..., Data 9999}.

[0197] S12.2. In this embodiment, the number of classifiers K is set to 3, and HistAdjData is randomly divided into three auxiliary sets of historical samples according to the session ID:

[0198]

[0199] S12.3, according to the KFoldCV data set partitioning method, obtain the folds used for training and verifying the three classifiers k A subset of historical samples:

[0200]

[0201] Among them, Train represents the training sample set, and Valid represents the validation sample set.

[0202] S12.4, initialization: parameter k = 1, multi-intent understanding model MultiIntRecognizer = {}.

[0203] S12.5, based on Dataset1 data, through the principle of RoFormer algorithm, using EMA and AT strategies, train fold1 and add it to MultiIntRecognizer = {fold1}, k = k + 1 = 2.

[0204] S12.6, determine that k=2 is less than K=3, so execute S12.5.

[0205] Repeat steps S12.5 to S12.6 until k=4:

[0206] S12.6, determine that k=4 is greater than 3, so execute S12.7.

[0207] S12.7, output: MultiIntRecognizer = {fold1, fold2, fold3}.

[0208] S13, based on Data 10000 Data, use MultiIntRecognize to make predictions, and get Conv through the multi-intent integration strategy (Bagging_Strategy) 10000 The final intent category set ConvIntentions. The specific steps are as follows:

[0209] S13.1, Input: Sample Data to be analyzed 10000 =[d1, d2, d3].

[0210] S13.2, Initialization: Parameter j = 1, Conv of multiple rounds of conversation text to be analyzed 10000 The corresponding intent category set ConvIntentions = {}.

[0211] S13.3, use MultiIntRecognizer to perform multi-intent recognition on d1, according to the predefined intent category set ClassLabels = {l1 = query, l2 = consultation, l3 = handling, l4 = fault, l5 = harassment, l6 = other} in this embodiment, get each fold k The prediction result set for d1.

[0212] Reference Figure 9 , which shows a schematic diagram of a multi-intent understanding model processing process in an embodiment of the present application. Figure 9 As shown, after processing each fold k The prediction result set Results1 for d1 is as follows:

[0213]

[0214] S13.4, according to all folds in Results1 k The prediction results are given by using the PV method for each conversation intent category label l in ClassLabels. t Voting to get each s in d1 i The conversation intent category label set:

[0215] S13.5, add non-repeated conversation intent category labels l to ConvIntentions according to Labels1 t , we get ConvIntentions = {query, consultation}, j = j + 1 = 2.

[0216] S13.6, determine if j=2 is less than length(Data 10000 )=3, so execute S13.3.

[0217] Repeat the above steps S13.3 to S13.6 until j=4:

[0218] S13.6, determine that j=4 is greater than 3, so execute S13.7.

[0219] S13.7, output Conv 10000 The corresponding intent category set ConvIntentions = {int1: query, int2: consultation, int3: fault, int4: other}.

[0220] In the embodiment of the present application, when dealing with the problem of multi-intent understanding of multi-round conversation texts, the results output by the multi-label text classification method are more comprehensive and more valuable; based on the correlation between the semantic information of the conversation text and the contextual position information of the conversation text and the conversation intent category label, a text sequence splicing strategy and a multi-intent understanding model structure are proposed, and a model training optimization strategy is proposed to train a multi-intent understanding model on a category-imbalanced data set; finally, based on the prediction results of all classifiers of the model, a multi-intent integration strategy is adopted to comprehensively obtain more interpretable multi-intent understanding results of multi-round conversations.

[0221] Reference Figure 10 , shows a structural block diagram of a model training device in an embodiment of the present application.

[0222] like Figure 10 As shown, the model training device may include the following modules:

[0223] The first splicing module 1001 is used to obtain a set of historical multi-round conversation texts, and splice each of the historical multi-round conversation texts to obtain a set of historical samples based on relative position coding;

[0224] A first acquisition module 1002 is configured to create a multi-intent understanding model to be trained comprising at least one classifier, and to obtain a subset of historical samples corresponding to each classifier from the historical sample set; the classifier is a multi-label classifier;

[0225] The training module 1003 is used to train each classifier using a subset of historical samples corresponding to the classifier, and obtain a multi-intent understanding model after all classifiers are trained.

[0226] Optionally, the first splicing module 1001 includes:

[0227] Initialization unit, used to initialize i=j=1, initialize the jth historical splicing sequence d corresponding to the historical multi-round conversation text j is empty, initializing the historical samples corresponding to the historical multi-round conversation texts to be empty;

[0228] The first judgment unit is used to judge the i-th historical conversation text s in the historical multi-round conversation text i With the d j is less than or equal to a preset threshold; if so, the first splicing unit is called; if not, the second splicing unit is called;

[0229] The first splicing unit is used to j The tail of the splicing s i , and in the s i Then add the position identifier, set i=i+1, and call the second judgment unit;

[0230] The second splicing unit is used to j Add a semantic identifier to the head, add a text delimiter to the tail, and add the d to the tail of the historical sample. j , let j = j + 1, initialize d j is empty, and the second judgment unit is called;

[0231] The second judgment unit is used to judge whether i and j are both less than or equal to the total number of historical conversation texts in the historical multi-round conversation texts; if so, call the first splicing unit; if not, obtain the historical samples corresponding to the historical multi-round conversation texts.

[0232] Optionally, the first acquisition module 1002 includes: a division unit, used to divide the historical sample set into the total number of historical sample auxiliary sets according to the total number of classifiers; a selection unit, used to select one historical sample auxiliary set as the verification sample set corresponding to the classifier for each classifier, and the remaining historical sample auxiliary sets as the training sample set corresponding to the classifier, and the training sample set and the verification sample set as the historical sample subsets corresponding to the classifier; wherein, the verification sample sets corresponding to different classifiers are different.

[0233] Optionally, the training module 1003 includes: an identification unit, used to input historical samples in the historical sample subset corresponding to the classifier into the classifier, identify the historical samples in the classifier, and obtain the predicted intention category of the historical samples identified by the classifier; a calculation unit, used to calculate the model loss value based on the predicted intention category of the historical samples and the preset actual intention category of the historical samples; and a determination unit, used to determine that the classifier training is completed when the model loss value meets a preset condition.

[0234] Optionally, the historical sample includes at least one historical splicing sequence, which includes at least one historical conversation text, and a position identifier is added after each historical conversation text; the recognition unit is specifically used to obtain the predicted candidate intention category of the historical splicing sequence based on the embedding vector corresponding to the position identifier in the historical splicing sequence for each historical splicing sequence in the historical sample in the classifier in turn; and determine the predicted intention category of the historical sample based on the predicted candidate intention category of each historical splicing sequence.

[0235] Optionally, the training module 1003 is specifically configured to train the classifier based on a subset of historical samples corresponding to the classifier using an exponential moving average operation and an adversarial training operation.

[0236] Reference Figure 11 , shows a structural block diagram of an intention recognition device in an embodiment of the present application.

[0237] like Figure 11 As shown, the intention recognition device may include the following modules:

[0238] The second splicing module 1101 is used to obtain multiple rounds of conversation texts to be analyzed, and splice the multiple rounds of conversation texts to be analyzed to obtain samples to be analyzed based on relative position coding;

[0239] A second acquisition module 1102 is configured to acquire a pre-trained multi-intent understanding model, wherein the multi-intent understanding model includes at least one classifier, which is a multi-label classifier; the multi-intent understanding model is trained by the method described in any one of the above methods;

[0240] The identification module 1103 is used to use each classifier in the multi-intention understanding model to identify the sample to be analyzed respectively, and determine the intent category of the multi-round conversation text to be analyzed based on the identification results of each classifier.

[0241] Optionally, the sample to be analyzed includes at least one spliced ​​sequence to be analyzed, and the spliced ​​sequence to be analyzed includes at least one conversation text to be analyzed, and a position identifier is added after each conversation text to be analyzed; the identification module 1103 is specifically used to input the sample to be analyzed into the classifier for each classifier, and in the classifier, for each spliced ​​sequence to be analyzed in the sample, obtain the candidate intent category of the spliced ​​sequence to be analyzed based on the embedding vector corresponding to the position identifier in the spliced ​​sequence to be analyzed, perform a voting operation based on the candidate intent categories of the spliced ​​sequence to be analyzed obtained by each classifier, and determine the target candidate intent category of the spliced ​​sequence to be analyzed; based on the target candidate intent category of each spliced ​​sequence to be analyzed, determine the intent category of the multiple rounds of conversation texts to be analyzed.

[0242] In an embodiment of the present application, on the one hand, by splicing historical multi-round conversation texts to obtain historical samples based on relative position encoding, the position information between the previous and next conversation texts in the multi-round conversation texts can be added to the input sequence of the multi-intention understanding model, so that the multi-intention understanding model can learn the semantic information of the conversation text and the correlation between the previous and next position information of the conversation text and the conversation intent category label, thereby making the analysis of the multi-intention understanding model more accurate; on the other hand, by setting at least one multi-label classifier for the multi-intention understanding model and using the samples corresponding to each classifier to train each classifier, the training process is simpler, and the prediction results of at least one classifier can be used to comprehensively analyze the final intent category subsequently, thereby making the recognition results of the multi-intention understanding model more comprehensive and explainable.

[0243] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0244] In an embodiment of the present application, an electronic device is further provided. The electronic device may include one or more processors and one or more computer-readable storage media storing instructions, such as application programs. When the instructions are executed by the one or more processors, the processors perform the method of any of the above embodiments.

[0245] Reference Figure 7 , shows a schematic diagram of an electronic device structure according to an embodiment of the present application. Figure 7 As shown, the electronic device includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704. The processor 701, the communication interface 702 and the memory 703 communicate with each other via the communication bus 704.

[0246] The memory 703 is used to store computer programs.

[0247] The processor 701 is configured to implement the method of any of the above embodiments when executing the program stored in the memory 703 .

[0248] The communication interface 702 is used for communication between the electronic device and other devices.

[0249] The communication bus 704 mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0250] The processor 701 mentioned above may include but is not limited to: a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0251] The memory 703 mentioned above may include but is not limited to: read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), electronic erasable programmable read-only memory (EEPROM), hard disk, floppy disk, flash memory, etc.

[0252] In an embodiment of the present application, a computer-readable storage medium is further provided, on which a computer program is stored. The program can be executed by a processor of an electronic device. When the computer program is executed by the processor, the processor executes the method described in any of the above embodiments.

[0253] The various embodiments in this specification are interrelated and are described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referenced to each other.

[0254] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or terminal device that includes the element.

[0255] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0256] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0257] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0258] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0259] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0260] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0261] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0262] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0263] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. In summary, the contents of this specification should not be construed as limiting the present application.

Claims

1. A model training method, characterized in that: The method comprises: Obtain a collection of historical multi-round conversation texts, and concatenate each of the historical multi-round conversation texts to obtain a collection of historical samples based on relative position coding; Creating a multi-intent understanding model to be trained comprising at least two classifiers, and obtaining a subset of historical samples corresponding to each classifier from the historical sample set; the classifier is a multi-label classifier; For each classifier, the classifier is trained using the historical sample subset corresponding to the classifier. After all classifiers are trained, a multi-intent understanding model is obtained. For any historical multi-round conversation text, the historical multi-round conversation text is spliced, including: Step 1: Initialize i=j=1 and initialize the jth historical concatenation sequence d corresponding to the historical multi-round conversation text. j is empty, initializing the historical samples corresponding to the historical multi-round conversation texts to be empty; Step 2: Determine the i-th historical conversation text s in the historical multi-round conversation text i With the d j Is the sum of the lengths of less than or equal to a preset threshold? If so, proceed to step 3; if not, proceed to step 4; Step 3, in the d j The tail of the splicing s i , and in the s i Then add the position identifier, set i = i + 1, and execute step 5; Step 4, in the d j Add a semantic identifier to the head, add a text delimiter to the tail, and add the d to the tail of the historical sample. j , let j = j + 1, initialize d j If it is empty, go to step 5; Step 5: Determine whether i and j are both less than or equal to the total number of historical conversation texts in the historical multi-round conversation texts; if so, execute step 2; if not, obtain the historical samples corresponding to the historical multi-round conversation texts.

2. The method according to claim 1, characterized in that The obtaining of a subset of historical samples corresponding to each classifier from the historical sample set includes: Dividing the historical sample set into auxiliary sets of historical samples of the total number according to the total number of classifiers; For each classifier, an auxiliary set of historical samples is selected as the verification sample set corresponding to the classifier, the remaining auxiliary sets of historical samples are used as the training sample set corresponding to the classifier, and the training sample set and the verification sample set are used as the historical sample subsets corresponding to the classifier; wherein, different classifiers correspond to different verification sample sets.

3. The method according to claim 1, characterized in that The training of the classifier using the historical sample subset corresponding to the classifier includes: Inputting historical samples from the historical sample subset corresponding to the classifier into the classifier, identifying the historical samples in the classifier, and obtaining a predicted intent category of the historical samples identified by the classifier; Calculating a model loss value according to the predicted intent category of the historical sample and the preset actual intent category of the historical sample; When the model loss value meets a preset condition, it is determined that the classifier training is completed.

4. The method according to claim 3, characterized in that The historical sample includes at least one historical splicing sequence, the historical splicing sequence includes at least one historical conversation text, and a position identifier is added after each historical conversation text; the historical sample is identified in the classifier to obtain the predicted intent category of the historical sample identified by the classifier, including: In the classifier, for each historical splicing sequence in the historical sample, based on the embedding vector corresponding to the position identifier in the historical splicing sequence, a predicted candidate intent category of the historical splicing sequence is obtained; Based on the predicted candidate intent categories of each of the historical splicing sequences, the predicted intent category of the historical sample is determined.

5. The method according to claim 1, wherein The training of the classifier using the historical sample subset corresponding to the classifier includes: Based on a subset of historical samples corresponding to the classifier, the classifier is trained using an exponential moving average operation and an adversarial training operation.

6. A method for identifying intention, characterized in that: The method comprises: Acquire multiple rounds of conversation texts to be analyzed, and perform splicing processing on the multiple rounds of conversation texts to be analyzed to obtain samples to be analyzed based on relative position coding; Obtain a pre-trained multi-intent understanding model, wherein the multi-intent understanding model includes at least two classifiers, and the classifiers are multi-label classifiers; the multi-intent understanding model is trained by the method according to any one of claims 1 to 5; The classifiers in the multi-intention understanding model are used to identify the samples to be analyzed respectively, and the intent categories of the multi-round conversation texts to be analyzed are determined based on the identification results of the classifiers.

7. The method according to claim 6, characterized in that The sample to be analyzed includes at least one spliced ​​sequence to be analyzed, and the spliced ​​sequence to be analyzed includes at least one conversation text to be analyzed, and a position identifier is added after each conversation text to be analyzed; using each classifier in the multi-intention understanding model to respectively identify the sample to be analyzed, and determining the intent category of the multi-round conversation text to be analyzed based on the recognition results of each classifier, including: For each classifier, the sample to be analyzed is input into the classifier, and the classifier sequentially obtains the candidate intent category of each spliced ​​sequence to be analyzed in the sample based on the embedding vector corresponding to the position identifier in the spliced ​​sequence to be analyzed. A voting operation is performed based on the candidate intent categories of the spliced ​​sequence to be analyzed obtained by each classifier to determine the target candidate intent category of the spliced ​​sequence to be analyzed; Based on the target candidate intent category of each of the splicing sequences to be analyzed, the intent category of the multi-round conversation texts to be analyzed is determined.

8. A model training device, characterized in that: The device comprises: The first splicing module is used to obtain a set of historical multi-round conversation texts, and splice each historical multi-round conversation text to obtain a historical sample set based on relative position coding; A first acquisition module is used to create a multi-intent understanding model to be trained comprising at least two classifiers, and obtain a subset of historical samples corresponding to each classifier from the historical sample set; the classifier is a multi-label classifier; The first splicing module includes: Initialization unit, used to initialize i=j=1, initialize the jth historical splicing sequence d corresponding to the historical multi-round conversation text j is empty, initializing the historical samples corresponding to the historical multi-round conversation texts to be empty; The first judgment unit is used to judge the i-th historical conversation text s in the historical multi-round conversation text i With the d j is less than or equal to a preset threshold; if so, the first splicing unit is called; if not, the second splicing unit is called; The first splicing unit is used to j The tail of the splicing s i , and in the s i Then add the position identifier, set i=i+1, and call the second judgment unit; The second splicing unit is used to j Add a semantic identifier to the head, add a text delimiter to the tail, and add the d to the tail of the historical sample. j , let j = j + 1, initialize d j is empty, and the second judgment unit is called; The second judgment unit is used to judge whether i and j are both less than or equal to the total number of historical conversation texts in the historical multi-round conversation texts; if so, calling the first splicing unit; if not, obtaining the historical sample corresponding to the historical multi-round conversation texts; The training module is used to train each classifier using a subset of historical samples corresponding to the classifier, and obtain a multi-intent understanding model after all classifiers are trained.

9. An intention recognition device, characterized in that: The device comprises: A second splicing module is used to obtain multiple rounds of conversation texts to be analyzed, and splice the multiple rounds of conversation texts to be analyzed to obtain samples to be analyzed based on relative position coding; A second acquisition module is configured to acquire a pre-trained multi-intent understanding model, wherein the multi-intent understanding model includes at least two classifiers, and the classifiers are multi-label classifiers; the multi-intent understanding model is trained by the method according to any one of claims 1 to 5; The recognition module is used to use each classifier in the multi-intention understanding model to respectively identify the samples to be analyzed, and determine the intent category of the multi-round conversation text to be analyzed based on the recognition results of each classifier.

10. An electronic device, characterized in that: include: one or more processors; and one or more computer-readable storage media having instructions stored thereon; When the instruction is executed by the one or more processors, the processor executes the model training method as described in any one of claims 1 to 5, or executes the intention recognition method as described in any one of claims 6 to 7.

11. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the processor performs the model training method as described in any one of claims 1 to 5, or executes the intention recognition method as described in any one of claims 6 to 7.

Images