Intention recognition model training method, intention recognition method, device and apparatus

Abstract

The present disclosure provides a training method of an intention recognition model, an intention recognition method, an apparatus and a device, relates to the technical field of artificial intelligence, in particular to the technical field of natural language processing and deep learning, and can be applied to a voice recognition scene. The specific implementation scheme is as follows: sample text data corresponding to at least two intention prediction networks is respectively acquired; the sample text data corresponding to each intention prediction network is different; the sample text data is input into a general feature extraction network to obtain sample text features, and the sample text features are input into a corresponding intention prediction network to obtain sample intentions predicted by the intention prediction network; the intention prediction network is trained according to the sample intentions predicted by the intention prediction network and intention supervision data of the sample text data; and the general feature extraction network is trained according to the sample intentions predicted by the at least two intention prediction networks and the intention supervision data of the sample text data.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the field of natural language processing and deep learning, and can be applied to scenarios such as speech recognition. BACKGROUND

[0002] With the development of artificial intelligence technology, the demand for intelligence in many application scenarios is increasing, and therefore good human-computer interaction is essential. It is particularly important to identify the true intention of the user during human-computer interaction.

[0003] The application scenarios of related intention recognition models in the prior art are too single, and can only cover intention problems in one scenario. There are some intention determination rules, and the rule-based intention recognition in different scenarios is not universal and has poor transferability. In addition, the user's historical text is not flexibly used to derive the intention of the current text during model training, and only the current text information is used. Therefore, improvement is urgently needed. SUMMARY

[0004] The present disclosure provides a training method of an intention recognition model, an intention recognition method, an apparatus and a device.

[0005] According to an aspect of the present disclosure, a training method of an intention recognition model is provided, the intention recognition model comprising a general feature extraction network and at least two intention prediction networks; the network structures of the at least two intention prediction networks are the same; the method comprises:

[0006] Respectively acquiring sample text data corresponding to the at least two intention prediction networks; the sample text data corresponding to each intention prediction network is different;

[0007] Inputting the sample text data into the general feature extraction network to obtain sample text features, and inputting the sample text features into the corresponding intention prediction network to obtain sample intentions predicted by the intention prediction network;

[0008] Training the intention prediction network according to the sample intentions predicted by the intention prediction network and intention supervision data of the sample text data;

[0009] Training the general feature extraction network according to the sample intentions predicted by the at least two intention prediction networks and the intention supervision data of the sample text data.

[0010] According to another aspect of the present disclosure, an intention recognition method is provided, the method comprising:

[0011] Inputting target text data into the general feature extraction network of the intention recognition model to obtain target text features, and inputting the target text features into the at least two intention prediction networks of the intention recognition model respectively to obtain candidate intentions predicted by the at least two intention prediction networks;

[0012] determine a target intent of the target text data from the at least two candidate intents; wherein the intent recognition model is trained based on the training method of the intent recognition model provided in any of the embodiments of the present disclosure.

[0013] According to another aspect of the present disclosure, a training device of an intent recognition model is provided, the intent recognition model comprising a general feature extraction network and at least two intent prediction networks; the network structures of the at least two intent prediction networks are the same; the device comprises:

[0014] a sample text data determination module configured to respectively acquire sample text data corresponding to the at least two intent prediction networks; the sample text data corresponding to each intent prediction network is different;

[0015] a sample intent determination module configured to input the sample text data into the general feature extraction network to obtain sample text features, and input the sample text features into the corresponding intent prediction network to obtain sample intents predicted by the intent prediction network;

[0016] an intent prediction network training module configured to train the intent prediction network according to the sample intents predicted by the intent prediction network and intent supervision data of the sample text data;

[0017] a feature extraction network training module configured to train the general feature extraction network according to the sample intents predicted by the at least two intent prediction networks and the intent supervision data of the sample text data.

[0018] According to another aspect of the present disclosure, an intent recognition device is provided, the device comprising:

[0019] a candidate intent determination module configured to input the target text data into the general feature extraction network of the intent recognition model to obtain target text features, and input the target text features into the at least two intent prediction networks of the intent recognition model respectively to obtain candidate intents predicted by the at least two intent prediction networks;

[0020] a target intent determination module configured to determine a target intent of the target text data from the at least two candidate intents; wherein the intent recognition model is trained based on the training device of the intent recognition model provided in any of the embodiments of the present disclosure.

[0021] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0022] at least one processor; and

[0023] a memory in communication with the at least one processor; wherein,

[0024] The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform the training method or intent recognition method of the intent recognition model provided in any embodiment of this disclosure.

[0025] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute a training method for an intent recognition model or an intent recognition method provided in any embodiment of this disclosure.

[0026] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a training method for an intent recognition model or an intent recognition method provided according to any embodiment of this disclosure.

[0027] The technology disclosed herein can improve the accuracy of intent recognition.

[0028] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0029] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0030] Figure 1A This is a flowchart of a training method for an intent recognition model provided according to an embodiment of the present disclosure;

[0031] Figure 1B This is a schematic diagram illustrating the training process of an intent recognition model provided according to an embodiment of this disclosure;

[0032] Figure 2A This is a flowchart of another training method for an intent recognition model provided according to an embodiment of the present disclosure;

[0033] Figure 2B This is a schematic diagram of the prediction process of an intent prediction network according to an embodiment of the present disclosure;

[0034] Figure 3A This is a flowchart of another training method for an intent recognition model provided according to an embodiment of the present disclosure;

[0035] Figure 3B This is a schematic diagram illustrating the training and prediction process of an intent recognition model according to an embodiment of this disclosure;

[0036] Figure 4 This is a flowchart of an intent recognition method provided according to an embodiment of the present disclosure;

[0037] Figure 5 This is a flowchart of yet another intent recognition method provided according to embodiments of the present disclosure;

[0038] Figure 6 This is a schematic diagram of the structure of a training device for an intent recognition model provided according to an embodiment of the present disclosure;

[0039] Figure 7 This is a schematic diagram of the structure of an intent recognition device provided according to an embodiment of the present disclosure;

[0040] Figure 8 This is a block diagram of an electronic device used to implement the intent recognition model of the embodiments of this disclosure, or the intent recognition method. Detailed Implementation

[0041] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0042] It should be noted that the terms "first," "second," "third," "sample," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0043] Furthermore, it should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of data to be processed in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0044] Figure 1A This is a flowchart of a training method for an intent recognition model provided according to an embodiment of the present disclosure. Figure 1BThis is a schematic diagram illustrating the training process of an intent recognition model according to an embodiment of this disclosure. This embodiment applies to situations involving the training of an intent recognition model. The method can be executed by a training device for the intent recognition model, which can be implemented in software and / or hardware and integrated into an electronic device that carries the training function of the intent recognition model, such as a server.

[0045] Optionally, the intent recognition model includes a general feature extraction network and at least two intent prediction networks; the at least two intent prediction networks have the same network structure. The general feature extraction network is used to extract features from the sample text data; the intent prediction networks are used to predict intent based on the extracted features from the sample text data.

[0046] like Figure 1A and Figure 1B As shown, the training method for the intent recognition model in this embodiment may include:

[0047] S101, acquire sample text data corresponding to at least two intent prediction networks respectively; the sample text data corresponding to each intent prediction network is different.

[0048] In this embodiment, the sample text data can be text data obtained after converting speech data, and may include multiple pieces of text data with a temporal relationship, such as current sample data and historical sample data. The current sample data refers to the latest text data, while the historical sample data refers to text data from moments before the current sample data. For example, in a multi-turn conversation scenario for speech recognition, the current sample data refers to the text data obtained from converting the user's speech data acquired in the current turn; the historical sample data refers to the text data obtained from converting the user's speech data in a predetermined number of turns prior to the current turn.

[0049] It should be noted that the sample text data can be obtained from different scenarios, such as devices with screens and devices without screens. In other words, the sample text data comes from different intent systems.

[0050] An intent prediction network refers to a neural network used for intent prediction. For example, it can be a fully connected network, specifically including at least one fully connected layer. For instance, an intent prediction network uses the sigmoid function to predict intent, outputting a multi-dimensional prediction vector. Each dimension of the multi-dimensional prediction vector represents a label prediction result, and the value of each dimension is expressed in probability form, indicating the probability of predicting the label. The prediction probability of a label belongs to the range (0, 1). For example, an output of a 3-dimensional prediction vector [0.6, 0.8, 0.9] indicates that the prediction probability for label A is 0.6, the prediction probability for label B is 0.8, and the prediction probability for label C is 0.9.

[0051] Specifically, sample text data corresponding to at least two intent prediction networks can be obtained from different scenarios. The sample text data corresponding to each intent prediction network is different.

[0052] S102, input the sample text data into the general feature extraction network to obtain sample text features, and input the sample text features into the corresponding intent prediction network to obtain the sample intent predicted by the intent prediction network.

[0053] In this embodiment, the general feature extraction network is used to extract features from the sample text data. It can be an encoding network, such as a pre-trained Transformer model.

[0054] The so-called sample text features refer to the text features extracted by a general feature extraction network, which can be represented in matrix or vector form.

[0055] The so-called sample intent refers to the intent predicted by the intent prediction network, which can include at least one intent, that is, a label whose probability prediction is greater than a preset threshold in the multi-dimensional prediction vector output by the intent prediction network.

[0056] Specifically, the sample text data is input into a general feature extraction network. After the network learns, the sample text features are obtained. Then, the sample text features are input into the corresponding intent prediction network. After the network learns and predicts, the sample intent predicted by the intent prediction network is obtained.

[0057] For example, an intent recognition model includes N intent prediction networks, denoted as Intent Prediction Network 1, Intent Prediction Network 2, ..., Intent Prediction Network N; the sample text data corresponding to each intent prediction network are sample text data 1, sample text data 2, ..., sample text data N, respectively; the sample text features corresponding to each intent prediction network are sample text feature 1, sample text feature 2, ..., sample text feature N, respectively; where N is a natural number greater than 1. Sample text data 1 can be input into a general feature extraction network to obtain sample text feature 1, and then input into intent prediction network 1 to obtain the sample intent 1 predicted by intent prediction network 1. Similarly, sample text data 2 can be input into a general feature extraction network to obtain sample text feature 2, and then input into intent prediction network 2 to obtain the sample intent 2 predicted by intent prediction network 2. And so on, sample text data N can be input into a general feature extraction network to obtain sample text feature N, and then input into intent prediction network N to obtain the sample intent N predicted by intent prediction network N.

[0058] S103, Train the Intent Prediction Network based on the sample intent predicted by the Intent Prediction Network and the intent supervision data of the sample text data.

[0059] In this embodiment, the intent supervision data of the sample text data refers to the label data of the sample text data.

[0060] Specifically, based on a preset loss function, the task training loss can be determined according to the sample intent predicted by the intent prediction network and the intent supervision data of the sample text data. This task training loss is then used to train the intent prediction network. Continuing the example above, task training loss 1 can be determined based on the intent supervision data of sample intent 1 and sample text data 1, and task training loss 1 can be used to train intent prediction network 1, i.e., updating the parameters in intent prediction network 1 using task training loss 1. Similarly, task training loss 2 can be determined based on the intent supervision data of sample intent 2 and sample text data 2, and task training loss 2 can be used to train intent prediction network 2, i.e., updating the parameters in intent prediction network 2 using task training loss 2. Likewise, task training loss N can be determined based on the intent supervision data of sample intent N and sample text data N, and task training loss N can be used to train intent prediction network N, i.e., updating the parameters in intent prediction network N using task training loss N.

[0061] S104, Train a general feature extraction network based on sample intents predicted by at least two intent prediction networks and intent supervision data of sample text data.

[0062] Specifically, S103 can obtain the sample intent and sample text data predicted by each intent prediction network, obtain the task training loss of each intent prediction network, and then determine the general training loss based on the task training loss, and train the general feature extraction network.

[0063] In other words, when training the general feature extraction network, all sample text data are considered; when training each intent prediction network, only the sample text data corresponding to that intent prediction network is considered. This phased updating of the intent recognition model's network parameters ensures that the shared representation layer (i.e., the general feature extraction network) can better learn text features without fitting to a specific intent task in a particular scenario when training the general feature extraction network; each intent prediction network is only affected by its corresponding sample text data (i.e., the specific scenario task), thus making the intent recognition model's learning process more robust.

[0064] Compared to the conventional approach of labeling different intent systems in a single sample text data for multi-task joint training, this disclosure supports simultaneous training on sample text data with different intent systems. Furthermore, when updating parameters in the intent recognition model, the parameters of the general feature extraction network and at least two intent prediction networks are updated separately, which can indirectly control the robustness of the model's performance and avoid negative impacts.

[0065] The technical solution provided in this disclosure acquires sample text data corresponding to at least two intent prediction networks, then inputs the sample text data into a general feature extraction network to obtain sample text features, and inputs the sample text features into the corresponding intent prediction network to obtain the sample intent predicted by the intent prediction network. The intent prediction network is then trained based on the sample intent predicted by the intent prediction network and the intent supervision data of the sample text data. Similarly, the general feature extraction network is trained based on the sample intent predicted by at least two intent prediction networks and the intent supervision data of the sample text data. Compared to existing technologies that use a single total loss to train the model, which is prone to overfitting, this disclosure updates the parameters of the general feature extraction network and each intent prediction network in stages. When training the general feature extraction network, all sample text data are used to determine the loss, allowing for better learning of text features and avoiding overfitting. When training each intent prediction network, its corresponding sample text data is used to determine the loss, avoiding the influence of intent tasks in different scenarios and improving the training efficiency of the model.

[0066] It should be noted that in this disclosure, the sample text data corresponding to each intent prediction network is used for model training separately. That is to say, this disclosure adopts a multi-intent task joint approach, using the sample text data corresponding to each intent task to train simultaneously in a single model. The dataset is not shared during the training of the intent prediction network, which provides great flexibility for the future expansion of the intent recognition model. Developers can easily add a new intent task without having to consider the issue of integrating the text data corresponding to the newly added intent task into the existing intent recognition model.

[0067] Furthermore, the cost of training, maintaining, and deploying intent recognition models can be greatly reduced by combining multiple intent tasks.

[0068] Figure 2A This is a flowchart of another training method for an intent recognition model provided according to an embodiment of the present disclosure. Figure 2B This is a schematic diagram illustrating the prediction process of an intent prediction network according to an embodiment of this disclosure. Based on the above embodiments, this embodiment further optimizes the process of "inputting sample text data into a general feature extraction network to obtain sample text features, and then inputting the sample text features into the corresponding intent prediction network to obtain the sample intent predicted by the intent prediction network," providing an optional real-time solution. For example... Figure 2A and Figure 2B As shown, the training method for the intent recognition model in this embodiment may include:

[0069] S201, Obtain sample text data corresponding to at least two intent prediction networks respectively; the sample text data corresponding to each intent prediction network is different.

[0070] S202, input the current sample data in the sample text data into the general feature extraction network to obtain the current sample features, and input the current sample features into the corresponding intent prediction network to obtain the current sample intent predicted by the intent prediction network.

[0071] In this embodiment, the current sample feature refers to the text features obtained by learning the current sample data through a general feature extraction network, which can be represented in matrix or vector form. The current sample intent refers to the intent obtained by learning and predicting the current sample features through an intent prediction network.

[0072] Specifically, the current sample data in the sample text data can be input into a general feature extraction network. After the network learns, the current sample features are obtained. Then, the current sample text features can be input into the corresponding intent prediction network. After the intent prediction network learns and predicts, the current sample intent predicted by the intent prediction network is obtained.

[0073] S203, input the historical sample data in the sample text data into the general feature extraction network to obtain the historical sample features, and input the historical sample features into the corresponding intent prediction network to obtain the historical sample intent predicted by the intent prediction network.

[0074] In this embodiment, historical sample features refer to the text features obtained by learning historical sample data through a general feature extraction network, which can be represented in matrix or vector form. Historical sample intent refers to the intent obtained by learning and predicting historical sample features through an intent prediction network.

[0075] Specifically, historical sample data from the sample text data can be input into a general feature extraction network. After the network learns, historical sample features are obtained. Then, the historical sample text features can be input into the corresponding intent prediction network. After the intent prediction network learns and predicts, the historical sample intent predicted by the intent prediction network is obtained.

[0076] S204. Through the corresponding intent prediction network, the intent of the fused sample is determined based on the current sample features, the current sample intent, the historical sample features, and the historical sample intent.

[0077] In this embodiment, the fused sample intent refers to the intent predicted by the corresponding intent prediction network by combining the current sample features and historical sample features.

[0078] An alternative approach is to use a pre-defined rule to concatenate the current sample features, current sample intent, historical sample features, and historical sample intent to obtain fused sample features. These fused sample features are then input into the corresponding intent prediction network to obtain the fused sample intent.

[0079] S205 uses the current sample intent, historical sample intent, and fused sample intent as the sample intent predicted by the intent prediction network.

[0080] Specifically, the current sample intent, historical sample intent, and fused sample intent can be used as the sample intent predicted by the intent prediction network.

[0081] S206. Train the intent prediction network based on the sample intents predicted by the intent prediction network and the intent supervision data of the sample text data.

[0082] S207, Train a general feature extraction network based on sample intents predicted by at least two intent prediction networks and intent supervision data of sample text data.

[0083] The technical solution provided in this disclosure involves acquiring sample text data corresponding to at least two intent prediction networks. The current sample data from the sample text data is then input into a general feature extraction network to obtain current sample features. These current sample features are then input into the corresponding intent prediction network to obtain the current sample intent predicted by the intent prediction network. Similarly, historical sample data from the sample text data is input into the general feature extraction network to obtain historical sample features. These historical sample features are then input into the corresponding intent prediction network to obtain the historical sample intent predicted by the intent prediction network. The corresponding intent prediction network then determines a fused sample intent based on the current sample features, current sample intent, historical sample features, and historical sample intent. This fused sample intent is used as the sample intent predicted by the intent prediction network. Finally, the intent prediction network is trained using the sample intent predicted by the intent prediction network and intent supervision data from the sample text data. The general feature extraction network is also trained using the sample intent predicted by at least two intent prediction networks and intent supervision data from the sample text data. This technical solution improves the completeness of the features by determining the current and historical sample features. Combining these features with the intent prediction results in improved accuracy.

[0084] Based on the above embodiments, as an optional approach of this disclosure, the determination of the fused sample intent through the corresponding intent prediction network based on the current sample features, current sample intent, historical sample features, and historical sample intent can be achieved by fusing the current sample intent and historical sample intent to obtain fused sample features; and inputting the fused sample features into the corresponding intent prediction network to obtain the fused sample intent predicted by the intent prediction network.

[0085] Specifically, based on preset rules, the intent of the current sample and the intent of the historical samples can be compared. Based on the comparison results, the features of the current sample and the features of the historical samples can be fused to obtain fused sample features. Then, the fused sample features are input into the corresponding intent prediction network to obtain the fused sample intent predicted by the intent prediction network.

[0086] It is understandable that by fusing the current sample intent and the historical sample intent to obtain fused sample features, and then inputting the fused sample features back into the corresponding intent prediction network to determine the fused sample intent, the prediction ability of the intent prediction network can be improved.

[0087] For example, the current sample intent and historical sample intent are used to fuse the current sample features and historical sample features to obtain fused sample features. Alternatively, the similarity between the current sample intent and historical sample intent can be determined; and the fused sample features can be determined based on the similarity, current sample features, and historical sample features.

[0088] Specifically, the similarity between the current sample intent and the historical sample intent can be calculated. For example, the similarity between the multidimensional prediction vector corresponding to the current sample intent and the multidimensional prediction vector of the historical sample intent can be calculated. Then, based on preset rules, the fused sample features are determined according to the similarity, current sample features and historical sample features. For example, the product between the similarity and the current sample features can be determined, and the product between the result of 1 minus the similarity and the historical sample features can be determined. The two products are added together, and the sum is used as the fused sample features.

[0089] Understandably, by introducing similarity, we can determine the probability of focusing on historical sample features and thus determine the fusion sample features. During model training, the similarity can be dynamically adjusted. This self-supervised approach can balance the temporal dependencies between current sample features and historical sample features, enabling more refined learning of text features and making the predicted intent more accurate. Furthermore, compared to existing technologies that require manual annotation of some data models to determine whether the current text needs to focus on the preceding text, similarity in this disclosure serves as a useful supervisory signal, avoiding manual annotation and significantly reducing the cost of manual annotation.

[0090] It should be noted that in voice scenarios, the questions expressed by users undergo a speech-to-text process. Due to the limitations of the voice module, the conversion process may suffer from quality loss, and the resulting text may not reflect the user's true needs. Furthermore, the user's speech may be truncated, resulting in the loss of key information. For example, in the case of "play a song," if the user previously requested to watch a video, then "play a song" might indicate a continued desire to watch a video. Conversely, if the user previously requested a song, then "play a song" might indicate a continued desire to listen to music. This disclosure employs a self-supervised approach to balance the relationship between historical and current conversations, thereby determining the user's true intent within a given timeframe.

[0091] Figure 3A This is a flowchart illustrating a training method for another intent recognition model provided according to an embodiment of this disclosure. Based on the above embodiments, this embodiment further optimizes the process of "training the intent prediction network based on the sample intent predicted by the intent prediction network and intent supervision data from the sample text data," providing an optional implementation scheme. For example... Figure 3AAs shown, the training method for the intent recognition model in this embodiment may include:

[0092] S301, acquire sample text data corresponding to at least two intent prediction networks respectively; the sample text data corresponding to each intent prediction network is different.

[0093] S302, input the sample text data into the general feature extraction network to obtain sample text features, and input the sample text features into the corresponding intent prediction network to obtain the sample intent predicted by the intent prediction network.

[0094] S303, determine the first training loss based on the current sample intent predicted by the intent prediction network and the intent supervision data of the sample text data.

[0095] Specifically, the first training loss can be determined based on a preset loss function, such as the cross-entropy loss function, according to the intent prediction network's prediction of the current sample intent and the intent supervision data of the sample text data.

[0096] S304. Determine the second training loss based on the historical sample intents predicted by the intent prediction network and the intent supervision data of the sample text data.

[0097] Specifically, a second training loss can be determined based on a preset loss function, such as the cross-entropy loss function, and the intent supervision data of the historical sample intents predicted by the intent prediction network and the sample text data.

[0098] S305, determine the third training loss based on the intent supervision data of the fused sample intent predicted by the intent prediction network and the sample text data.

[0099] Specifically, a third training loss can be determined based on a preset loss function, such as the cross-entropy loss function, and the intent supervision data of the fused sample intent predicted by the intent prediction network and the sample text data.

[0100] S306, Based on the first training loss, the second training loss, and the third training loss, determine the task training loss of the intention prediction network.

[0101] Alternatively, the first training loss, the second training loss, and the third training loss can be summed, and the sum can be used as the task training loss of the intention prediction network.

[0102] Another option is to average the first training loss, the second training loss, and the third training loss, and use the average result as the task training loss of the intention prediction network.

[0103] S307 uses task training loss to train the intention prediction network.

[0104] Specifically, the task training loss is used to update the parameters of the intent prediction network.

[0105] S308, Train a general feature extraction network based on sample intents predicted by at least two intent prediction networks and intent supervision data of sample text data.

[0106] The technical solution of this disclosure involves acquiring sample text data corresponding to at least two intent prediction networks, inputting the sample text data into a general feature extraction network to obtain sample text features, and then inputting the sample text features into the corresponding intent prediction network to obtain the sample intent predicted by the intent prediction network. A first training loss is then determined based on the current sample intent predicted by the intent prediction network and the intent supervision data of the sample text data. A second training loss is determined based on the historical sample intent predicted by the intent prediction network and the intent supervision data of the sample text data. A third training loss is determined based on the fused sample intent predicted by the intent prediction network and the intent supervision data of the sample text data. A task training loss for the intent prediction network is then determined based on the first, second, and third training losses. The intent prediction network is trained using the task training loss. Finally, the general feature extraction network is trained based on the sample intent predicted by at least two intent prediction networks and the intent supervision data of the sample text data. This technical solution, by determining the task training loss based on different sample intents corresponding to each intent prediction network, can improve the accuracy of intent prediction network training.

[0107] Based on the above embodiments, as an optional approach of this disclosure, a general feature extraction network is trained according to the sample intent predicted by at least two intent prediction networks and the intent supervision data of the sample text data, including: determining a general training loss based on the task training loss of at least two intent prediction networks; and training the general feature extraction network using the general training loss.

[0108] Specifically, the average of the task training losses for each intent prediction network can be calculated to obtain a general training loss. Then, the general training loss can be used to train the general feature extraction network.

[0109] Understandably, determining a general training loss based on the training loss for all tasks, and considering the overall data, can provide the learning capability of a general feature extraction network.

[0110] Based on the above embodiments, Figure 3BThis diagram illustrates the training and prediction process of an intent recognition model. Two different devices (the display device "show" and the loudspeaker "loudspeaker"), representing two different intent systems, are used as examples of learning within the same model for two different tasks. Sample text data, s1, s2, s3…, s8, and query, are obtained from both devices. Here, s1, s2, s3…, s8 represent historical sample data, and query represents the current sample data. The sample text data from both devices are then input into a general feature extraction network (Encoder, a Dumi pretrained transformer model). This yields the historical sample features (LSTM show) and current sample features (Q_E) for the sample text data of device "show," and the historical sample features (LSTM loudspeaker) and current sample features (Q_E) for the sample text data of device "loudspeaker."

[0111] Next, the historical and current sample features corresponding to the device `show` and the device `loudspeaker` are input into their respective intent prediction networks (Shared Fully Connected) to obtain the sample intent. Specifically, taking the device `show` as an example, the historical sample features (LSTM `show`) and the current sample features (Q_E) are input into the intent prediction network (Shared Fully Connected). After sigmoid prediction in the intent prediction network, the historical sample intent (Session_E) is obtained, and its corresponding multidimensional prediction vector is the Session distribution. The current sample intent (query_E) has its corresponding multidimensional prediction vector as the Query distribution. The similarity W between the multidimensional prediction vectors (Session distribution) corresponding to the historical sample intent and the multidimensional prediction vectors (query distribution) corresponding to the current sample intent is determined, i.e., the distributions similarity between the Session distribution and the Query distribution. Then, based on the similarity W, historical sample features (lstm show is also denoted as context_show) and current sample features (Q_E), the fused sample features (1-W)*context_show+W*Q_E can be determined, and the fused sample features can be input into the intent prediction network to predict the fused sample intent (supervised_E).

[0112] Similar to the loudspeaker device, historical sample features (LSTM loudspeaker) and current sample features (Q_E) are input into the intent prediction network (Shared Fully Connected). The intent prediction network uses sigmoid prediction to obtain the historical sample intent (Session_E), whose corresponding multidimensional prediction vector is the Session distribution, and the current sample intent (query_E), whose corresponding multidimensional prediction vector is the Query distribution. The similarity W between the multidimensional prediction vectors corresponding to the historical sample intent (Session distribution) and the current sample intent (query distribution) is determined; that is, the distribution similarity between the Session distribution and the Query distribution. Then, based on the similarity W, the historical sample features (LSTM loudspeaker, also denoted as context_loudspeaker), and the current sample features (Q_E), the fused sample features (1-W)*context_loudspeaker+W*Q_E are determined. These fused sample features are then input into the intent prediction network to predict the fused sample intent (supervised_E).

[0113] Furthermore, based on the sample intents predicted by the intent prediction networks corresponding to the device show and the device loudspeaker, and the intent supervision data of the sample text data, the respective intent prediction networks are trained. Taking the device show as an example, the intent supervision data of its sample text data is denoted as the Show Label interaction matrix. Based on the current sample intent (query_E) predicted by the intent prediction network and the intent supervision data of the sample text data, the first training loss (q_l) is determined. Based on the historical sample intents (Session_E) predicted by the intent prediction network and the intent supervision data of the sample text data, the second training loss (sess_l) is determined. Simultaneously, based on the fused sample intents (supervised_E) predicted by the intent prediction network and the intent supervision data of the sample text data, the third training loss (super_l) is determined. Based on the first training loss, the second training loss, and the third training loss, the task training loss (Loss(sess_l+super_l+q_l)) of the intent prediction network is determined. The task training loss is then used to train the intent prediction network, i.e., to update the parameters of the task layer.

[0114] Similarly, for the device loudspeaker, the intent supervision data of its sample text data is denoted as Loud Label interaction matrix. As shown above, the process of determining the task training loss of the device show is as follows: determine the task training loss of the intent prediction network corresponding to the device loudspeaker, and use the task training loss to train the intent prediction network, that is, update the task layer parameters.

[0115] Simultaneously, based on the task training loss of the intent prediction network corresponding to device show and the task training loss of the intent prediction network corresponding to device loudspeaker, the general feature extraction network is trained, that is, the general layer parameters are updated.

[0116] It should be noted that when the device show and device loudspeaker make predictions, they vote on the historical sample intent (session_E) and the fused sample intent (supervised_E) to obtain the final predicted intent, namely Vote(Sess, super). The specific voting method is detailed in the following example.

[0117] Figure 4 This is a flowchart illustrating an intent recognition method according to an embodiment of this disclosure. This embodiment applies to situations involving intent recognition. The method can be executed by an intent recognition device, which can be implemented in software and / or hardware and integrated into an electronic device carrying intent recognition functionality, such as a server. Figure 4 As shown, the intent recognition method in this embodiment may include:

[0118] S401, the target text data is input into the general feature extraction network of the intent recognition model to obtain the target text features, and the target text features are input into at least two intent prediction networks of the intent recognition model to obtain the candidate intents predicted by at least two intent prediction networks.

[0119] In this embodiment, target text data refers to the text data for intent prediction, which may include multiple pieces of text data. These multiple pieces of text data have a temporal relationship, such as current target data and historical target data. The current target data refers to the text data acquired at the current moment; the historical target data refers to the text data acquired within a certain period prior to the current moment. For example, in a multi-turn dialogue scenario for speech recognition, the current target data refers to the text data after text conversion of the user speech data acquired in the current turn; the historical target data refers to the text data after text conversion of the user speech data acquired in a predetermined number of turns prior to the current turn.

[0120] It should be noted that the target text data can be obtained from different scenarios, such as devices with screens or devices without approved screens.

[0121] The intent recognition model includes a general feature extraction network and at least two intent prediction networks; it can be trained based on the training method of the intent recognition model provided in any of the above embodiments.

[0122] Target text features refer to the text features learned through a general feature extraction network; they can be in matrix or vector form.

[0123] Candidate intents refer to the intents predicted by the intent prediction network; there are at least two candidate intents, and the intent prediction network outputs one candidate intent for each intent.

[0124] Specifically, the target text data can be input into the general feature extraction network of the intent recognition model. After network learning, the target text features are obtained. Then, the target text features are input into at least two intent prediction networks of the intent recognition model. After prediction by each intent prediction network, candidate intents predicted by at least two intent prediction networks are obtained. For example, the intent recognition model includes N intent prediction networks, namely intent prediction network 1, intent prediction network 2, ..., intent prediction network N; the target text features can be input into intent prediction network 1, intent prediction network 2, ..., intent prediction network N respectively to obtain candidate intent 1, candidate intent 2, ..., candidate intent N. It should be noted that a candidate intent can include multiple intent prediction results.

[0125] S402, determine the target intent of the target text data from at least two candidate intents.

[0126] In this embodiment, the target intent refers to the final prediction result of the target text data.

[0127] Alternatively, the candidate intent with the highest accuracy can be selected from at least two candidate intents as the target intent for the target text data.

[0128] Another optional approach involves determining the target intent of the target text data from at least two candidate intents, based on the method of acquiring the target text data and the method of acquiring the sample text data corresponding to each intent prediction network during training. For example, if the target text data is obtained by converting speech data acquired from a device without a screen, then the candidate intent output by the intent prediction network corresponding to the sample text data obtained by converting speech data acquired from a device without a screen is taken as the target intent of the target text data. In other words, the candidate intent predicted by the intent prediction network corresponding to sample text data belonging to the same graph system as the target text data is taken as the target intent of the target text data.

[0129] The technical solution of this disclosure involves inputting target text data into a general feature extraction network of an intent recognition model to obtain target text features, and then inputting the target text features into at least two intent prediction networks of the intent recognition model to obtain candidate intents predicted by the at least two intent prediction networks. Finally, the target intent of the target text data is determined from the at least two candidate intents. This technical solution, through the intent recognition model, can improve the accuracy of intent prediction.

[0130] Figure 5 This is a flowchart of another intent recognition method provided according to an embodiment of this disclosure. Based on the above embodiments, this embodiment further optimizes the process of "inputting target text data into a general feature extraction network of an intent recognition model to obtain target text features; inputting the target text features into at least two intent prediction networks of the intent recognition model respectively to obtain at least two candidate intents predicted by the intent prediction networks," providing an optional implementation scheme. For example... Figure 5 As shown, the intent recognition method in this embodiment may include:

[0131] S501, input the current target data in the target text data into the general feature extraction network to obtain the current target features, and input the current target features into the intent prediction network to obtain the current target intent predicted by the intent prediction network.

[0132] In this embodiment, the current target feature refers to the text features obtained by learning the current target data through a general feature extraction network, which can be represented in matrix or vector form. The current target intent refers to the intent obtained by learning and predicting the current target features through an intent prediction network.

[0133] Specifically, the current target data in the target text data can be input into a general feature extraction network. After the network learns, the current target features are obtained. Then, the current target text features can be input into the corresponding intent prediction network. After the intent prediction network learns and predicts, the current target intent predicted by the intent prediction network is obtained.

[0134] S502, input the historical target data in the target text data into the general feature extraction network to obtain historical target features, and input the historical target features into the intent prediction network to obtain the historical target intent predicted by the intent prediction network.

[0135] In this embodiment, historical target features refer to the text features obtained by learning historical target data through a general feature extraction network, which can be represented in matrix or vector form. Historical target intent refers to the intent obtained by learning and predicting historical target features through an intent prediction network.

[0136] Specifically, historical target data from the target text data can be input into a general feature extraction network. After the network learns, historical target features are obtained. Then, the historical target text features can be input into the corresponding intent prediction network. After the intent prediction network learns and predicts, the historical target intent predicted by the intent prediction network is obtained.

[0137] S503 uses an intent prediction network to determine the fused target intent based on current target features, current target intent, historical target features, and historical target intent.

[0138] In this embodiment, the fused target intent refers to the intent predicted by the corresponding intent prediction network by combining the current target features and historical target features.

[0139] An alternative approach is to use preset rules to concatenate the current target features, current target intent, historical target features, and historical target intent to obtain fused target features. These fused target features are then input into the corresponding intent prediction network to obtain fused target intent.

[0140] S504. Based on historical target intentions and fused target intentions, determine the candidate intentions predicted by the intention prediction network.

[0141] An alternative approach is to identify intents with high prediction accuracy from historical and fused target intents, and use these as candidate intents predicted by the intent prediction network.

[0142] S505, determine the target intent of the target text data from at least two candidate intents.

[0143] The technical solution provided in this disclosure involves inputting current target data from the target text data into a general feature extraction network to obtain current target features, then inputting these current target features into an intent prediction network to obtain the current target intent predicted by the intent prediction network. Similarly, historical target data from the target text data is input into the general feature extraction network to obtain historical target features, and these historical target features are input into the intent prediction network to obtain the historical target intent predicted by the intent prediction network. Then, the intent prediction network determines a fused target intent based on the current target features, current target intent, historical target features, and historical target intent. Finally, based on the historical target intent and the fused target intent, candidate intents predicted by the intent prediction network are determined, and the target intent of the target text data is determined from at least two candidate intents. This technical solution improves the completeness of features by determining current and historical target features, and then improves the accuracy of intent prediction by combining the current and historical target features.

[0144] Based on the above embodiments, as an optional approach of this disclosure, the intent prediction network determines the fused target intent according to the current target features, the current target intent, the historical target features, and the historical target intent. This includes: using the current target intent and the historical target intent to fuse the current target features and the historical target features to obtain fused target features; and inputting the fused target features into the intent prediction network to obtain the fused target intent predicted by the intent prediction network.

[0145] Specifically, based on preset rules, the current target intent and historical target intent can be compared. Based on the comparison results, the current target features and historical target features can be fused to obtain fused target features. Then, the fused target features are input into the corresponding intent prediction network to obtain the fused target intent predicted by the intent prediction network.

[0146] Among them, the current target intent and historical target intent are used to fuse the current target features and historical target features to obtain the fused target features. This can be done by determining the similarity between the current target intent and historical target intent; and determining the target fusion features based on the similarity, current target features and historical target features.

[0147] Specifically, the similarity between the current target intent and the historical target intent can be calculated. For example, the similarity between the multidimensional prediction vector corresponding to the current target intent and the multidimensional prediction vector of the historical target intent can be calculated. Then, based on preset rules, the fused target features are determined according to the similarity, the current target features and the historical target features. For example, the product between the similarity and the current target features can be determined, and the product between the result of 1 minus the similarity and the historical target features can be determined. The two products are added together, and the sum is used as the fused target features.

[0148] It is understandable that by fusing the current target intent and the historical target intent, the current target features and the historical target features are fused to obtain fused target features. Then, the fused target features are fed back into the corresponding intent prediction network to determine the fused target intent, thus laying the foundation for the determination of the target intent.

[0149] Based on the above embodiments, as an optional approach of this disclosure, the candidate intent predicted by the intent prediction network is determined according to the historical target intent and the fused target intent, including: fusing the historical target intent and the fused target intent; and determining the candidate intent predicted by the intent prediction network based on the fusion result.

[0150] For example, if the historical target intent and the fused target intent are the same, and the mean probability of the predicted probability of the historical target intent and the predicted probability of the fused target intent is greater than a probability threshold, then the historical target intent or the fused target intent is used as a candidate intent predicted by the intent prediction network. The probability threshold can be determined by those skilled in the art based on the actual situation, for example, 0.5. It should be noted that if the historical target intent and the fused target intent are the same, and the mean probability of the predicted probability of the historical target intent and the predicted probability of the fused target intent is less than the probability threshold, then the candidate intent predicted by the intent prediction network is not considered, i.e., it is not selected as a target intent.

[0151] For example, the mean vector is obtained by averaging the multidimensional prediction vectors corresponding to historical target intentions and the multidimensional prediction vectors corresponding to the fused target intentions. Labels with prediction probabilities greater than a probability threshold in the mean vector are considered candidate intentions. In a specific example, the intention prediction network outputs multidimensional prediction vectors for labels A, B, and C. Specifically, the multidimensional prediction vectors output by the intention prediction network corresponding to historical target intentions are [0.9, 0.6, 0.2], and the multidimensional prediction vectors output by the intention prediction network corresponding to the fused target intentions are [0.8, 0.9, 0.3]. The mean vector is [0.85, 0.75, 0.25]. Since 0.85 and 0.75 are both greater than the probability threshold of 0.5, labels A and B are considered candidate intentions.

[0152] Understandably, by integrating historical and convergent target intentions, the resulting target intentions become more credible.

[0153] Figure 6 This is a schematic diagram of a training device for an intent recognition model according to an embodiment of the present disclosure. This embodiment applies to situations involving the training of an intent recognition model. The device can be implemented in software and / or hardware and can be integrated into an electronic device that carries the training function of the intent recognition model, such as a server.

[0154] Optionally, the intent recognition model includes a general feature extraction network and at least two intent prediction networks; the at least two intent prediction networks have the same network structure. The general feature extraction network is used to extract features from the sample text data; the intent prediction networks are used to predict intent based on the extracted features from the sample text data.

[0155] like Figure 6 As shown, the training device 600 for the intent recognition model in this embodiment may include:

[0156] The sample text data determination module 601 is used to acquire sample text data corresponding to at least two intent prediction networks respectively; the sample text data corresponding to each intent prediction network is different.

[0157] The sample intent determination module 602 is used to input sample text data into a general feature extraction network to obtain sample text features, and input the sample text features into a corresponding intent prediction network to obtain the sample intent predicted by the intent prediction network.

[0158] The intent prediction network training module 603 is used to train the intent prediction network based on the sample intent predicted by the intent prediction network and the intent supervision data of the sample text data.

[0159] The feature extraction network training module 604 is used to train a general feature extraction network based on sample intents predicted by at least two intent prediction networks and intent supervision data of sample text data.

[0160] The technical solution provided in this disclosure acquires sample text data corresponding to at least two intent prediction networks, then inputs the sample text data into a general feature extraction network to obtain sample text features, and inputs the sample text features into the corresponding intent prediction network to obtain the sample intent predicted by the intent prediction network. The intent prediction network is then trained based on the sample intent predicted by the intent prediction network and the intent supervision data of the sample text data. Similarly, the general feature extraction network is trained based on the sample intent predicted by at least two intent prediction networks and the intent supervision data of the sample text data. Compared to existing technologies that use a single total loss to train the model, which is prone to overfitting, this disclosure updates the parameters of the general feature extraction network and each intent prediction network in stages. When training the general feature extraction network, all sample text data are used to determine the loss, allowing for better learning of text features and avoiding overfitting. When training each intent prediction network, its corresponding sample text data is used to determine the loss, avoiding the influence of intent tasks in different scenarios and improving the training efficiency of the model.

[0161] Furthermore, the sample intent determination module includes:

[0162] The current sample intent determination unit is used to input the current sample data in the sample text data into the general feature extraction network to obtain the current sample features, and input the current sample features into the corresponding intent prediction network to obtain the current sample intent predicted by the intent prediction network;

[0163] The historical sample intent determination unit is used to input historical sample data from the sample text data into a general feature extraction network to obtain historical sample features, and input the historical sample features into the corresponding intent prediction network to obtain the historical sample intent predicted by the intent prediction network.

[0164] The fusion sample intent determination unit is used to determine the fusion sample intent by using the corresponding intent prediction network based on the current sample features, the current sample intent, the historical sample features, and the historical sample intent.

[0165] The sample intent determination unit is used to take the current sample intent, historical sample intent, and fused sample intent as the sample intent predicted by the intent prediction network.

[0166] Furthermore, the fusion sample intent determination unit includes:

[0167] The fusion sample feature determination sub-unit is used to fuse the current sample features and historical sample features by adopting the current sample intent and historical sample intent to obtain fusion sample features;

[0168] The fusion sample intent determination subunit is used to input the fusion sample features into the corresponding intent prediction network to obtain the fusion sample intent predicted by the intent prediction network.

[0169] Furthermore, the fusion of sample features to determine the sub-unit is specifically used for:

[0170] Determine the similarity between the current sample intent and the historical sample intent;

[0171] Based on similarity, current sample features, and historical sample features, the sample fusion features are determined.

[0172] Furthermore, the intent prediction network training module is specifically used for:

[0173] The first training loss is determined based on the intent prediction network's prediction of the current sample intent and the intent supervision data of the sample text data.

[0174] The second training loss is determined based on the historical sample intents predicted by the intent prediction network and the intent supervision data of the sample text data.

[0175] The third training loss is determined based on the intent prediction of the fused sample intent and the intent supervision data of the sample text data.

[0176] Based on the first training loss, the second training loss, and the third training loss, determine the task training loss of the intention prediction network;

[0177] The intention prediction network is trained using task training loss.

[0178] Furthermore, the feature extraction network training module is specifically used for:

[0179] Determine the general training loss based on the task training loss of the network with at least two intentions;

[0180] A general training loss is used to train a general feature extraction network.

[0181] Figure 7 This is a schematic diagram of an intent recognition device according to an embodiment of this disclosure. This embodiment applies to situations involving intent recognition. The device can be implemented in software and / or hardware and can be integrated into an electronic device that carries intent recognition functionality, such as a server. Figure 7 As shown, the intent recognition device 700 of this embodiment may include:

[0182] The candidate intent determination module 701 is used to input the target text data into the general feature extraction network of the intent recognition model to obtain the target text features, and input the target text features into at least two intent prediction networks of the intent recognition model to obtain the candidate intents predicted by at least two intent prediction networks.

[0183] The target intent determination module 702 is used to determine the target intent of the target text data from at least two candidate intents; wherein the intent recognition model is trained based on the training device of the intent recognition model provided in any embodiment of the present disclosure.

[0184] The technical solution of this disclosure involves inputting target text data into a general feature extraction network of an intent recognition model to obtain target text features, and then inputting the target text features into at least two intent prediction networks of the intent recognition model to obtain candidate intents predicted by the at least two intent prediction networks. Finally, the target intent of the target text data is determined from the at least two candidate intents. This technical solution, through the intent recognition model, can improve the accuracy of intent prediction.

[0185] Furthermore, the candidate intent determination module includes:

[0186] The current target intent determination unit is used to input the current target data in the target text data into the general feature extraction network to obtain the current target features, and input the current target features into the intent prediction network to obtain the current target intent predicted by the intent prediction network;

[0187] The historical target intent determination unit is used to input historical target data from the target text data into a general feature extraction network to obtain historical target features, and then input the historical target features into an intent prediction network to obtain the historical target intent predicted by the intent prediction network.

[0188] The fusion target intent determination unit is used to determine the fusion target intent through the intent prediction network based on the current target features, current target intent, historical target features, and historical target intents.

[0189] The candidate intent determination unit is used to determine the candidate intents predicted by the intent prediction network based on historical target intents and fused target intents.

[0190] Furthermore, the fusion target intent determination unit is specifically used for:

[0191] By combining the current target intent and the historical target intent, the current target features and the historical target features are fused to obtain the fused target features;

[0192] The fused target features are input into the intent prediction network to obtain the fused target intent predicted by the intent prediction network.

[0193] Furthermore, the candidate intent determination unit is specifically used for:

[0194] Integrate historical objectives and objectives with those of the integration goal;

[0195] Based on the fusion results, the candidate intentions predicted by the intention prediction network are determined.

[0196] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0197] Figure 8 This is a block diagram of an electronic device used to implement the intent recognition model of the embodiments of this disclosure, or the intent recognition method. Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0198] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0199] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0200] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the training method for an intent recognition model or the intent recognition method. For example, in some embodiments, the training method for an intent recognition model or the intent recognition method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the training method for an intent recognition model or the intent recognition method described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a training method or an intent recognition method for the intent recognition model.

[0201] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0202] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0203] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0204] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0205] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0206] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0207] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0208] Cloud computing refers to a technology system that enables access to a shared pool of physical or virtual resources via a network. These resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for applications such as artificial intelligence and blockchain, as well as for model training.

[0209] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0210] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training an intent recognition model, the intent recognition model comprising a general feature extraction network and at least two intent prediction networks; The at least two intention prediction networks have the same network structure; the method includes: Sample text data corresponding to at least two intent prediction networks are obtained respectively; the sample text data corresponding to each intent prediction network is different; the sample text data are obtained from different scenarios; wherein, the scenarios include devices configured with screens and devices not configured with screens; The current sample data in the sample text data is input into the general feature extraction network to obtain the current sample features, and the current sample features are input into the corresponding intent prediction network to obtain the current sample intent predicted by the intent prediction network; The historical sample data in the sample text data is input into the general feature extraction network to obtain historical sample features, and the historical sample features are input into the corresponding intent prediction network to obtain the historical sample intent predicted by the intent prediction network. The intent of the fused sample is determined by the corresponding intent prediction network based on the current sample features, the current sample intent, the historical sample features, and the historical sample intent. The current sample intent, the historical sample intent, and the fused sample intent are used as the sample intent predicted by the intent prediction network. The intent prediction network is trained based on the sample intents predicted by the intent prediction network and the intent supervision data of the sample text data. The general feature extraction network is trained based on sample intents predicted by at least two intent prediction networks and intent supervision data of sample text data.

2. The method according to claim 1, wherein, The step of determining the fused sample intent through a corresponding intent prediction network based on the current sample features, the current sample intent, the historical sample features, and the historical sample intent includes: The current sample intent and the historical sample intent are used to fuse the current sample features and the historical sample features to obtain fused sample features; The fused sample features are input into the corresponding intent prediction network to obtain the fused sample intent predicted by the intent prediction network.

3. The method according to claim 2, wherein, The step of fusing the current sample features and the historical sample features to obtain fused sample features includes: Determine the similarity between the current sample intent and the historical sample intent; Based on the similarity, the current sample features, and the historical sample features, the fused sample features are determined.

4. The method according to claim 1, wherein, The step of training the intent prediction network based on the sample intents predicted by the intent prediction network and the intent supervision data of the sample text data includes: The first training loss is determined based on the intent prediction network's prediction of the current sample intent and the intent supervision data of the sample text data. The second training loss is determined based on the historical sample intents predicted by the intent prediction network and the intent supervision data of the sample text data. The third training loss is determined based on the intent prediction of the fused sample intent and the intent supervision data of the sample text data. The task training loss of the intent prediction network is determined based on the first training loss, the second training loss, and the third training loss. The intention prediction network is trained using the task training loss.

5. The method according to claim 4, wherein, The step of training the general feature extraction network based on sample intents predicted by at least two intent prediction networks and intent supervision data of sample text data includes: A general training loss is determined based on the task training loss of the at least two intent prediction networks; The general feature extraction network is trained using the general training loss.

6. An intent recognition method, comprising: The target text data is input into a general feature extraction network of the intent recognition model to obtain target text features. These target text features are then input into at least two intent prediction networks of the intent recognition model to obtain candidate intents predicted by at least two intent prediction networks, including: The current target data in the target text data is input into the general feature extraction network to obtain the current target features, and the current target features are input into the intent prediction network to obtain the current target intent predicted by the intent prediction network; The historical target data in the target text data is input into the general feature extraction network to obtain historical target features, and the historical target features are input into the intent prediction network to obtain the historical target intent predicted by the intent prediction network; The intent prediction network determines the fused target intent based on the current target features, the current target intent, the historical target features, and the historical target intent. Based on the historical target intent and the fused target intent, the candidate intent predicted by the intent prediction network is determined; The target text data is obtained from different scenarios; the scenarios include those with or without screen devices. The target intent of the target text data is determined from the at least two candidate intents; wherein the intent recognition model is trained based on the training method of the intent recognition model according to any one of claims 1-5.

7. The method according to claim 6, wherein, The step of determining the fused target intent through an intent prediction network based on the current target features, the current target intent, the historical target features, and the historical target intent includes: The current target intent and the historical target intent are used to fuse the current target features and the historical target features to obtain fused target features; The fused target features are input into the intent prediction network to obtain the fused target intent predicted by the intent prediction network.

8. The method according to claim 7, wherein, The step of determining the candidate intent predicted by the intent prediction network based on the historical target intent and the fused target intent includes: The historical target intent and the fusion target intent are fused together; Based on the fusion results, the candidate intentions predicted by the intention prediction network are determined.

9. A training apparatus for an intent recognition model, the intent recognition model comprising a general feature extraction network and at least two intent prediction networks; The at least two intention prediction networks have the same network structure; the device includes: The sample text data determination module is used to acquire sample text data corresponding to at least two intent prediction networks respectively; the sample text data corresponding to each intent prediction network is different; the sample text data is acquired from different scenarios; wherein, the scenarios include those with screen devices and those without screen devices; The sample intent determination module includes: The current sample intent determination unit is used to input the current sample data in the sample text data into the general feature extraction network to obtain the current sample features, and input the current sample features into the corresponding intent prediction network to obtain the current sample intent predicted by the intent prediction network; The historical sample intent determination unit is used to input the historical sample data in the sample text data into the general feature extraction network to obtain historical sample features, and input the historical sample features into the corresponding intent prediction network to obtain the historical sample intent predicted by the intent prediction network. The fusion sample intent determination unit is used to determine the fusion sample intent by means of a corresponding intent prediction network, based on the current sample features, the current sample intent, the historical sample features, and the historical sample intent. The sample intent determination unit is used to take the current sample intent, the historical sample intent, and the fused sample intent as the sample intent predicted by the intent prediction network. The intent prediction network training module is used to train the intent prediction network based on the sample intents predicted by the intent prediction network and the intent supervision data of the sample text data. The feature extraction network training module is used to train the general feature extraction network based on sample intents predicted by at least two intent prediction networks and intent supervision data of sample text data.

10. The apparatus according to claim 9, wherein, The fusion sample intent determination unit includes: The fusion sample feature determination subunit is used to fuse the current sample features and the historical sample features using the current sample intent and the historical sample intent to obtain fused sample features; The fusion sample intent determination subunit is used to input the fusion sample features into the corresponding intent prediction network to obtain the fusion sample intent predicted by the intent prediction network.

11. The apparatus according to claim 10, wherein, The fusion sample feature determination subunit is specifically used for: Determine the similarity between the current sample intent and the historical sample intent; Based on the similarity, the current sample features, and the historical sample features, the fused sample features are determined.

12. The apparatus according to claim 9, wherein, The intent prediction network training module is specifically used for: The first training loss is determined based on the intent prediction network's prediction of the current sample intent and the intent supervision data of the sample text data. The second training loss is determined based on the historical sample intents predicted by the intent prediction network and the intent supervision data of the sample text data. The third training loss is determined based on the intent prediction of the fused sample intent and the intent supervision data of the sample text data. The task training loss of the intent prediction network is determined based on the first training loss, the second training loss, and the third training loss. The intention prediction network is trained using the task training loss.

13. The apparatus according to claim 12, wherein, The feature extraction network training module is specifically used for: A general training loss is determined based on the task training loss of the at least two intent prediction networks; The general feature extraction network is trained using the general training loss.

14. An intent recognition device, comprising: The candidate intent determination module is used to input target text data into a general feature extraction network of the intent recognition model to obtain target text features, and input the target text features into at least two intent prediction networks of the intent recognition model to obtain candidate intents predicted by at least two intent prediction networks. The target text data is obtained from different scenarios, including scenarios with or without screen devices. The candidate intent determination module includes: The current target intent determination unit is used to input the current target data in the target text data into the general feature extraction network to obtain the current target features, and input the current target features into the intent prediction network to obtain the current target intent predicted by the intent prediction network; The historical target intent determination unit is used to input the historical target data in the target text data into the general feature extraction network to obtain historical target features, and input the historical target features into the intent prediction network to obtain the historical target intent predicted by the intent prediction network; The fusion target intent determination unit is used to determine the fusion target intent through an intent prediction network based on the current target features, the current target intent, the historical target features, and the historical target intent. A candidate intent determination unit is used to determine the candidate intent predicted by the intent prediction network based on the historical target intent and the fused target intent. A target intent determination module is used to determine the target intent of the target text data from the at least two candidate intents; wherein the intent recognition model is trained based on the training device of the intent recognition model according to any one of claims 1-5.

15. The apparatus according to claim 14, wherein, The fusion target intent determination unit is specifically used for: The current target intent and the historical target intent are used to fuse the current target features and the historical target features to obtain fused target features; The fused target features are input into the intent prediction network to obtain the fused target intent predicted by the intent prediction network.

16. The apparatus according to claim 14, wherein, The candidate intent determination unit is specifically used for: The historical target intent and the fusion target intent are fused together; Based on the fusion results, the candidate intentions predicted by the intention prediction network are determined.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the training method of the intent recognition model according to any one of claims 1-5, or the intent recognition method according to claims 6-8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the training method of the intent recognition model according to any one of claims 1-5, or the intent recognition method according to claims 6-8.

19. A computer program product comprising a computer program that, when executed by a processor, implements a training method for an intent recognition model according to any one of claims 1-5, or an intent recognition method according to claims 6-8.

Images