Method for training unknown intent detection model, unknown intent detection method and device

Abstract

Embodiments of the application disclose a method for training an unknown intention detection model, an unknown intention detection method and device, and relate to the technical field of artificial intelligence. The method for training an unknown intention detection model comprises: obtaining feature representations of a plurality of unknown intention samples and feature representations of a plurality of known intention samples by using a feature extraction model; constructing a fully connected graph by using the feature representations of the plurality of known intention samples and the feature representations of the plurality of unknown intention samples, wherein the feature representations of the known intention samples and the feature representations of the unknown intention samples are respectively taken as nodes in the fully connected graph, and the weight of an edge between the nodes is determined according to the similarity between the two nodes at the ends of the edge; determining soft labels of the unknown intention samples by using the fully connected graph, wherein the soft label is a probability value of the unknown intention sample belonging to an unknown intention; and training a classifier by using the soft labels of the unknown intention samples to obtain an unknown intention detection model. The application can improve the accuracy of unknown intention detection.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, method and apparatus for training an unknown intent detection model. Background Technology

[0002] Intent classification plays a crucial role in intelligent dialogue systems; only by identifying user intents can accurate services be provided. However, in real-world scenarios, existing intent categories often fail to cover all user intents, and new intents emerge over time. If these newly generated intents are not detected, the intelligent dialogue system may mistakenly classify them into existing intent types, thus providing services that do not meet user needs and resulting in a poor user experience. Therefore, unknown intent detection has become an important task. Summary of the Invention

[0003] In view of this, this application provides a method for training an unknown intent detection model, an unknown intent detection method, and an apparatus for realizing unknown intent detection.

[0004] This application provides the following solution:

[0005] Firstly, a method for training an unknown intent detection model is provided, the method comprising:

[0006] Obtain feature representations of multiple unknown intent samples and feature representations of multiple known intent samples;

[0007] A fully connected graph is constructed using the feature representations of the multiple known intent samples and the feature representations of the multiple unknown intent samples. The feature representations of the known intent samples and the feature representations of the unknown intent samples are respectively used as nodes in the fully connected graph, and the weight of the edge between the nodes is determined based on the similarity between the two ends of the edge.

[0008] The soft label for each unknown intent sample is determined using the fully connected graph, where the soft label is the probability value that the unknown intent sample belongs to an unknown intent.

[0009] A classifier is trained using the soft labels of each unknown intent sample to obtain an unknown intent detection model.

[0010] According to one achievable method in an embodiment of this application, the acquisition of feature representations of multiple unknown intent samples includes:

[0011] Obtain multiple known intent samples from the known intent sample set;

[0012] By adding perturbations to multiple known intent samples, feature representations of multiple unknown intent samples are obtained.

[0013] According to one achievable method in the embodiments of this application, obtaining feature representations of multiple unknown intent samples by adding perturbations to multiple known intent samples includes:

[0014] Feature extraction models are used to extract features from multiple known intent samples to obtain feature representations for each known intent sample; the feature representations of known intent samples from different intent categories are then combined to obtain feature representations for unknown intent samples; or,

[0015] A generative adversarial network (GAN) is trained using multiple known intent samples and random noise as input. The trained GAN then generates fake samples based on the known intent samples. These fake samples are then used as unknown intent samples, and a feature extraction model is used to extract feature representations for each unknown intent sample. Alternatively...

[0016] The process involves randomly adding, replacing, deleting, masking, or changing the position of at least one word in a known intent sample to obtain an unknown intent sample. A feature extraction model is then used to extract features from each unknown intent sample to obtain a feature representation of each unknown intent sample.

[0017] According to one achievable method in an embodiment of this application, determining the soft label of each unknown intent sample using the fully connected graph includes:

[0018] The soft label of the unknown intent sample is determined by using the hard label of the unknown intent sample, the hard labels of all neighboring nodes of the unknown intent sample in the fully connected graph, and the weights of the edges between the unknown intent sample and all neighboring nodes.

[0019] The hard label mentioned above is a label value used to indicate the category of the schematic diagram.

[0020] According to one achievable method in an embodiment of this application, determining the soft label of the unknown intent sample using the hard label of the unknown intent sample, the hard labels of all neighboring nodes of the unknown intent sample in the fully connected graph, and the weights of the edges between the unknown intent sample and all neighboring nodes includes:

[0021] Using formula Determine the soft label l of the unknown intent sample x g (x);

[0022] Among them, l p (x) is the hard label for x, l p (x j ) is x j The hard label of x, D is the set of all neighbor nodes of x in the fully connected graph, a j For x and x in the fully connected graph j The weights of the edges between them, where α is an empirical or experimental value.

[0023] According to one achievable method in an embodiment of this application, before obtaining the feature representations of multiple unknown intent samples and the feature representations of multiple unknown intent samples, the feature extraction model is further pre-trained in the following manner:

[0024] A pre-trained language model is used as the initial feature extraction model, and known intent samples of multiple intent categories are obtained from the known intent sample set as training data.

[0025] Based on the training data, the feature extraction model is trained using a contrastive learning approach. The training objective is to maximize the similarity between the feature representations of known intent samples of the same intent category and minimize the similarity between the feature representations of known intent samples of different intent categories.

[0026] According to one achievable method in the embodiments of this application, the unknown intent detection model is obtained by training a classifier using the soft labels of each unknown intent sample, including:

[0027] A classifier is trained using the feature representations of each unknown intent sample and the soft labels of the unknown intent samples. An unknown intent detection model is obtained using a feature extraction model and the trained classifier. This model is used to detect whether the input text to be detected belongs to the unknown intent type; or...

[0028] Using samples of unknown intent and their soft labels, an unknown intent detection model, including a feature extraction model and a classifier, is trained. This model is used to detect whether the input text to be detected belongs to the unknown intent type; or...

[0029] A classifier is trained using the feature representations and labels of known intent samples, the feature representations and soft labels of unknown intent samples, and the trained classifier. The unknown intent detection model is then obtained using a feature extraction model and the trained classifier. This model is used to detect the intent type of the input text to be detected, including known intent types and unknown intent types; or...

[0030] Using multiple known intent samples and their labeled tags, as well as each unknown intent sample and its soft label, an unknown intent detection model including a feature extraction model and a classifier is trained. The unknown intent detection model is used to detect the intent type of the input text to be detected, and the intent type includes known intent types and unknown intent types.

[0031] According to one achievable manner in an embodiment of this application, the classifier includes a first classifier and a second classifier;

[0032] During the training process of the unknown intent detection model, the first classifier and the second classifier are trained using a co-training method.

[0033] The unknown intent detection model trained is used to determine the unknown intent detection result by combining the classification results of the first classifier and the second classifier.

[0034] Secondly, a method for detecting unknown intent is provided, the method comprising:

[0035] Obtain the text to be detected;

[0036] The text to be detected is input into the unknown intent detection model, and the detection result of the unknown intent detection model on the text to be detected is obtained. The detection result includes whether it is an unknown intent type.

[0037] The unknown intent detection model is pre-trained using the method described in the first aspect above.

[0038] Thirdly, a smart dialogue method is provided for use in a smart dialogue system, the method comprising:

[0039] Obtain text input by the user or text obtained by recognizing the user's speech;

[0040] The acquired text is input into the unknown intent detection model as the text to be detected, and the detection result of the unknown intent detection model on the text to be detected is obtained.

[0041] If the detection result of the text to be detected is of unknown intent type, then a response text for the user is generated using a preset script, or a human service is triggered to access the intelligent dialogue system to respond to the user.

[0042] Fourthly, an apparatus for training an unknown intent detection model is provided, the apparatus comprising:

[0043] The feature acquisition unit is configured to acquire feature representations of multiple unknown intent samples and feature representations of multiple unknown intent samples;

[0044] The graph construction unit is configured to construct a fully connected graph using the feature representations of the plurality of known intent samples and the feature representations of the plurality of unknown intent samples, wherein the feature representations of the known intent samples and the feature representations of the unknown intent samples are respectively used as nodes in the fully connected graph, and the weight of the edge between the nodes is determined based on the similarity between the nodes at both ends of the edge.

[0045] The label determination unit is configured to determine a soft label for each unknown intent sample using the fully connected graph, wherein the soft label is a probability value that the unknown intent sample belongs to an unknown intent;

[0046] The first training unit is configured to train a classifier using the soft labels of the unknown intent samples to obtain an unknown intent detection model.

[0047] Fifthly, an unknown intent detection device is provided, the device comprising:

[0048] The text acquisition unit is configured to acquire the text to be detected.

[0049] An intent detection unit is configured to input the text to be detected into an unknown intent detection model and obtain the detection result of the unknown intent detection model on the text to be detected, wherein the detection result includes whether it is an unknown intent type;

[0050] The unknown intent detection model is pre-trained using the apparatus described in the fourth aspect above.

[0051] According to a sixth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first, second, and third aspects above.

[0052] According to the seventh aspect, an electronic device is provided, comprising:

[0053] One or more processors; and

[0054] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first, second, and third aspects above.

[0055] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0056] 1) This application constructs a fully connected graph using the feature representations of known intent samples and unknown intent samples, and uses the fully connected graph to determine the soft labels of unknown intent samples. Based on the soft labels, a classifier is trained, making the obtained unknown intent detection model more accurate in detecting unknown intents and significantly improving the unknown intent detection effect.

[0057] 2) By adding perturbations to multiple known intent samples, feature representations of unknown intent samples can be obtained, eliminating the need for additional manual searching or construction of unknown intent samples and reducing labor costs.

[0058] 3) In this application, the soft label of the unknown intent sample is determined by using the hard label of the unknown intent sample, the hard labels of all neighboring nodes of the unknown intent sample in the fully connected graph, and the weights of the edges between the unknown intent sample and all neighboring nodes. This method of determining the soft label of the unknown intent sample through graph smoothing makes the soft label of the unknown intent sample more accurate, thereby further improving the detection performance of the trained unknown intent detection model.

[0059] 4) This application uses known intent samples of multiple intent categories in the known intent sample set as training data and adopts a contrastive learning approach to train the feature extraction model. This enables the trained feature extraction model to quickly learn the sample features of various intent categories based on the pre-trained language model, making the extraction of sample feature representations more accurate.

[0060] 5) This application can use co-training to train the dual-head classifier during the training process of the unknown intent detection model, so as to learn the soft labels of unknown intent samples more smoothly. The trained unknown intent detection model combines the classification results of the dual-head classifier to determine the unknown intent detection result, thereby further improving the detection effect.

[0061] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is an exemplary system architecture diagram that can be applied to embodiments of this application;

[0064] Figure 2 A flowchart illustrating the method for training an unknown intent detection model provided in this application embodiment;

[0065] Figure 3 A schematic diagram illustrating the principle of a generative adversarial network provided in an embodiment of this application;

[0066] Figure 4 A flowchart illustrating the method for training a feature extraction model provided in this application embodiment;

[0067] Figure 5 A schematic diagram illustrating the principle of the training feature extraction model provided in the embodiments of this application;

[0068] Figure 6a and Figure 6b These are schematic diagrams illustrating the structures of two unknown intent detection models provided in embodiments of this application;

[0069] Figure 7 A flowchart of the unknown intent detection method provided in the embodiments of this application;

[0070] Figure 8 A schematic block diagram of an apparatus for training an unknown intent detection model provided in an embodiment of this application;

[0071] Figure 9 A schematic block diagram of a training unknown intent detection device provided in an embodiment of this application;

[0072] Figure 10 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0074] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0075] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0076] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0077] The challenge of unknown intent detection lies in the lack of unknown intent samples; it's difficult to collect such samples in specific scenarios. Currently, some methods exist to construct unknown intent samples and label them with "hard labels." Hard labels are valued to indicate the category of the symbol, for example, 1 for unknown intent and 0 for known intent. The constructed unknown intent samples and their hard labels are then used to train an unknown intent detection model. However, this approach is limited by the accuracy of the generated unknown intent samples (e.g., the generated unknown intent samples are actually known intent samples), which can easily lead to poor accuracy in unknown intent detection.

[0078] In view of this, this application proposes a novel approach to train an unknown intent detection model based on soft labels. To facilitate understanding of this application, the system architecture used in this application will first be described.

[0079] Figure 1 An exemplary system architecture to which embodiments of this application can be applied is shown. For example... Figure 1 As shown, the system architecture includes a model training device that trains the unknown intent detection model offline, and an unknown intent detection device that performs unknown intent detection on the text to be detected online.

[0080] The model training device uses the method provided in the embodiments of this application to generate training samples and train the model to obtain an unknown intent detection model.

[0081] The unknown intent detection device uses an established unknown intent detection model to detect unknown intent in the text to be detected and outputs the detection result as to whether it is an unknown intent.

[0082] The model training device and the unknown intent detection device can be configured as separate servers, or they can be configured on the same server or server group, or they can be configured on a separate or the same cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPs) services, such as high management difficulty and weak service scalability. The model training device and the unknown intent detection device can also be configured on computer terminals with strong computing capabilities.

[0083] It should also be noted that Figure 1 Taking the online execution of detection by the unknown intent detection device as an example, in some application scenarios, the unknown intent detection device can also perform unknown intent detection offline, such as batch detection of the text to be detected, and this application does not limit this.

[0084] It should be understood that Figure 1The number of model training devices, unknown intent detection devices, and unknown intent detection models shown in the diagram is merely illustrative. Depending on implementation needs, any number of model training devices, unknown intent detection devices, and unknown intent detection models can be included.

[0085] Figure 2 This is a flowchart of a method for training an unknown intent detection model provided in an embodiment of this application. The method can be described by… Figure 1 The model training device in the system shown is executed. For example... Figure 2 As shown, the method may include the following steps:

[0086] Step 202: Obtain the feature representations of multiple unknown intent samples.

[0087] Step 204: Construct a fully connected graph using the feature representations of multiple known intent samples and multiple unknown intent samples. The feature representations of known intent samples and unknown intent samples are used as nodes in the fully connected graph, and the weight of the edge between nodes is determined based on the similarity between the nodes at both ends of the edge.

[0088] Step 206: Use the fully connected graph to determine the soft label of each unknown intent sample. The soft label is the probability value of the unknown intent sample belonging to the unknown intent.

[0089] Step 208: Train a classifier using the soft labels of each unknown intent sample to obtain an unknown intent detection model.

[0090] As can be seen from the above process, this application constructs a fully connected graph using the feature representations of known intent samples and unknown intent samples, and uses the fully connected graph to determine the soft labels of unknown intent samples. Based on the soft labels, a classifier is trained, making the detection effect of the obtained unknown intent detection model more accurate.

[0091] The following describes each step in the above process. First, step 202, namely "obtaining the feature representations of multiple unknown intent samples and the feature representations of multiple unknown intent samples", will be described in detail with reference to the embodiment.

[0092] In this application embodiment, "intent" refers to the purpose or theme contained in the text. Therefore, intent detection is to identify the purpose or theme contained in the text, which is a type of natural language understanding task.

[0093] The goal of unknown intent detection is to identify intents that do not belong to known intent types. Known intent types refer to intent types for which annotations and models have been trained. Assuming k known intent types are known, the known intent sample set can be represented as D. I ={(x i ,yi )}, where x i This represents a known intent sample, such as text labeled with intent type tags, y i x represents i The intent type label is typically a hard label that indicates the label value of the schematic category. For example, 0, 1, etc. are used to indicate different intent categories.

[0094] In this embodiment of the application, a known intent sample set D can be used. I Perturbations are added to known intent samples to obtain feature representations of unknown intent samples. This can be achieved, but is not limited to, the following methods:

[0095] The first approach: using the feature mixing method.

[0096] First, a feature extraction model is used to extract features from multiple known intent samples, obtaining feature representations for each known intent sample. Then, the feature representations of known intent samples from different intent categories are blended to obtain feature representations for unknown intent samples. This blending can be achieved using methods such as interpolation or weighted averaging.

[0097] As can be seen, this method is equivalent to mixing the feature representations of known intent samples from different intent categories to produce a feature representation that does not belong to any intent category as an unknown intent sample.

[0098] The second approach: Using GANs (Generative Adversarial Networks)

[0099] GANs consist of two neural networks: a Generator and a Discriminator. The generator aims to learn to generate fake sample distributions to fool the discriminator, while the discriminator aims to learn to distinguish between the real and fake distributions generated by the generator.

[0100] In this embodiment, a generative adversarial network is trained using multiple known intent samples and random noise as input. The trained generative adversarial network generates fake samples for the multiple known intent samples and uses the fake samples as unknown intent samples.

[0101] like Figure 3As shown, known intent samples are used as real samples, and these real samples serve as one input to the discriminator. Random noise is used as input to the generator, which generates fake samples and uses these fake samples as the other input to the discriminator. The discriminator continuously learns to better distinguish between real and fake samples, while the generator continuously learns to generate fake samples that can confuse the discriminator. The two work against each other to achieve optimization. GANs can generate fake samples corresponding to known intent samples, and these fake samples can be used as samples with unknown intent.

[0102] After obtaining the unknown intent samples, the feature extraction model is used to extract the feature representation of each unknown intent sample.

[0103] The third method: Editing known intent samples

[0104] The known intent sample is edited by at least one word to obtain the unknown intent sample. Then, the feature extraction model is used to extract features from each unknown intent sample to obtain the feature representation of each unknown intent sample.

[0105] The editing process can include one or any combination of random addition, replacement, deletion, masking, or repositioning. In other words, at least one word in a known intent sample is randomly added, replaced, deleted, masked, or repositioned, and the resulting text is considered an unknown intent sample.

[0106] Besides the three methods mentioned above, other methods can be used to add perturbations to known intent samples. The first method actually obtains the feature representation of the unknown intent sample directly through feature mixing, while the second and third methods actually obtain the unknown intent sample first, and then further obtain the feature representation of the unknown intent sample.

[0107] All three methods mentioned above involve feature extraction models, which are used to extract features from input samples to obtain feature representations of the samples.

[0108] One feasible approach is to use existing models, such as pre-trained language models, as feature extraction models.

[0109] This application provides another, more preferred method for training the above-mentioned feature extraction model. For example... Figure 4 The above can specifically include the following steps:

[0110] Step 402: Obtain a pre-trained language model as the initial feature extraction model.

[0111] In this application embodiment, the feature extraction model can be trained based on a pre-trained language model. Pre-trained language models such as BERT (Bidirectional Encoder Representation from Transformers), XLNet, and GPT (Generative Pre-Training) are used as the initial feature extraction model, and further training is performed on this basis.

[0112] Step 404: Obtain known intent samples of multiple intent categories from the known intent sample set as training data.

[0113] Since this embodiment employs contrastive learning to train the feature extraction model, the principle of contrastive learning is to narrow the distance between features of samples of the same category and widen the distance between features of samples of different categories. Therefore, when selecting training data, known intent samples of different intent categories can be obtained from the known intent sample set to construct the training data.

[0114] Training data can contain multiple training samples, each containing pairs of positive and negative samples. For example, one sample x and other samples x belonging to the same intent category as that sample. + To form a positive sample pair, consider sample x and other samples x belonging to different intent categories. - This constitutes a negative sample pair.

[0115] Step 406: Based on the above training data, the feature extraction model is trained using contrastive learning. The training objective is to maximize the similarity between the feature representations of known intent samples of the same intent category and minimize the similarity between the feature representations of known intent samples of different intent categories.

[0116] like Figure 5 As shown, the above training objective can be understood as:

[0117] d(f(x),f(x + ))<<d(f(x),f(x - ))(1)

[0118] Here, f() is the feature extraction function used by the feature extraction model. d() is the function that calculates the distance between vectors, such as Euclidean distance. In other words, a loss function can be constructed such that the distance between the feature representations of samples with the same intent category is as small as possible compared to the distance between the feature representations of samples with different intent categories. In each iteration, the model parameters of the feature extraction model can be updated using methods such as gradient descent based on the value of the loss function, until a preset training termination condition is met. The training termination condition may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0119] The following describes step 204, namely "constructing a fully connected graph using the feature representations of multiple known intent samples and multiple unknown intent samples", in detail with reference to the embodiments.

[0120] For a known intent sample, the feature extraction model obtained through the above training can also obtain the feature representation of each known intent sample.

[0121] Since some of the unknown intent samples obtained by adding perturbations to the known intent samples as shown in step 202 do indeed belong to the unknown intent type, while others still belong to the known intent category after adding perturbations, it is highly likely that known intent samples are mixed in with the unknown intent samples obtained in step 202. If the obtained unknown intent samples are directly assigned hard labels to train the unknown intent detection model, it will cause the unknown intent detection model to be inaccurate and have a low recognition accuracy. Therefore, in this step, the feature representations of known intent samples and unknown intent samples are used to construct a fully connected graph. The purpose of this fully connected graph is to obtain the soft labels of the unknown intent samples (the probability that they belong to unknown intent), that is, to obtain the true distribution of unknown intent samples.

[0122] The constructed fully connected graph includes nodes and edges. Assume that the multiple known intent sample sets D used to construct the fully connected graph are... I Multiple samples with unknown intent constitute the sample set D. P Then using D I ∪D P The set D IP The feature representations of each sample in the graph are used as nodes in the fully connected graph. The edges between nodes represent the relationships between them, and the weight of an edge is determined by the similarity between the nodes at both ends of the edge. That is, the more similar the feature representations of two nodes are, the larger the weight of the edge between them, and vice versa.

[0123] The fully connected graph created in the above manner is actually a D IP The fully connected graph after each sample is mapped to the feature vector space reflects DIP The distribution of each sample in the feature vector space.

[0124] The following describes step 206, namely "determining the soft labels of each unknown intent sample using a fully connected graph", in detail with reference to the embodiments.

[0125] As one possible approach, this application employs graph smoothing to determine the soft label of each unknown intent sample. During graph smoothing, the soft label value of an unknown intent sample is related not only to itself but also to all its connected neighboring nodes. That is, the soft label of the unknown intent sample is determined using its hard label, the hard labels of all its neighboring nodes in the fully connected graph, and the weights of the edges between the unknown intent sample and all its neighboring nodes.

[0126] For example, the soft label l of an unknown intent sample x can be determined using the following formula. g (x):

[0127]

[0128] Among them, l p (x) is the hard label for x, l p (x j ) is x j hard label, D N Let a be the set of all neighbor nodes of x in the fully connected graph. j For x and x in a fully connected graph j The weights of the edges between them, where α is an empirical or experimental value.

[0129] The following describes step 208, namely "training a classifier using the soft labels of each unknown intent sample to obtain an unknown intent detection model," in detail with reference to an embodiment.

[0130] This step can be implemented in, but is not limited to, the following ways:

[0131] The first approach involves directly training a classifier using the feature representations and soft labels of the unknown intent samples. This classifier, obtained through a feature extraction model and the trained classifier, forms the unknown intent detection model. In this approach, the classifier can be a binary classifier, with the feature representations of the unknown intent samples serving as input and the corresponding soft labels as the target output.

[0132] When training the classifier, the training objective is to minimize the difference between the probability that the classifier predicts a sample with unknown intent as belonging to an unknown intent and the soft label of that unknown intent sample.

[0133] In each iteration, the classifier's model parameters are updated using the loss function value and methods such as gradient descent, until a preset training termination condition is met. This termination condition may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0134] After the classifier is trained, the feature extraction model and the classification model are used to construct an unknown intent detection model.

[0135] The second approach involves training an unknown intent detection model, including a feature extraction model and a classifier, using each unknown intent sample and its soft label. The classifier in this approach is also a binary classifier. Unlike the first approach, however, the feature extraction model and classifier are trained using unknown intent samples and their soft labels. Specifically, the unknown intent samples are used as input to the feature extraction model, and the corresponding soft labels are used as the target output of the classifier. The model parameters of both the feature extraction model and the classifier are updated during training.

[0136] When training the unknown intent detection model consisting of the feature extraction model and the classifier, the training objective is the same as in the first method: to minimize the difference between the probability predicted by the classifier for an unknown intent sample belonging to an unknown intent and the soft label of that unknown intent sample.

[0137] In each iteration, the model parameters of the feature extraction model and classifier are updated using methods such as gradient descent, based on the value of the loss function, until a preset training termination condition is met. This termination condition may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0138] In both of the above methods, the unknown intent detection model is used to detect the text to be detected, and the output detection result is a binary classification result, that is, whether the text to be detected belongs to the unknown intent type.

[0139] The third approach involves using the feature representations of known intent samples and the labels attached to them, as well as the feature representations and soft labels of unknown intent samples, to train a classifier. The trained classifier is then used to obtain an unknown intent detection model through a feature extraction model and the trained classifier.

[0140] Unlike the first approach, this method uses a multi-classifier. The feature representations of both known and unknown intent samples serve as input to the classifier, while the corresponding labels are the target output. In other words, the classifier learns not only to predict the known intent type of known intent samples but also to predict the unknown intent type of unknown intent samples.

[0141] In each iteration, the classifier's model parameters are updated using the loss function value and methods such as gradient descent, until a preset training termination condition is met. This termination condition may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0142] The fourth approach: Use multiple known intent samples and their labeled tags, as well as each unknown intent sample and its soft label, to train an unknown intent detection model that includes a feature extraction model and a classifier.

[0143] Unlike the second approach, this method uses a multi-classifier. Both known and unknown intent samples serve as input to the feature extraction model, with their corresponding labels as the classifier's target output. In other words, the feature extraction model and classifier must learn not only to predict the known intent type of known intent samples but also to predict the unknown intent type of unknown intent samples.

[0144] In each iteration, the model parameters of the feature extraction model and classifier are updated using methods such as gradient descent, based on the value of the loss function, until a preset training termination condition is met. This termination condition may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0145] In the third and fourth methods mentioned above, the unknown intent detection model is used to detect the text to be detected, and the output detection result is the intent type to which the text to be detected belongs. This intent type includes known intent types and unknown intent types.

[0146] In the four methods described above, the classifier can be a single-head classifier, with the structure as follows: Figure 6a As shown in the diagram. Taking the first method described above as an example, the loss function L1 used can be, for example:

[0147]

[0148] Where CE() is the cross-entropy loss function, and g[] is the function used by the classifier.

[0149] As another feasible approach, the classifier described above can also be a two-headed classifier, with a structure as follows: Figure 6b As shown in the diagram. This structure can employ a co-training framework to better learn soft labels. In the co-training framework, two classifiers can be used on top of a shared feature extraction model: a first classifier and a second classifier. Specifically, the first and second classifiers are trained using the feature representations and soft labels of the unknown intent samples, respectively.

[0150] It should be noted that the terms "first," "second," and "third" used in this disclosure do not have any limitations in terms of size, order, or number; they are merely used to distinguish them by name. For example, "first classifier" and "second classifier" are used to distinguish two classifiers by name. Similarly, "first training," "second training," and "third training" are used to distinguish three training processes by name.

[0151] The collaborative training process can specifically include:

[0152] First, based on the feature representations and soft labels of each unknown intent sample, the first and second classifiers are trained separately. To improve the generalization performance of the first and second classifiers, techniques such as droupout can be used during the first training process. Drupout involves randomly ignoring or disabling a certain proportion of neurons during training; these neurons are randomly "discarded," making the trained model less sensitive to the weights of specific neurons. Since droupout is an existing technique, it will not be detailed here.

[0153] Then, the classification result of the first classifier obtained from the first training on the feature representation of the unknown intent sample is used as the training target of the second classifier, and the first classifier is trained a second time; and the classification result of the second classifier obtained from the second training on the feature representation of the unknown intent sample is used as the training target of the first classifier, and the first classifier is trained a third time.

[0154] The collaborative training process embodied in the second and third training sessions described above is essentially a process of redefining the soft labels of samples with unknown intent, using the detection results of one classifier for samples with unknown intent to optimize the detection of the other classifier. For example, the loss function L in the second and third training sessions... CO It can be represented as:

[0155]

[0156] in,

[0157] The function used by the second classifier is g1[f(x)], and the function used by the third classifier is g2[f(x)], where β is an empirical or experimental value.

[0158] After training the second and third classifiers, the feature extraction model, the second classifier, and the third classifier are used to construct an unknown intent detection model, such as... Figure 6b As shown in the image.

[0159] After the unknown intent detection model is pre-trained using the method provided in the above embodiments, it can be used for detection. Figure 7 This is a flowchart of an unknown intent detection method provided in an embodiment of this application. The method can be... Figure 1 The unknown intent detection device in the system shown is executed. For example... Figure 7 As shown, the following steps may be included:

[0160] Step 702: Obtain the text to be detected.

[0161] Depending on the specific scenario, the text to be detected can be of different types. It can originate from user input, be text resources from the internet, and so on. Several practical application scenarios will be listed later.

[0162] Step 704: Input the text to be detected into the unknown intent detection model and obtain the detection result of the text to be detected. The detection result includes whether it is an unknown intent type.

[0163] If the structure of the unknown intent detection model is as follows Figure 6a As shown in the diagram, after inputting the text to be detected, the feature extraction module in the unknown intent detection model extracts features from the text to obtain its feature representation. The first classifier uses this feature representation to obtain the detection result for the text. If the classifier in the unknown intent detection model is a binary classifier, the output is the probability that the text belongs to the unknown intent type, further determining whether the text belongs to the unknown intent type. If the classifier in the unknown intent detection model is a multi-classifier, the output is the intent type to which the text belongs, which includes known intent types and unknown intent types. For example, k+1 types can be detected, where k are known intent types and the (k+1)th type is an unknown intent type.

[0164] If the structure of the unknown intent detection model is as follows Figure 6bAs shown in the diagram, after inputting the text to be detected, the feature extraction module in the unknown intent detection model extracts features from the text to obtain its feature representation. The first classifier and the second classifier use the feature representation of the text to obtain the probabilities that the text belongs to each known intent type and the unknown intent type, denoted as g1[f(x)] and g2[f(x)], respectively. The unknown intent detection model combines the detection results of the first classifier and the second classifier to determine the detection result. Specifically, both the first classifier and the second classifier output probability values ​​for the text to be detected. Combining the probabilities output by the first classifier and the second classifier yields a total probability value g(y|x), which is used to determine whether the text to be detected belongs to each intent type. Taking the probability mapped to the unknown intent type as an example, the total probability g(y|x) mapped to the unknown intent type can be expressed by the following formula:

[0165] g(y|x)=(g1[f(x)]+g2[f(x)]) / 2 (6)

[0166] The methods provided in this application embodiment can be applied to various application scenarios, including but not limited to:

[0167] 1) Intelligent Dialogue

[0168] The system acquires text input by the user or text obtained by recognizing the user's speech; it inputs the acquired text as the text to be detected into the unknown intent detection model, and obtains the detection result of the unknown intent detection model on the text to be detected; if the detection result of the text to be detected is of the unknown intent type, it generates a response text for the user using a preset script, or triggers human service access to the intelligent dialogue system to respond to the user.

[0169] For example, when a user enters text on the interface for chatting with customer service while using the intelligent customer service system, the intelligent customer service system can use the method provided in this application embodiment to detect unknown intent. If the detection result indicates that the input text belongs to the type of unknown intent, a response text for the user can be generated using a preset script, such as "Sorry, I don't quite understand what you mean, please rephrase it"; or the user can be transferred to a human customer service representative for processing.

[0170] For another example, when a user is using a smart speaker, the smart speaker acquires the user's voice and uses the speech recognition result corresponding to the user's voice as the text to be detected. It then uses the method provided in this application's embodiment to detect unknown intent. If the detection result indicates an unknown intent, a response text can be generated for the user using a preset script, and the response speech can be synthesized and played back to the user using speech synthesis technology, for example, generating a speech saying "Sorry, I didn't understand you."

[0171] 2) News Classification;

[0172] Given the large amount of news text available on news websites, it is necessary to categorize and display these texts within their respective categories before they are uploaded and presented. Therefore, the method provided in this application embodiment can be used to detect unknown intent in each news text to be uploaded. News texts identified as having unknown intent can be manually identified and processed; news texts identified as having known intent can be further identified in terms of their specific type and displayed under that specific type.

[0173] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0174] According to another embodiment, an apparatus for training an unknown intent detection model is provided. Figure 8 A schematic block diagram of an apparatus for training an unknown intent detection model according to one embodiment is shown, the apparatus being disposed in... Figure 1 The model training device in the architecture shown. Figure 8 As shown, the device 800 includes: a feature acquisition unit 801, a graph construction unit 802, a label determination unit 803, and a first training unit 804, and may further include a second training unit 805. The main functions of each component are as follows:

[0175] The feature acquisition unit 801 is configured to acquire feature representations of multiple unknown intent samples and feature representations of multiple unknown intent samples.

[0176] The graph construction unit 802 is configured to construct a fully connected graph using the feature representations of multiple known intent samples and multiple unknown intent samples. The feature representations of the known intent samples and the feature representations of the unknown intent samples are used as nodes in the fully connected graph, and the weight of the edge between the nodes is determined based on the similarity between the nodes at both ends of the edge.

[0177] The label determination unit 803 is configured to determine the soft label of each unknown intent sample using a fully connected graph. The soft label is the probability value that the unknown intent sample belongs to an unknown intent.

[0178] The first training unit 804 is configured to train a classifier using the soft labels of each unknown intent sample to obtain an unknown intent detection model.

[0179] As one possible implementation method, the feature acquisition unit 801 can be specifically configured to acquire multiple known intent samples from a set of known intent samples; and to acquire feature representations of multiple unknown intent samples by adding perturbations to the multiple known intent samples.

[0180] When the feature acquisition unit 801 acquires feature representations of multiple unknown intent samples by adding perturbations to multiple known intent samples, it may use, but is not limited to, the following four methods:

[0181] The first approach involves using a feature extraction model to extract features from multiple known intent samples to obtain feature representations for each known intent sample; then, the feature representations of known intent samples from different intent categories are combined to obtain feature representations for unknown intent samples.

[0182] The second approach involves training a generative adversarial network (GAN) using multiple known intent samples and random noise as input. The trained GAN then generates fake samples based on these fake samples, which are then used as unknown intent samples. A feature extraction model is then used to extract the feature representations of each unknown intent sample.

[0183] The third approach involves randomly adding, replacing, deleting, masking, or changing the position of at least one word in a known intent sample to obtain an unknown intent sample. Then, a feature extraction model is used to extract features from each unknown intent sample to obtain a feature representation of each unknown intent sample.

[0184] As one possible implementation, the label determination unit 803 can be specifically configured to: determine the soft label of the unknown intent sample using the hard label of the unknown intent sample, the hard labels of all neighboring nodes of the unknown intent sample in the fully connected graph, and the weights of the edges between the unknown intent sample and all neighboring nodes; wherein the hard label is a label value used to indicate the category of the schematic diagram.

[0185] For example, the label determination unit 803 can use formula l g (x)=α·l p (x)+(1-α)∑a j ·l p (x j ), determine the soft label l of the unknown intent sample x g (x); where l p (x) is the hard label for x, l p (x j ) is x j The hard label of x, D is the set of all neighbor nodes of x in the fully connected graph, and a j For x and x in a fully connected graph j The weights of the edges between them, where α is an empirical or experimental value.

[0186] Furthermore, the second training unit 805 is configured to acquire a pre-trained language model as the initial feature extraction model, and acquire known intent samples of multiple intent categories as training data from the known intent sample set; based on the training data, the feature extraction model is trained using a contrastive learning approach, with the training objective being to maximize the similarity between the feature representations of known intent samples of the same intent category and minimize the similarity between the feature representations of known intent samples of different intent categories.

[0187] Specifically, the first training unit 804 mentioned above can be implemented in, but is not limited to, the following four ways:

[0188] The first approach involves training a classifier using the feature representations of each unknown intent sample and the soft labels of the unknown intent samples. The unknown intent detection model is then obtained using the feature extraction model and the trained classifier. This model is used to detect whether the input text to be detected belongs to the unknown intent type.

[0189] The second approach involves using each unknown intent sample and its soft label to train an unknown intent detection model that includes a feature extraction model and a classifier. This model is used to detect whether the input text to be detected belongs to the unknown intent type.

[0190] The third approach involves using the feature representations of known intent samples and the labels attached to them, as well as the feature representations and soft labels of unknown intent samples, to train a classifier. The classifier is then used to obtain an unknown intent detection model, which is used to detect the intent type of the input text to be detected. The intent type includes known intent types and unknown intent types.

[0191] The fourth approach is to train an unknown intent detection model, which includes a feature extraction model and a classifier, by using multiple known intent samples and their labeled tags, as well as each unknown intent sample and its soft label. The unknown intent detection model is used to detect the intent type of the input text to be detected, which includes known intent types and unknown intent types.

[0192] As one possible approach, the classifier described above could be a single-head classifier.

[0193] As another possible approach, the classifier described above can be a two-headed classifier, which includes a first classifier and a second classifier.

[0194] In the training process of the unknown intent detection model, the first training unit 804 uses co-training to train the first classifier and the second classifier; the trained unknown intent detection model combines the classification results of the first classifier and the second classifier to determine the unknown intent detection result.

[0195] Figure 9 A schematic block diagram of an unknown intent detection device according to one embodiment is shown. Figure 9 As shown, the device 900 includes a text acquisition unit 901 and an intent detection unit 902. The main functions of each component are as follows:

[0196] The text acquisition unit 901 is configured to acquire the text to be detected.

[0197] The intent detection unit 902 is configured to input the text to be detected into the unknown intent detection model, obtain the detection result of the unknown intent detection model on the text to be detected, and the detection result includes whether it is an unknown intent type; wherein the unknown intent detection model adopts, for example, Figure 8 The device described herein is obtained through pre-training.

[0198] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. Components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0199] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0200] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0201] And an electronic device, comprising:

[0202] One or more processors; and

[0203] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0204] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0205] in, Figure 10 An exemplary architecture of an electronic device is shown, which may include a processor 1010, a video display adapter 1011, a disk drive 1012, an input / output interface 1013, a network interface 1014, and a memory 1020. The processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, and memory 1020 can communicate with each other via a communication bus 1030.

[0206] The processor 1010 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0207] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system 1021 for controlling the operation of the electronic device 1000, and the basic input / output system (BIOS) 1022 for controlling the low-level operations of the electronic device 1000. Additionally, it can store a web browser 1023, a data storage management system 1024, and a model training device / unknown intent detection device 1025, etc. The aforementioned model training device / unknown intent detection device 1025 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 1020 and executed by the processor 1010.

[0208] Input / output interface 1013 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0209] The network interface 1014 is used to connect the communication module (not shown in the figure) to enable communication and interaction between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0210] Bus 1030 includes a pathway for transmitting information between various components of the device (e.g., processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, and memory 1020).

[0211] It should be noted that although the above-described device only shows the processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, memory 1020, bus 1030, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0212] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0213] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for training an unknown intent detection model, characterized in that, The method includes: Obtain feature representations of multiple unknown intent samples and multiple known intent samples, wherein the unknown intent samples are text to be labeled with intent type tags, and the known intent samples are text already labeled with the intent type tags; A fully connected graph is constructed using the feature representations of the multiple known intent samples and the feature representations of the multiple unknown intent samples. The feature representations of the known intent samples and the feature representations of the unknown intent samples are respectively used as nodes in the fully connected graph, and the weight of the edge between the nodes is determined based on the similarity between the nodes at both ends of the edge. The soft label for each unknown intent sample is determined using the fully connected graph, where the soft label is the probability value that the unknown intent sample belongs to an unknown intent. A classifier is trained using the soft labels of each unknown intent sample to obtain an unknown intent detection model, wherein the unknown intent detection model is used to detect whether the input text to be detected belongs to the unknown intent type.

2. The method according to claim 1, characterized in that, The feature representation of obtaining multiple unknown intent samples includes: Obtain multiple known intent samples from the known intent sample set; By adding perturbations to the multiple known intent samples, feature representations of the multiple unknown intent samples are obtained.

3. The method according to claim 2, characterized in that, By adding perturbations to multiple known intent samples, feature representations of multiple unknown intent samples are obtained, including: Feature extraction models are used to extract features from the multiple known intent samples to obtain feature representations for each known intent sample; the feature representations of the known intent samples from different intent categories are then combined to obtain feature representations for the unknown intent samples; or... A generative adversarial network (GAN) is trained using the multiple known intent samples and random noise as input. The trained GAN generates fake samples based on the known intent samples. These fake samples are then used as unknown intent samples, and a feature extraction model is used to extract feature representations for each unknown intent sample. Alternatively... The known intent samples are processed by randomly adding, replacing, deleting, masking, or changing the position of at least one word to obtain unknown intent samples. Feature extraction models are then used to extract features from each unknown intent sample to obtain the feature representation of each unknown intent sample.

4. The method according to claim 1, characterized in that, Determining the soft labels for each unknown intent sample using the fully connected graph includes: The soft label of the unknown intent sample is determined by using the hard label of the unknown intent sample, the hard labels of all neighbor nodes of the unknown intent sample in the fully connected graph, and the weights of the edges between the unknown intent sample and all neighbor nodes. The hard label mentioned above is a label value used to indicate the category of the schematic diagram.

5. The method according to claim 4, characterized in that, The process of determining the soft label of the unknown intent sample using the hard label of the unknown intent sample, the hard labels of all neighbor nodes of the unknown intent sample in the fully connected graph, and the weights of the edges between the unknown intent sample and all neighbor nodes includes: Using formula Determine the unknown intent sample soft tags ; in, for Hard labels, for Hard labels, for The set consisting of all neighbor nodes in the fully connected graph. For the fully connected graph and The weight of the edges between them. These are empirical or experimental values.

6. The method according to claim 3, characterized in that, Before obtaining the feature representations of multiple unknown intent samples, the feature extraction model is pre-trained using the following method: A pre-trained language model is used as the initial feature extraction model, and known intent samples of multiple intent categories are obtained from the known intent sample set as training data. Based on the training data, the feature extraction model is trained using a contrastive learning approach. The training objective is to maximize the similarity between the feature representations of known intent samples of the same intent category and minimize the similarity between the feature representations of known intent samples of different intent categories.

7. The method according to any one of claims 1 to 5, characterized in that, The classifier is trained using the soft labels of the aforementioned unknown intent samples to obtain the unknown intent detection model, which includes: The classifier is trained using the feature representations of each unknown intent sample and the soft labels of the unknown intent samples; the unknown intent detection model is obtained using the feature extraction model and the trained classifier; or, Using each unknown intent sample and its soft label, train the unknown intent detection model, which includes a feature extraction model and the classifier; or, The classifier is trained using the feature representations and labels of the known intent samples, the feature representations and soft labels of the unknown intent samples, and the trained classifier using a feature extraction model. The unknown intent detection model is then used to detect the intent type of the input text to be detected, including known intent types and unknown intent types; or... Using multiple known intent samples and the labels labeled on the multiple known intent samples, as well as each unknown intent sample and the soft labels of each unknown intent sample, an unknown intent detection model including a feature extraction model and the classifier is trained. The unknown intent detection model is used to detect the intent type to which the input text to be detected belongs, and the intent type includes the known intent type and the unknown intent type.

8. The method according to claim 7, characterized in that, The classifier includes a first classifier and a second classifier; During the training process of the unknown intent detection model, the first classifier and the second classifier are trained using a co-training method. The unknown intent detection model trained is used to determine the unknown intent detection result by combining the classification results of the first classifier and the second classifier.

9. A method for detecting unknown intent, characterized in that, The method includes: Obtain the text to be detected; The text to be detected is input into the unknown intent detection model, and the detection result of the unknown intent detection model on the text to be detected is obtained. The detection result includes whether it is an unknown intent type. The unknown intent detection model is pre-trained using the method described in any one of claims 1 to 8.

10. An intelligent dialogue method, applied to an intelligent dialogue system, characterized in that, The method includes: Obtain text input by the user or text obtained by recognizing the user's speech; The acquired text is input into the unknown intent detection model as the text to be detected, and the detection result of the unknown intent detection model on the text to be detected is obtained. The unknown intent detection model is pre-trained using the method described in any one of claims 1 to 8. If the detection result of the text to be detected is of unknown intent type, then a response text for the user is generated using a preset script, or a human service is triggered to access the intelligent dialogue system to respond to the user.

11. An apparatus for training an unknown intent detection model, characterized in that, The device includes: The feature acquisition unit is configured to acquire feature representations of multiple unknown intent samples and feature representations of multiple known intent samples, wherein the unknown intent samples are text to be labeled with intent type tags, and the known intent samples are text already labeled with the intent type tags; The graph construction unit is configured to construct a fully connected graph using the feature representations of the plurality of known intent samples and the feature representations of the plurality of unknown intent samples, wherein the feature representations of the known intent samples and the feature representations of the unknown intent samples are respectively used as nodes in the fully connected graph, and the weight of the edge between the nodes is determined based on the similarity between the nodes at both ends of the edge. The label determination unit is configured to determine a soft label for each unknown intent sample using the fully connected graph, wherein the soft label is a probability value that the unknown intent sample belongs to an unknown intent; The first training unit is configured to train a classifier using the soft labels of each unknown intent sample to obtain an unknown intent detection model, wherein the unknown intent detection model is used to detect whether the input text to be detected belongs to the unknown intent type.

12. An unknown intent detection device, characterized in that, The device includes: The text acquisition unit is configured to acquire the text to be detected. An intent detection unit is configured to input the text to be detected into an unknown intent detection model and obtain the detection result of the unknown intent detection model on the text to be detected, wherein the detection result includes whether it is an unknown intent type; The unknown intent detection model is pre-trained using the apparatus described in claim 11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 10.

14. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 10.

Images