A new intent discovery method

Abstract

The present application relates to the field of artificial intelligence and dialogue system intent understanding, and proposes a new intent discovery detection method, medium, equipment and product, the method comprises: constructing a semantic neighbor pair of a query based on a semantic retrieval model, and optionally using a large language model to perform consistency discrimination on the candidate neighbor to refine the neighbor pair, and aligning the target query with its neighbor in the representation space through neighbor pair alignment target constraint; using a large language model to generate a descriptive variant and a prototype variant for each query, and comparing the original query with its variants in the representation space through a contrastive target constraint; periodically performing ambiguity detection and correction during the training process, and determining a calibration sample for ambiguous queries near the clustering boundary through the cooperation of a large language model and a semantic retrieval model, and further narrowing the representation of the ambiguous query and the calibration sample through an ambiguity correction loss. By uniformly optimizing the neighbor alignment loss, the variant alignment loss and the ambiguity correction loss, the present application forms a clearer global clustering structure and improves the clustering performance in the unsupervised and semi-supervised new intent discovery task.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing, dialogue systems, intent discovery and representation learning, and in particular to a novel intent discovery method, apparatus, computer-readable storage medium, and computer program product. Background Technology

[0002] Current intent detection is typically performed in supervised learning scenarios with closed sets, relying on predefined intent categories and labeled samples. However, in real-world applications, user queries are dynamic, with new intents constantly emerging. This results in only a small amount of known intent-labeled data during the training phase, while a large number of unlabeled queries containing unknown intents exist. New Intent Discovery (NID) aims to identify latent intent clusters from a set of queries and learn an intent representation space where queries with the same latent intent cluster together with clear boundaries. Reliably forming a global clustering structure remains challenging in the absence of explicit supervision. Summary of the Invention

[0003] The purpose of this invention is to provide a novel intent discovery scheme that moves from local relationships to global structure, by directly training the intent encoder through multiple complementary local supervision signals. This allows local constraints to continue acting on the training process, gradually forming a global clustering structure with both intra-cluster compactness and inter-cluster separability. The steps include:

[0004] S1. Constructing local relations from semantic neighbors: using a semantic retrieval model For each query Search for the most semantically similar query Construct a set of local neighbor pairs ,in The total number of queries in the training dataset;

[0005] S2. Construct local relations from query variants: for each query Generate descriptive variants With prototype variant And through the consistency between the comparative target constraint query representation and its variant representation;

[0006] S3. Perform global structure disambiguation: Disaggregate the set of ambiguous queries detected during training. Perform corrections for each ambiguous query. Select calibration positive samples Construct a calibration cell And with ambiguity correction loss constraint and The word indicates proximity;

[0007] S4. Adjust the neighbor alignment loss from step S1. Variant alignment loss in step S2 Disambiguity correction loss in step S3 Composition of total loss To train the intent encoder .

[0008] The present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the novel intent discovery method described above.

[0009] The present invention also proposes an electronic device including a processor and a memory, wherein the processor is interconnected with the memory, the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to execute the novel intent discovery method described above.

[0010] The present invention also proposes a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the novel intent discovery method described above.

[0011] The beneficial effects of the technical solution provided by this invention are:

[0012] 1. Forming a clearer global clustering structure: By continuously introducing local relation constraints during training, the intent representation space gradually evolves into a global structure that simultaneously possesses "intra-cluster compactness" and "inter-cluster separability", thereby improving the separability and stability of new intent clusters.

[0013] 2. Local supervision signals are easier to obtain and more stable: Compared with directly inferring the global structure or relying on noisy bootstrap pseudo-labels, this invention uses local relationships such as semantic neighbor relationships and query intravariant consistency as supervision signals. Semantic consistency is easier to reliably judge, and the supervision granularity is finer and the noise is lower.

[0014] 3. Integrating complementary local relationships to improve representation learning quality: Simultaneously utilizing cross-query semantic neighbor alignment and intra-query variant alignment, it not only enhances the aggregation between samples with the same intent, but also alleviates the problem of insufficient semantic expression caused by the original query being too short, verbose, or noisy, making it easier for the encoder to extract intent-level semantic features.

[0015] 4. Adaptive disambiguation capability for boundary samples: The disambiguation detection and correction mechanism identifies ambiguous queries near the cluster boundary and provides calibration positive samples based on the collaboration of a large language model and a semantic retrieval model. This can directionally correct unreliable local relationships, reduce boundary uncertainty, and further improve the clustering effect.

[0016] 5. Applicable to unsupervised and semi-supervised scenarios: when the set of intentions is known. It can work under unsupervised settings; when there are a small number of labeled samples with known intent, cross-entropy supervision and local relation supervision can be introduced to complement each other, so as to maintain good performance under low supervision ratio conditions.

[0017] 6. Achieve better clustering performance and robustness: On multiple datasets, as components are gradually introduced (neighbor alignment, variant alignment, ambiguity correction), clustering metrics continue to improve, demonstrating that the overall framework has strong robustness to queries with different data scales, different numbers of intents, and different expression styles.

[0018] 7. Controllable computational overhead and pluggable modules: Neighbor refinement and ambiguity correction can be enabled as optional modules according to budget, and ambiguity correction is only triggered on a small number of ambiguous samples, thereby improving the performance while controlling the cost of calling large language models, and has good engineering feasibility. Attached Figure Description

[0019] Figure 1 This is a flowchart of a novel intent discovery method according to an embodiment of the present invention.

[0020] Figure 2 This is a block diagram of an electronic device according to an exemplary embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0022] The flowchart of the novel intent discovery method in this embodiment of the invention is as follows: Figure 1 Specifically, it includes the following steps:

[0023] S1. Constructing local relations from semantic neighbors: using a semantic retrieval model For each query Search for the most semantically similar query Construct a set of local neighbor pairs ,in The total number of queries in the training dataset;

[0024] Specifically:

[0025] S11. Using semantic retrieval models (e.g., all-mpnet-base-v2) to construct neighbor pairs. Specifically, for the training set... Each query in ,use Encode its representation, and in Search for the most semantically similar query in the search engine:

[0026]

[0027] sim represents cosine similarity, and the resulting set of local neighbor pairs is:

[0028]

[0029] This set remains unchanged during training, thus providing stable local neighbor supervision.

[0030] S12. To further refine neighbor pairs, an optional neighbor refinement module based on a large language model is introduced. This module is related to... Collaboration provides complementary semantic judgments.

[0031] For each query Firstly, based on the retrieval model Search its The three semantically most similar candidates are used to form a candidate set. And design a prompt word to associate these candidates with... Use them together. Then, the large language model compares the meanings within the candidate set. Figure 1 To be compatible, and to choose with The most consistent query. Only retain the refined neighbor pairs. In this process, it is used as a more reliable supervisory signal for encoder training.

[0032] S13, By querying each query Its semantic neighbors Pairing is used to construct local neighbor supervision. Let and Let each of them represent a different value, and define the similarity of neighbor pairs using cosine similarity.

[0033]

[0034] For all For each query, a softmax normalization is applied to their similarity scores to obtain a calibrated weight distribution, and the cross-query neighbor alignment loss is defined as...

[0035]

[0036] exp is the natural exponential function, and τ is the temperature coefficient.

[0037] By continuously increasing the similarity between each query and its neighbors. Encouraging semantically similar neighbor queries to be pulled closer in the intent representation space enhances the intra-cluster compactness of semantic neighbors with the same intent and ultimately promotes the formation of a clear global structure.

[0038] S2. Construct local relations from query variants: for each query Generate descriptive variants With prototype variant And through the consistency between the comparative target constraint query representation and its variant representation;

[0039] S21. In order to further capture the meaning within a query Figure 1 Consistency is also used as a local relation for encoder training. To this end, two cue words were designed to allow the large language model to perform each query... Generate two complementary variants: one descriptive variant ( ), which is for querying Generate a semantically complete description; and a prototype variant ( The two variants summarize the query into a compact, prototype-level label. They reflect a query from descriptive and abstract perspectives: the descriptive variant preserves fine-grained semantic cues, while the prototype variant provides a highly condensed conceptual expression. Although the two variants differ in expression, their semantic core must remain consistent with the original query. This can be leveraged to train an intent encoder to encode them into the same cluster within the intent representation space.

[0040] S22. Use contrastive learning to enforce consistency between the original query representation and its variant representations in the intent representation space. Calculate for all queries... With the Each variant represents The cosine similarity between them. Given For each query, the in-query variant alignment loss is defined as:

[0041] exp is the natural exponential function, τ is the temperature coefficient, and z is the query term.

[0042] This loss helps the intent encoder by treating variants as alternative semantic perspectives of the same query. To better capture the intent-level semantics of queries, while maintaining the training set size. Instead of increasing the amount of data.

[0043] S3. Perform global structure disambiguation: Disaggregate the set of ambiguous queries detected during training. Perform corrections for each ambiguous query. Select calibration positive samples Construct a calibration cell And with ambiguity correction loss constraint and The word indicates proximity;

[0044] S31. Under the application of supervisory signals from local relations (including intra-query variant consistency and cross-query semantic neighborhood relations), the intent encoder... We learn a representation space where most queries exhibit stable clustering results. However, a small number of queries may still maintain unstable clustering results and tend to be located near cluster boundaries. To explicitly handle this situation, an Ambiguity Detection and Correction (ADC) mechanism is proposed to identify and correct such ambiguous queries. The ADC module starts from the first... The system is activated at the start of the training epoch, and thereafter every [number] training epochs. Each epoch is re-executed.

[0045] S32, Order Indicates query The current representation, and These represent the representations of its two variants, respectively. In Perform KMeans clustering to obtain the cluster centers. and the clustering assignment for each query. For each variant representation $\mathbf{z}_i^{v_j}$, its nearest cluster center is identified as follows: If a variant exists satisfy ,Will It is considered an ambiguous query because its clustering results are inconsistent with those of its variants.

[0046] All such ambiguous queries are collected into an ambiguity set.

[0047]

[0048] S33. For each ambiguous query ,calculate With each cluster center Cosine similarity between them: make Indicates that the cluster centers have Top- The highest A set of clusters of scores. For each candidate cluster... Using retrieval models From the training set Retrieve a small subset of cluster centers The representative query is denoted as Note each Already assigned to this cluster Construct a prompt word and use ambiguous queries. Candidate cluster set and their representative queries Given the prompt word, the large language model evaluates... Semantic consistency with candidate clusters, and output the best matching cluster. Subsequently, a retrieval model was used. To identify calibration positive sample queries, the method is to start from (i.e., with the selected cluster) Select from related representative queries) The most similar one (i.e., the one with the highest) sim represents cosine similarity. This indicates that such retrieved queries are... Positive samples. Maintain a calibration pool. This is used to store ambiguous queries and their corresponding positive sample pairs. Note In each of the first One epoch was updated.

[0049] S34. For each pair ,calculate and ,as well as In addition, calculation The similarity between positive samples associated with other ambiguous queries is denoted as . Given For each ambiguous query, the ambiguity correction loss is defined as:

[0050]

[0051] exp is the natural exponential function, τ is the temperature coefficient, and s is the query term.

[0052] S4. Adjust the neighbor alignment loss from step S1. Variant alignment loss in step S2 Disambiguity correction loss in step S3 Composition of total loss To train the intent encoder .

[0053] S41. The final training objective consists of three core losses: cross-query neighbor alignment loss. Query intravariant alignment loss and ambiguity correction loss .

[0054] S42. Under a completely unsupervised setup, the training objective is:

[0055]

[0056] S43. In a semi-supervised setting, when a small subset of labeled known intent queries and their known intent labels are available, a standard cross-entropy loss is further introduced to supervise the classification of known intents. Specifically, for labeled queries... and Cross-entropy loss is defined as:

[0057]

[0058] in This represents the predicted probability of a known intent label $y_i$.

[0059] S44. Set all loss weights to $1.0$, without performing any additional parameter tuning.

[0060] In one exemplary embodiment, a computer-readable storage medium is included, which stores a computer program that, when executed by a processor, implements the novel intent discovery method described above.

[0061] Please see Figure 2 In one exemplary embodiment, the device further includes an electronic device including at least one processor, at least one memory, and at least one communication bus.

[0062] The memory stores a computer program, which includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through the communication bus to execute the new intent discovery method described above.

[0063] To verify the effectiveness of the method of the present invention, experiments were conducted on three widely used new intent discovery benchmark datasets: BANKING77, CLIC150, and stackoverflow20. The experiments were set with known intents accounting for 0% and 25%, respectively, corresponding to unsupervised scenarios (all intents are unknown) and semi-supervised scenarios (25% of intents are known intents). The complete original test set was used for evaluation.

[0064] Comparison methods include: BERT (BERT: Deep Bidirectional Transformer Pre-training for Language Understanding), BERT-CE (BERT-CE: Deep Bidirectional Transformer Pre-training for Language Understanding), Retriever / Retriever (Sentence-BERT: Sentence embedding using Siamese BERT network; MPNet: Masking and permutation pre-training for Language Understanding), DeepCluster (Unsupervised deep embedding for clustering analysis), DeepAligned (Novel intent discovery based on deep alignment clustering), MTP-CLNN (Novel intent discovery based on pre-training and contrastive learning), IDAS (IDAS: Intent discovery based on generative summarization), IntentGPT (IntentGPT: Few-shot intent discovery based on large language models), LLMClust (Large language model-enabled few-shot clustering), CsePL (ClusterPrompt: Clustering semantic enhancement cue learning for novel intent discovery), LANID (LANID: Novel intent discovery assisted by large language models), and USNID (A clustering framework for unsupervised and semi-supervised novel intent discovery). ), INTENDD (IntenDD: A unified contrastive learning method for intent detection and discovery), GeoID (Geometric perceptual representation learning for novel intent discovery), SynCID (Collaborative large language model and pre-trained small model for dialogue intent discovery), ALUP (Active learning from large language models based on uncertainty propagation for generalized category discovery), SPILL (SpILL: Domain-adaptive intent clustering based on large language model selection and pooling), FCSLM (Research on collaborative small and large models for few-shot intent detection), CoCoID (CoCoID: Contrastive representation and compact clustering learning for semi-supervised intent discovery), NILC (NILC: Novel intent discovery based on large language model-assisted clustering), WATCH (Focus on neighbors: A unified K-nearest neighbor contrastive learning framework for out-of-distribution (OOD) intent discovery).

[0065] Evaluation metrics: This invention uses clustering accuracy (Accuracy). ), Adjusted Rand Index ) and Normalized Mutual Information (MMI) ) is used as an evaluation indicator. Among them, Used to measure the overall consistency between predicted clustering results and true intent labels under the best one-to-one matching alignment; It is used to measure the consistency between the predicted and actual partitions in terms of pairwise homo-cluster / hetero-cluster relationships of samples, and to correct for random consistency. This is used to measure and normalize the mutual information between predicted clusters and true labels to eliminate the impact of differences in cluster size, thereby reflecting the overall quality of the cluster structure.

[0066] This invention evaluates the novel intent discovery method on three widely used intent discovery benchmarks. Table 1 shows that the model of this invention consistently outperforms all competing baseline methods across all settings. Results for all comparative methods are derived from those reported in their respective papers.

[0067] 1. Unsupervised setting: In the most challenging scenario without any known intent labels, the framework of this invention consistently achieves state-of-the-art performance on all datasets, where higher ACC indicates that most queries are correctly assigned to their intent categories, while improvements in ARI and NMI reflect a more compact intra-cluster organization and clearer inter-cluster boundaries in the learned representation space.

[0068] Compared to the Retriever baseline (which provides the semantic retrieval signals used in the framework of this invention), the method of this invention shows that training... Integrating such signals with other local relational cues can reorganize the space of intent representation and ultimately transcend the Retriever itself.

[0069] 2. Semi-supervised setup

[0070] The method of this invention achieves further improvements and continues to outperform all comparative methods when there are only a few labeled intentions, demonstrating that limited known intention supervision can effectively complement local relational signals, thereby shaping the representation space.

[0071] Table 1

[0072]

[0073] In one exemplary embodiment, a computer program product is proposed, including a computer program / instructions that, when executed by a processor, implement the steps of the novel intent discovery method described above.

[0074] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for discovering novel intents, characterized in that, Training the intent encoder based on the training dataset Among them, the new intent discovery task aims to discover the complete intent space. , For a set of known intentions, The training data consists of a set of unknown intents that are mutually exclusive with the known intent set; the training data includes a labeled set of known intent queries. and unlabeled query sets Covering both known and unknown intents; denoted as the entire training set as It contains a total of Query; based on the training set The goal is to train the intention encoder. Among them, when When the task degenerates into a completely unsupervised setting, the method includes: S1. Constructing local relations from semantic neighbors: using a semantic retrieval model For each query Search for the most semantically similar query Construct a set of local neighbor pairs ,in The total number of queries in the training dataset; S2. Construct local relations from query variants: for each query Generate descriptive variants With prototype variant And through the consistency between the comparative target constraint query representation and its variant representation; S3. Perform global structure disambiguation: Disaggregate the set of ambiguous queries detected during training. Perform corrections for each ambiguous query. Select calibration positive samples Construct a calibration cell And with ambiguity correction loss constraint and The word indicates proximity; S4. Adjust the neighbor alignment loss from step S1. Variant alignment loss in step S2 Disambiguity correction loss in step S3 Composition of total loss To train the intent encoder .

2. The method for discovering a new intent according to claim 1, characterized in that, In step S1, for each query The semantically most similar query is determined in the following way: .

3. A method for discovering a new intent according to claim 1 or 2, characterized in that, Step S1 also includes optional neighbor refinement based on a large language model: for each query First, use semantic retrieval models Search its The most similar candidates form a candidate set. Then, the large language model compares intent consistency in the candidate set and selects the one that matches the intent. The most consistent query is the refined .

4. The method for discovering a new intent according to claim 1, characterized in that, The neighbor alignment loss in step S1 is defined as: Let , and order The loss function is: .

5. The method for discovering a new intent according to claim 1, characterized in that, Descriptive variants in step S2 For "for query" A variant of "generating a semantically complete description", a prototype variant For "query" This can be summarized as a variant of the "compact prototype-level tag".

6. The method for discovering a new intent according to claim 1, characterized in that, The variant alignment loss in step S2 is defined as follows: Given N queries, let ,make ,in ,but: ; The ambiguity detection in step S3 includes: KMeans clustering is used to obtain cluster centers. Cluster allocation for each query ; for each variant, indicate Calculate its nearest cluster center: If it exists Make Then Added as an ambiguous query: .

7. A method for discovering a new intent according to claim 1 or 7, characterized in that, The ambiguity correction in step S3 includes: correcting ambiguous queries. Calculate its representation With each cluster center similarity Take the one with the highest similarity Each cluster forms a candidate cluster set. For each candidate cluster Using semantic retrieval models From the training set Searching for nearby The representative query set And based on the large language model , and Output the best matching cluster ; to obtain the best matching cluster Afterwards, from Select with In retrieval model The query with the highest similarity is used as the calibration positive sample. And construct a calibration pool The ambiguity correction loss in step S3 is defined as: for each pair ,make , $, let and order Given For each ambiguous query: Under unsupervised settings, the total loss is: In the semi-supervised setting, cross-entropy loss is also applied to labeled queries. And incorporate it into the training objective; global structure disambiguation from the first This feature is enabled at the start of the training epoch and then every subsequent training epoch. Each epoch is re-executed.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a method for novel intent discovery as described in any one of claims 1-7.

9. An electronic device, characterized in that, The device includes a processor and a memory interconnected thereto, wherein the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to perform a novel intent discovery method as described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method for discovering a novel intent as described in any one of claims 1-7.

Images