User intention mining method, system and program based on double-tower model and storage medium

Abstract

The invention provides a user intention mining method and system based on a double-tower model, a program and a storage medium. The system comprises a data acquisition device, a text graph enhanced representation device, a model training device and an intention mining device. The method comprises the following steps: firstly, mining and converting a user search intention into a dense retrieval problem, and realizing formalized definition of the user search intention and a dialogue text through a Sension-BERT model; then constructing a retrieval architecture based on a double-tower model, performing feature enhancement in combination with a text graph neural network, and capturing text deep association through a semantic graph, a syntactic graph and a sequence graph; meanwhile, an interval constraint negative sample sampling strategy is put forward, effective negative samples are mined from front and back, and the model is trained through a noise comparison estimation method. The method can be deployed at the rear end of each server room, and can widely provide efficient user intention mining support for decision-making systems such as an intelligent medical auxiliary system and the like.

Description

Technical Field

[0001] This invention relates to the field of intent mining technology, and in particular to a user intent mining method, system, program, and storage medium based on a dual-tower model. Background Technology

[0002] In routine medical consultations, patients often spend a significant amount of time waiting for face-to-face interaction with doctors, but the actual consultation time is relatively limited. Therefore, one of the key factors in improving consultation efficiency is ensuring effective communication between doctors and patients. However, patients often lack a medical background and may use informal language rather than medical terminology to describe their health conditions. Doctors often need to engage in multiple rounds of dialogue with patients to accurately understand their health status. Therefore, intelligent medical assistance systems employ artificial intelligence technologies such as natural language processing and medical knowledge graphs to detect medical intent from patients' dialogue requests. This means that the system can automatically collect and process patient information before the patient's visit, thereby automatically generating a preliminary draft of the medical record [Zhang S, Sun J, Huang Y, et al. Medical Symptom Detection in Intelligent Pre-Consultation Using Bi-directional Hard-Negative NoiseContrastive Estimation[C]. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022: 4551-4559.].

[0003] However, identifying intent from patient conversations still faces many challenges, including: First, the conversations are often colloquial. Since most patients lack a medical background, they typically use everyday language to express their intent. Second, when describing intent, patients tend to narrate their feelings and observations rather than directly using medically specific names for the intent. Mapping patient conversations to one or more intents requires overcoming the problem of lexical mismatch. Current research on lexical matching for intent detection often employs biomedical entity normalization methods, but using only named entities to map intent may miss a large amount of intent information described by words with the same semantics but different forms. Some studies have addressed these issues by transforming intent detection into a retrieval problem. As a retrieval problem, it eliminates the need to identify intent, directly mapping patient conversations to one or more intents. Traditional retrieval models rely on text matching, typically using TF-IDF or BM25 models to effectively match conversation keywords with inverted indexes in a document set to find the most relevant documents. This can be considered a high-dimensional sparse vector space representing the conversation and document set, but it is difficult to retrieve synonyms. In recent years, dense retrieval models have overcome the information omission problem by matching queries and documents in a continuous dense embedding space [Karpukhin V, Oguz B, Min S, et al. Dense Passage Retrieval for Open-Domain Question Answering[C]. Proceedings of the 2020 Conference on EmpiricalMethods in Natural Language Processing (EMNLP), 2020: 6769-6781.], and use dot product or cosine similarity to calculate relevance or similarity scores. Various methods have been proposed in recent years for training dense retrieval models, but they are often relatively inefficient and not well-suited for practical use.For example, knowledge extraction [Lin SC, Yang JH, Lin J J. Distilling Dense Representations for Ranking using Tightly-Coupled Teachers[J]. ArXiv, 2020, abs / 2010.11386.] and periodic index refreshing [Guu K, Lee K, Tung Z, et al. Retrieval Augmented Language Model Pre-Training[C]. Proceedings of the 37th International Conference on Machine Learning, 2020: 3929--3938.] are used, but computationally expensive methods are employed during training. Currently popular and effective dense retrieval models are built using Noise Contrastive Estimation (NCE) [Denize J, Rabarisoa J, Orcesi A, et al. Similarity contrastive estimation for self-supervised softcontrastive learning[C]. Proceedings of the IEEE / CVF Winter Conference on Applications of Computer Vision, 2023: 2706-2716.]. One challenge in using NCE to learn dense retrieval models is finding informative negative samples to learn dense representations. In unsupervised contrastive learning methods, negative samples are constructed by modifying the input. Most dense retrieval models use in-batch negative samples and employ NCE (Negative Contrast Equation) as the method for calculating loss. Using in-batch negative samples avoids the process of constructing negative samples, but this often leads to slow convergence during training [Xiong L, Xiong C, Li Y, et al. Approximate nearest neighbor negative contrastive learning for dense text retrieval[J]. arXiv preprintarXiv:2007.00808, 2020.]. To overcome this problem, some studies have used hard negative sample mining strategies, which solve the slow convergence issue, but they do not consider the deep-seated relationship between dialogues and candidate sets.

[0004] The user's possible behaviors and potential interaction item categories are considered as the user's intent. The study investigates how to fuse candidate item lists generated with different goals while perceiving user intent. A common approach to user intent discovery is through biomedical entity normalization techniques. Biomedical entity normalization is a long-standing and important task in the field of biomedical NLP. The goal of biomedical normalization is to map terms in documents to unique concept IDs in a biomedical ontology. Various challenges have been organized to address the normalization problem, and many researchers have participated in evaluating these NLP methods. Ji et al. [JiZ, Wei Q, Xu H. Bert-based ranking for biomedical entity normalization[J]. AMIA Summits on Translational Science Proceedings, 2020, 2020: 269.] used the ShARe / CLEF corpus to organize and conduct experiments aimed at identifying diseases mentioned in clinical texts and converting these disease names into unique concept identifiers in the SNOMED-CT medical vocabulary to achieve normalization. The most common traditional normalization methods are dictionary-based and rule-based methods, which use dictionary-based pattern matching and heuristic matching rules, respectively. Apache Lucene [Liu R, Liang J, Jin P, et al. MMH-index: Enhancing Apache Lucene with High-Performance Multi-Modal Indexing and Searching[C]. Proceedings of the 30th ACM International Conference on Multimedia, 2022:7279-7289.] is a Java-based text indexing and search engine library that calculates the similarity between documents and queries. While these methods can be easily applied to a wide range of fields such as disease, gene, and chemical nomenclature normalization tasks [IslamajR, Leaman R, Kim S, et al. NLM-Chem, a new resource for chemical entity recognition in PubMed full text literature[J]. Scientific data, 2021, 8(1):91.], they are generally inefficient and inaccurate for words not found in dictionaries or ungrammatical texts with misspellings. Biomedical entity normalization is typically divided into two steps: named entity recognition and entity normalization.It requires a model to first identify named medical entities in the text and then map them to unique concept IDs in a biomedical ontology. Traditional entity normalization methods rely heavily on pattern matching. For example, Ghiasvand et al. [Ghiasvand O, Kate R.UWM: Disorder Mention Extraction from Clinical Text Using CRFs and Normalization Using Learned Edit Distance Patterns[C]. Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), 2014: 828-832.] proposed a method that first generates variants of a given unordered word or phrase based on a set of learned edit distance patterns to transform one word or phrase into another, and then attempts to normalize these dialogue phrase variants by precisely matching them with trained unordered phrases or concept terms. However, dialogues under intent detection often use spoken language and describe intent by interpreting the patient's feelings and perceptions. Mapping intent using only identified named entities may miss a significant amount of intent information. While existing tools rely on the accuracy of normalized dictionaries or domain-specific rules, some studies have applied machine learning and deep learning techniques to overcome previous limitations. Han et al. [Han H, HwangS-W, Song YI, et al. Training data optimization for pairwise learning torank[C]. Proceedings of the 2020 ACM SIGIR on international conference ontheory of information retrieval, 2020: 13-20.] proposed a pairwise learning ranking method to measure the similarity between entity mentions and candidate concepts.Chen et al. [Chen L, Fu W, Gu Y, et al. Clinicalconcept normalization with a hybrid natural language processing system combining multilevel matching and machine learning ranking[J]. Journal of the American Medical Informatics Association, 2020, 27(10): 1576-1584.] proposed a machine learning-based system to develop a hybrid clinical NLP system that combines a general multilevel matching framework, customizable matching components, and a machine learning ranking system. Liang et al. [J. Liang, L. Liao, H. Fei, J. Jiang, Synergizing large language models and pre-trained smaller models for conversational intent discovery, in: Findings of the Association for Computational Linguistics ACL 2024, pp. 14133–14147.] proposed SynCID, which leverages the semantic understanding capabilities of large language models (LLMs) and the computational flexibility of small language models (SLMs) to discover and label new unknown intents as well as known intents in unlabeled data. This method overcomes the overfitting problem to known intents and can assign labels to new intents in open-world scenarios.Zhang et al. [S. Zhang, C. Yan, J. Yang, C. Ren, J. Bai, T. Li, Z. Li, Ronid: New intent discovery with generated-reliable labels and cluster-friendly representations, in: Database Systems for Advanced Applications - 29th International Conference, DASFAA 2024, Gifu, Japan, July 2-5, 2024, Proceedings, Part V, Vol. 14854 of Lecture Notes in Computer Science, Springer, 2024, pp. 104–119.] proposed the RoNID model, which can generate reliable pseudo-labels and easily clustered representations, discovering and clustering new unknown intents using limited labeled data. Wei et al. [X. Wei, X. Wang, N. Zhuang, C. Wang, L. Wang, J. Dang, Integration of old and new knowledge for generalized intent discovery: Aconsistency-driven prototype-prompting framework, in: Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2025, Montreal, Canada, August 16-22, 2025, ijcai.org,2025, pp. 8286–8294.] proposed a consistency-driven prototype prompting framework, CPP, for generalized intent discovery, using a large language model as a knowledge source.

[0005] In summary, traditional normalization methods are often inefficient and inaccurate for words not found in dictionaries or for ungrammatical texts containing typos, potentially missing a significant amount of intent information. Recent research has utilized machine learning and deep learning to measure the similarity between entities and candidate concepts.

[0006] Dense vector representations have been used for retrieval since latent semantic analysis. In recent years, dense encoders based on semantic matching [Lin SC, Lin J. A dense representation framework for lexical and semantic matching[J]. ACM Transactions on Information Systems, 2023, 41(4): 1-29.] have become a research hotspot. They use labeled dialogue and document pairs and are often applied to cross-language document retrieval, advertising relevance prediction, web search, and entity retrieval. Dense vector retrieval methods complement sparse vector methods because they can potentially assign high similarity scores to semantically related text pairs even without precise label matching. Although dense representations alone are generally not as good as sparse representations, dense representations from pre-trained models, along with cross-attention mechanisms, have proven effective in paragraph or dialogue ranking tasks [Wang L, Yang N, Huang X, et al. Simlm: Pre-training with representation bottleneck for dense passage retrieval[J]. arXiv preprintarXiv:2207.02578, 2022.]. Khattab et al. [Khattab O, Zaharia M. ColBERT: Efficient and Effective Passage Search via Contextualized LateInteraction over BERT[C]. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020: 39–48.] demonstrated the feasibility of dense retrieval in retrieval tasks. Instead of employing a dual-encoder framework, they introduced a late-interaction overlay on top of the BERT encoder. Dense retrieval is a relatively new approach that uses deep learning models to understand and compare the semantic content of dialogues and documents. This approach typically involves embedding text into a high-dimensional space, where the similarity between dialogues and documents is evaluated by calculating the distance between their vectors. Dense retrieval does not rely on traditional keyword matching but improves retrieval accuracy by capturing deep semantic connections within the text.Mao et al. [Mao Y, He P, Liu X, et al. RIDER: Reader-Guided Passage Reranking for Open-Domain Question Answering[J]. arXiv preprint arXiv:2101.00294, 2021.] explored dense retrieval in open-domain QA. They proposed using a dense retrieval model to obtain a vector of the question statement and retrieve relevant paragraphs. Paragraph retrieval has always been an important part of open-domain QA. It not only effectively reduces the search space for answer extraction but also identifies the context, allowing users to identify answers using paragraph retrieval. Strongly sparse vector space models such as TF-IDF or BM25 have been used as standard methods widely applied to various QA tasks [Sachan DS, Patwary M, Shoeybi M, et al. End-to-end training of neural retrievers for open-domain questionanswering[J]. arXiv preprint arXiv:2101.00408, 2021.]. Zhang et al. [Zhang S, Sun J, Huang Y, et al. Medical Symptom Detection in Intelligent Pre-Consultation Using Bi-directional Hard-Negative Noise Contrastive Estimation[C]. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and DataMining, 2022: 4551-4559.] encoded candidate answer phrases as vectors and retrieved answers to input questions. Singh et al.

[33] pre-trained a question encoder and decoder to match alternatives to questions and relevant paragraphs. Their method outperformed BM25 in QA accuracy on multiple open-domain QA datasets.Guu et al. [Guu K, Lee K, Tung Z, et al. Retrieval Augmented Language Model Pre-Training[C]. Proceedings of the 37th International Conference on Machine Learning, 2020:3929-3938.] asynchronously tuned the paragraph encoder by reindexing paragraphs during training. In summary, dense retrieval models have significant potential applications in natural language processing. Summary of the Invention

[0007] The purpose of this invention is to provide a user intent mining method, system, program, and storage medium based on a dual-tower model, the core of which includes the following technical solutions:

[0008] A user intent mining method based on a dual-tower model, the specific steps of which are as follows:

[0009] Step 1: Obtain raw network data from the network data source, set interval thresholds and hard negative sample thresholds; process and clean the raw network data to obtain user dialogue text and intent document text.

[0010] Step 2: Construct a dual-encoder retrieval model that includes encoding user dialogue text and encoding intent document text, and output a fixed-dimensional set of dialogue embedding vectors and intent embedding vectors.

[0011] Step 3: Based on the user dialogue text, intent document text, and the embedding vectors in the fixed-dimensional dialogue embedding vector set and intent embedding vector set, construct a semantic graph, a syntactic graph, and a temporal graph. Figure 3 The fine-grained graph generates enhanced dialogue node embeddings and enhanced intent node embeddings through intra-graph and inter-graph propagation.

[0012] Step 4: Using the similarity score of positive sample pairs as anchor points, the enhanced dialogue node embedding representation and the enhanced intent node embedding representation are mined through interval thresholds to obtain hard negative samples.

[0013] Step 5: Using the noise contrast estimation method, the dual encoder retrieval model is trained by combining the hard negative samples. The model parameters are optimized by minimizing the forward loss and backward loss to obtain the trained dual tower model.

[0014] Step 6: Input the user dialogue to be mined into the trained dual-tower model, calculate the similarity with the candidate intents, sort the similarity and return the intent mining results.

[0015] The aforementioned user intent mining system based on a dual-tower model includes a data acquisition device, a model building device, a text graph enhancement representation device, an interval sampling device, a model training device, and an intent mining device.

[0016] The data acquisition device collects raw data from the network platform and processes it into logical relationships of dialogue intent, thus preparing for the subsequent construction of a user search intent mining system based on a dual-tower model.

[0017] The model building device constructs a dual-encoder retrieval model. Both encoders adopt the Sentence-BERT structure and share weights. The dialogue encoder takes user dialogue text as input and outputs dialogue embedding; the intent encoder takes intent description text as input and outputs intent embedding. The encoder output is transformed into a fixed-dimensional vector through pooling operations, providing a basis for subsequent similarity calculation.

[0018] The text graph enhancement representation device constructs semantic graphs, syntactic graphs, and temporal graphs. Figure 3 The fine-grained graph type integrates heterogeneous information from the three types of graphs through intra-graph propagation and inter-graph propagation to generate enhanced node embeddings.

[0019] The interval negative sample sampling device calculates a cosine similarity score for each positive sample pair, which serves as a sampling anchor point. The optimal threshold is used to define the sampling interval centered on the anchor point, ensuring that the selected negative samples have a high similarity to the positive samples, thus effectively improving the model's discrimination ability.

[0020] The model training device sets appropriate model parameters and combines forward loss and backward loss to comprehensively optimize the model's learning of the relationship between dialogue and intent matching.

[0021] The intent mining device receives relevant dialogue input from the user, automatically performs format standardization processing, inputs the standardized dialogue into the model to obtain a dialogue embedding vector, calculates the cosine similarity between this vector and all intent embedding vectors in the intent set, and outputs the intent based on the similarity score.

[0022] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0023] A computer program product includes computer instructions that, when executed by a processor, implement the steps of the method described above.

[0024] The beneficial effects of this invention are as follows: This invention can accurately identify colloquial intent descriptions, improve the performance of intent retrieval models, provide accurate references for decision-makers, reduce communication costs between roles, shorten communication time, and provide efficient intent mining support for scenarios such as intelligent pre-consultation systems. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a typical application environment of the present invention.

[0026] Figure 2 This is a structural block diagram of the system of the present invention.

[0027] Figure 3 This is a schematic diagram illustrating the user intent mining principle of the present invention.

[0028] Figure 4 This is a graph showing the running results of different epochs on the IPCD dataset in an embodiment of the present invention.

[0029] Figure 5 This is a graph showing the performance impact of the text graph neural network embedding enhancement method in this embodiment of the invention.

[0030] Figure 6 This is a diagram showing the running results of the feature enhancement strategy in an embodiment of the present invention.

[0031] Figure 7 This is a diagram showing the operation results of the propagation strategy in an embodiment of the present invention.

[0032] Figure 8 These are experimental results demonstrating the effectiveness of multi-granularity feature modeling under different combinations in the embodiments of the present invention. Detailed Implementation

[0033] The following is combined Figure 2 The present invention will be further described below.

[0034] A user intent mining method based on a dual-tower model, the specific steps of which are as follows:

[0035] Step 1: Obtain raw network data from the network data source, set interval thresholds and hard negative sample thresholds; process and clean the raw network data to obtain user dialogue text and intent document text.

[0036] Furthermore, step 1 specifically includes the following steps:

[0037] Step 1.1: Collect raw network data containing dialogue and intent from the data source.

[0038] Step 1.2: Process and clean the collected raw data to extract valid text information.

[0039] Step 1.3: Obtain the user dialogue text and intent document text.

[0040] Step 2: Construct a dual-encoder retrieval model that includes encoding user dialogue text and encoding intent document text, and output a fixed-dimensional set of dialogue embedding vectors and intent embedding vectors.

[0041] Furthermore, the dual encoder retrieval model described in step 2 is built based on Sentence-BERT (SBERT), adopts a Siamese network structure, and the two encoders share weights, both of which include the BERT infrastructure and pooling layer.

[0042] Specifically, a dual-tower Sentence-BERT (SBERT) retrieval model is used as the base model. Both towers of this model employ BERT encoders, outputting fixed-dimensional sentence embeddings to independently encode user queries and intents. The dual-encoder retrieval model aims to enhance sentence-level embedding representations by fine-tuning BERT on semantically labeled triple data. The SBERT processing flow in the dual-tower model includes two phases: training and prediction. During training, SBERT fine-tunes the BERT model using labeled sentence triples through supervised learning to learn better sentence embedding representations. During prediction, SBERT directly uses the fine-tuned BERT to generate sentence embeddings and performs semantic retrieval based on cosine similarity.

[0043] The dual-encoder retrieval framework proposed in this invention achieves deterministic retrieval by constructing a representation space primarily based on target domain data. This design essentially restricts the model's operation to an observed, reasonable pattern space, making it structurally resistant to generating unfounded outputs. Therefore, the framework design ensures terminological accuracy and reduces the risk of illusion, making it suitable for high-risk domains with high deterministic reliability requirements.

[0044] Step 3: Based on the user dialogue text, intent document text, and the embedding vectors in the fixed-dimensional dialogue embedding vector set and intent embedding vector set, construct a semantic graph, a syntactic graph, and a temporal graph. Figure 3 The fine-grained graph generates enhanced dialogue node embeddings and enhanced intent node embeddings through intra-graph and inter-graph propagation.

[0045] Furthermore, step 3 specifically includes the following steps:

[0046] Step 3.1: Calculate the edge weights between nodes in the semantic graph based on the cosine similarity of the embedded vectors.

[0047] Step 3.2: Use CoreNLP tools to extract syntactic dependencies between words, construct a syntactic graph in the form of undirected relations, and calculate edge weights.

[0048] Step 3.3: Calculate the edge weights of word co-occurrence in the time series graph using point mutual information and a sliding window strategy.

[0049] Step 3.4: Fuse the heterogeneous information of the semantic graph, syntactic graph and temporal graph through intra-graph propagation and inter-graph propagation to generate enhanced dialogue node embedding representations and enhanced intent node embedding representations.

[0050] Specifically, for each heterogeneous text graph (TG), TextGNN is used to construct three isomorphic graphs: a semantic graph, a syntactic graph, and a sequence graph. Feature extraction across these graphs relies on the graph topology itself to encode domain knowledge. Therefore, all nodes share the same feature space and the same weight matrix. The feature representation fusion order is as follows: first, intra-graph propagation is performed independently on each graph, followed by synchronous inter-graph propagation, and finally, pooling is performed along the graph dimensions at the output layer, achieving direct information fusion across heterogeneous views.

[0051] Specifically, by fusing Sentence Bidirectional Encoder Representation (SBERT) embeddings with graph neural networks, a text graph containing sentence nodes (including dialogue and intent) and lexical nodes is constructed, thereby optimizing the embedding vectors generated by SBERT to improve representation accuracy. By constructing a multi-granularity dialogue graph, the interdependencies between semantics, syntax, and temporal sequences are explicitly encoded. Subsequently, intra-graph and inter-graph propagation operations are performed on the text graph to aggregate node information and fuse features of identical nodes, ultimately obtaining the feature representations of the text nodes. This method, by comprehensively considering lexical associations and structural information, supports negative sample mining and noise contrast estimation, enabling the discovery of deep associations between positive samples and the candidate set, mitigating the missed detections or misjudgments caused by negative sample mining based solely on the similarity between the intent set and positive dialogue samples.

[0052] Specifically, the enhanced graph structure representation achieves the fusion of domain knowledge through a data-driven learning paradigm. This invention learns domain-specific representations directly from specialized datasets, rather than relying on manually constructed ontologies. Unlike methods that depend on explicit external ontologies, this invention is designed to learn domain-specific constraints and knowledge structures directly from specialized training data. The real-world annotations in the domain-specific datasets are not merely text corpora; they originate from real-world entity interactions and thus inherently reflect domain concepts and their interrelationships. By training a text graph neural network on this data, the model implicitly constructs a dynamic, data-driven soft ontology. Semantic graphs learn conceptual relationships and nuanced terminology from contextual embeddings, effectively normalizing synonyms and related terms without external resources. Syntactic graphs identify and prioritize dependency patterns that are most predictive of domain-specific intentions, thereby capturing pragmatic rules. Sequence graphs simulate the co-evolutionary relationships and temporal associations of core concepts that naturally emerge in real-world domain interactions.

[0053] Step 4: Using the similarity score of positive sample pairs as anchor points, the enhanced dialogue node embedding representation and the enhanced intent node embedding representation are mined through interval thresholds to obtain hard negative samples.

[0054] Furthermore, step 4 specifically includes:

[0055] For each positive sample pair ( It can extract M negative dialogue samples. and N intentional negative samples When mining hard negative samples, select positive sample pairs ( The similarity score between the two points is used as the anchor point anch.

[0056] Centered on the anchor point anch, define the sampling interval [anch-range, anch+range] based on the interval threshold range.

[0057] Traverse the intent embedding vector set and filter out non-target intents whose similarity scores with the user dialogue fall within the sampling interval, as hard negative samples of the intent corresponding to the dialogue.

[0058] Traverse the set of dialogue embedding vectors and filter out non-matching dialogues whose similarity scores with the target intent fall within the sampling interval, as hard negative samples of the dialogue corresponding to the intent.

[0059] If the number of hard negative samples is less than the hard negative sample threshold, then they are randomly supplemented from the fixed-dimensional embedding vectors that have been traversed.

[0060] The interval-constrained sampling strategy proposed in this invention possesses inherent topological awareness and domain adaptation characteristics. Embedding the topological structure, through a modeling process enhanced by TextGNN, defines a semantic space for concepts within a reasonable domain based on domain-specific data. By dynamically sampling negative samples within the constraints of this space, the model is trained to learn fine-grained distinctions between semantically similar and domain-related candidate concepts. This process essentially avoids cross-domain decision boundaries, constraining the learning space to the range presented by the domain data, enabling the model to adapt to high-precision recognition tasks and effectively reducing errors. This invention, by introducing an interval-constrained negative sample sampling method, can more comprehensively consider similarity scores, avoiding the limitation of only considering candidates with high similarity scores. Unlike other methods, it considers not only candidates with high similarity scores but also candidates with similarity scores greater than or less than the anchor point but close to it. After obtaining the interval hard negative sample data, this data can be used to retrain the model, update the model parameters, and thus improve the accuracy of intent mining.

[0061] Step 5: Using the noise contrast estimation method, the dual encoder retrieval model is trained by combining the hard negative samples. The model parameters are optimized by minimizing the forward loss and backward loss to obtain the trained dual tower model.

[0062] Furthermore, the forward loss function described in step 5 and inverse loss function The specific calculation methods include:

[0063]

[0064]

[0065]

[0066] in, The cosine similarity function; This represents the number of negative sample pairs. For hyperparameters; The number of hard negative samples. For dialogue embedding vectors, For the intended embedding vector, To embed vectors with positive intent, Negative intention embedding vector, Negative dialogue embedding vectors For exponential operations, This represents the total loss.

[0067] Specifically, the model is trained using bidirectional noise contrast estimation (NCE). Unlike traditional unidirectional NCE, which only focuses on bringing positive sample pairs closer and pushing negative intents further away from the given dialogue, bidirectional NCE considers negative samples from both dialogue encoding and intent encoding perspectives. This bidirectional training approach significantly improves the model's accuracy and robustness. Given a user dialogue... With intention set The intended target is from The subset of intents relevant to the dialogue is selected from the positive samples. Expectations for their dialogue With positive intention The distance is relatively close, while the distance represents a negative intention. The goal of forward noise contrastive estimation is to train a model that can accurately identify the true intent from all intents for a given dialogue; the goal of backward noise contrastive estimation is to train a model that can identify the correct dialogue from all dialogues for a given intent. This invention proposes a novel interval-constrained hard negative sample mining method that extracts negative sample results from indistinguishable incorrect predictions as interval hard negative samples. It selects to generate interval hard negative samples from the model at the beginning of each training cycle and adds them to the NCE (Network Objective-Cycle), ultimately obtaining the model's training representation.

[0068] Step 6: Input the user dialogue to be mined into the trained dual-tower model, calculate the similarity with the candidate intents, sort the similarity and return the intent mining results.

[0069] Furthermore, step 6 specifically includes the following steps:

[0070] Step 6.1: Input the user dialogue to be mined into the trained dual encoder retrieval model to obtain the dialogue embedding vector.

[0071] Step 6.2: Calculate the cosine similarity between the dialogue embedding vector and all intent embedding vectors in the intent embedding vector set.

[0072] Step 6.3: Sort the results in descending order of similarity scores to obtain the most likely intent as the user search intent mining results.

[0073] Example

[0074] The present invention will now be further described with reference to the accompanying drawings.

[0075] Figure 1 To illustrate the typical application environment for implementing this invention, specifically:

[0076] The typical application environment of this invention is a collaborative diagnosis and treatment platform, focusing on medical and consultation collaboration scenarios. End users such as ordinary residents and patients with chronic diseases can provide feedback through multiple channels, including apps, mini-programs, and self-service terminals, using spoken language or voice to provide dialogue text. The platform integrates patient dialogue and intent information. Through the dual-encoder model, text graph enhancement, and interval negative sample sampling technologies of this invention, the core intent of users is accurately mined. Ultimately, a closed-loop service is achieved, encompassing pre-diagnosis triage, precise doctor consultation, and rehabilitation follow-up. This not only solves the problem of matching patients' spoken descriptions with standardized medical services but also optimizes the allocation of medical resources, shortens consultation waiting times, improves the compliance rate of chronic disease management, and significantly enhances diagnosis and treatment efficiency and service accuracy.

[0077] Figure 2 The diagram shown is a structural block diagram of the system described in this invention, comprising:

[0078] Data acquisition and processing device: Collects raw data from the network platform and processes it into logical relationships of dialogue intent, preparing for the subsequent construction of a user intent mining system based on a dual-tower model.

[0079] The data acquisition and processing device collects raw medical-related data in batches from multiple online platforms, including medical forums, online consultation platforms, and medical sections of social platforms. This data includes user health consultation dialogues and disease diagnosis intentions. The device performs cleaning operations on the collected raw data, such as deduplication, removal of advertisements and meaningless characters, and correction of typos, to filter out the effective core information.

[0080] Model building apparatus: A dual-encoder retrieval model is built, with both encoders adopting the Sentence-BERT structure and sharing weights. The dialogue encoder takes user dialogue text as input and outputs dialogue embedding; the intent encoder takes intent description text as input and outputs intent embedding. The encoder output is transformed into a fixed-dimensional vector through pooling operations, providing a foundation for subsequent similarity calculation.

[0081] The model building mechanism uses Sentence-BERT as its basic architecture to construct a dual-encoder retrieval model. The two encoders share weights and both include the BERT infrastructure and pooling layers. This effectively captures the contextual semantics of colloquial dialogues and standardized intent descriptions in texts such as those in the medical field, adapting to specific language expression scenarios in these areas. Specifically, the user dialogue encoder receives colloquial consultation dialogue texts from users in fields such as healthcare. After deep semantic analysis by the BERT layer, the variable-length dialogue text is converted into fixed-dimensional dialogue embedding vectors through pooling operations. The intent encoder outputs intent embedding vectors with the same vector dimensions, ensuring that the two can be directly used for similarity calculation. This provides a unified vector calculation foundation for subsequent interval sampling, model training, and intent matching.

[0082] Text Graph Enhancement Representation Device: Constructing Semantic Graphs, Syntactic Graphs, and Temporal Graphs Figure 3 The fine-grained graph type integrates heterogeneous information from the three types of graphs through intra-graph propagation and inter-graph propagation to generate enhanced node embeddings.

[0083] The text graph augmentation representation device constructs semantic graphs, syntactic graphs, and temporal graphs based on the relationships between data. Figure 3 Fine-grained graphs are used to fuse heterogeneous information from three types of graphs through intra-graph propagation (aggregating neighbor node features in a single graph based on Graph Convolutional Network GCN) and inter-graph propagation (connecting nodes of different types of graphs through virtual graphs) to generate enhanced node embeddings, providing high-quality data support for subsequent model training and intent mining.

[0084] Interval sampling device: For each positive sample pair, the cosine similarity score is calculated and used as the sampling anchor point. The optimal threshold is used to define the sampling interval centered on the anchor point, ensuring that the selected negative samples have a high similarity with the positive samples, which can effectively improve the model's discrimination ability.

[0085] After calculating the cosine similarity score of the positive sample to the embedding vector, the interval sampling device uses the score as the sampling anchor point. Based on the optimal threshold (0.15) verified by a large number of experiments, a sampling interval centered on the anchor point is defined. This interval setting can ensure that the selected negative samples have a high similarity with the positive samples, and avoid the inclusion of false negative samples. Bidirectional negative sample mining is performed to filter intention hard negative samples and dialogue hard negative samples. If the number of hard negative samples is less than the preset value, effective samples are randomly supplemented from the candidate set.

[0086] Model training device: Set appropriate model parameters and combine forward loss and backward loss to comprehensively optimize the model's learning of dialogue and intent matching relationship.

[0087] The parameters in the base model training device were set as follows: learning rate 2e-05, dropout probability 0.1, maximum sequence length 32, training rounds 8, positive sample threshold 0.5, interval parameter k 0.15; hard negative sample dialogues 10, hard negative sample candidates 3; and forward and backward loss were combined to comprehensively optimize the model's learning of dialogue and intent matching relationships. The model parameters were iteratively updated using the gradient descent algorithm, and core indicators such as Precision, Recall, F1 score, and NDCG were regularly evaluated to adjust the training strategy in a timely manner, avoid model overfitting, and continuously improve the model's recognition accuracy for complex intents in medical scenarios.

[0088] Intent mining device: Receives relevant dialogue input from the user, automatically performs format standardization processing, inputs the standardized dialogue into the model to obtain dialogue embedding vectors, calculates the cosine similarity between this vector and all intent embedding vectors in the intent set, and outputs the intent based on the similarity score.

[0089] After receiving relevant dialogue input from the user, the intent mining device automatically performs format standardization processing, including unifying text encoding format, removing special symbols, and standardizing sentence structure, to ensure that the input text meets the model's processing requirements. The standardized dialogue input is then used to train a dual-encoder retrieval model, which is then processed through model encoding and pooling to obtain a fixed-dimensional dialogue embedding vector. Based on the cosine similarity algorithm, the similarity between this dialogue embedding vector and all intent embedding vectors in the intent set is calculated. The vectors are then sorted in descending order of similarity score, and the most probable intent result is output, providing a reference for clinical pre-diagnosis or user health consultation.

[0090] Figure 3 This paper presents a schematic diagram of UIMTH, a user intent mining method based on a dual-tower model, illustrating its overall architecture. UIMTH achieves performance breakthroughs by explicitly modeling three key relevance dimensions in dialogues: time series, domain semantics, and syntactic structure, and by combining interval constraints to improve the sampling strategy. UIMTH comprises four core components: a Sentence-BERT-based dual-tower user intent retrieval model responsible for embedding and retrieving user intent; a feature extraction module that effectively extracts latent relevance features using a text graph neural network model; an interval-constrained negative sample extraction module used to filter challenging negative sample data; and a noise contrast estimation module that trains the retrieval model using negative samples obtained from the interval-constrained negative sample extraction strategy.

[0091] This invention employs a dual-tower Sentence-BERT (SBERT) retrieval model as its foundation. Each tower acts as a BERT encoder, outputting fixed-dimensional sentence embedding vectors to encode user dialogue and intent respectively, achieving performance improvements over the original BERT model. The dual-encoder retrieval model optimizes sentence-level embedding performance by fine-tuning BERT on semantically labeled triplet data. The SBERT processing flow consists of two phases: training and prediction. During training, SBERT uses labeled sentence triples to fine-tune the BERT model through supervised learning, learning superior sentence embedding representations. During prediction, SBERT directly generates sentence embedding vectors using the fine-tuned BERT and performs semantic retrieval based on cosine similarity.

[0092] This invention proposes a feature embedding method based on TextGNN, which integrates SBERT embedding with graph neural networks. By constructing a text graph containing sentence nodes (covering both dialogue and intent) and word nodes, the embedding vectors generated by SBERT are optimized, improving the accuracy of vector representation. Specifically, we construct a multi-granularity dialogue graph to explicitly encode the interdependencies between semantics, syntax, and sequence:

[0093] Semantic graph: The existence and weight of edges are determined based on the cosine similarity of word embedding vectors; specifically, the edge weights corresponding to word pairs (nodes in the graph) with semantic relationships are calculated using the following formula [Liu X, You X, Zhang X, et al. Tensor graph convolutional networks for text classification[C]. Proceedings of the AAAI conference on artificial intelligence, 2020: 8409-8416.]:

[0094]

[0095] in, Representative words and The weight values ​​of the edges between them. Representative words and The number of times a text has a semantic relationship within the text collection. Representative words and The number of times a sentence appears in the entire text collection.

[0096] Syntactic Graph: CoreNLP is used to extract syntactic dependencies between words and simplify them into undirected relationships to calculate edge weights; specifically, words... and Edge weights between The calculation formula [Liu X, You X, Zhang X, et al. Tensor graph convolutional networks for text classification[C]. Proceedings of the AAAI conference on artificial intelligence, 2020: 8409-8416.] is as follows:

[0097]

[0098] in, Indicator and The number of times that have syntactic relations in a set. Indicator and The number of times a sentence appears in the corpus.

[0099] Sequence graphs: Combining Point Mutual Information (PMI) and a sliding window strategy, this approach characterizes local co-occurrence patterns of words. Specifically, it calculates the edge weights between each pair of words. The calculation formula [Liu X, You X, Zhang X, et al. Tensor graph convolutional networks for text classification[C]. Proceedings of the AAAI conference on artificial intelligence, 2020: 8409-8416.] is as follows:

[0100]

[0101] in, It is a word and The probability of two things appearing together in the same sliding window is determined by... To calculate, where This represents the total number of sliding windows across the entire text collection. Indicator and The number of times they appear together in the same sliding window on a text collection. Indicator The probability of appearing in a specific window on a text set, where Indicator The number of times it appears in all sliding windows across the entire text collection.

[0102] Subsequently, intra-graph and inter-graph propagation operations are performed on the text graph to aggregate node information and fuse features of identical nodes, ultimately obtaining the feature representation of the text nodes. This method, by comprehensively considering word associations and structural information, provides support for negative sample mining and noise contrast estimation. It can discover deep associations between positive samples and the candidate set, avoiding the missed detections or false positives that occur when relying solely on the similarity between the intent set and positive sample dialogues to mine negative samples. Specifically,

[0103] Intra-graph propagation learning aggregates information from the neighboring nodes of each node in the graph. Given a graph adjacency matrix... , This is the number of adjacency matrices, which is also the number of graphs. The features of the i-th graph in the l-th layer are updated by intra-graph propagation as follows [Liu X, You X, Zhang X, et al. Tensor graph convolutional networks for text classification[C]. Proceedings of the AAAI conference on artificial intelligence, 2020: 8409-8416.]:

[0104]

[0105] in, It is the symmetric normalization of the adjacency matrix of the i-th graph, where . It is an activation function; this study uses the ReLU function. It is the input of the i-th graph in the l-th layer. It is the first The output of the i-th graph in layer 1.

[0106] Specifically, inter-graph propagation is the process of information transmission and exchange between different graphs. Each graph has the same set of nodes, but its edge set and weights differ. This similarity and difference allows heterogeneous information from different graphs to gradually merge into consistent information. A virtual graph is constructed by connecting nodes in different graphs; all virtual graphs share the same set of nodes. Ultimately, n virtual graphs are obtained. By collecting the edge weights of these virtual graphs, a new graph adjacency matrix is ​​formed. The specific formula for graph propagation learning on virtual graphs [Liu X, You X, Zhang X, et al. Tensor graph convolutional networks for text classification[C]. Proceedings of the AAAI conference on artificial intelligence, 2020: 8409-8416.] is as follows:

[0107]

[0108] in, This represents the characteristics of node j across all graphs after inter-graph propagation. It is the adjacency matrix of the j-th virtual graph that connects nodes in different graphs. These are the input features that propagate within the graph from all nodes j in the graph. It is the trainable weight matrix of node j in layer l. This is a non-linear activation function. Inter-graph information transfer can further optimize the feature representation of nodes. By fusing node features from the semantic graph with node features from the syntactic graph and sequence graph, a more comprehensive text representation can be obtained.

[0109] After obtaining the embedding representations of the dialogue and candidate set using TextGNN, interval-constrained hard negative sample extraction is performed based on these representations. In the hard negative sample mining process, existing research typically selects a threshold as an anchor point: for positive sample pairs... Dialogues or intents with embedding similarity higher than the anchor point constitute a hard negative sample candidate set, while negative samples that are close to the dialogue in the embedding space but ranked relatively low are ignored. These ignored negative samples may affect retrieval performance. Therefore, the main goal of the rangeHNSE interval negative sample sampling method proposed in this invention is to discover these potential negative samples. For each positive sample pair... Suppose we need to mine M negative dialogue samples. and N intentional negative samples During excavation, with The similarity score is used as the anchor point anch, and a threshold is selected to define the sampling range: [anch-ranger, anch+range]. This invention introduces an interval-constrained negative sample mining method, which can more comprehensively consider candidate samples with different similarity scores, avoiding the limitation of only focusing on high-similarity candidate samples, and simultaneously including candidate samples with similarity higher or lower than the anchor point.

[0110] To improve the model's ability to recognize user dialogue intent, this invention proposes a bidirectional contrastive estimation method with interval constraints to handle hard negative sample noise during model training. The model is trained using bidirectional noise contrastive estimation (NCE): unlike traditional unidirectional NCE which only focuses on bringing positive sample pairs closer and pushing negative intents further away from the given dialogue distance, bidirectional NCE considers negative samples from both dialogue encoding and intent encoding perspectives. This bidirectional training approach significantly improves the model's accuracy and robustness. Given a patient dialogue d and an intent set... The intended target is from The subset of intents relevant to the dialogue is selected from the positive samples. Expectations for dialogue With positive intention The distance is relatively short, while the negative intention is expressed. The distance is relatively far. The goal of forward noise contrastive estimation is to train the model so that it can accurately identify the true intent from all intents for a given dialogue; the goal of backward noise contrastive estimation is to train the model so that it can identify the correct dialogue from all dialogues for a given intent. This study proposes a novel interval-constrained hard negative sample mining method, which extracts negative sample results from indistinguishable incorrect predictions as interval hard negative samples. It selects to generate interval hard negative samples from the model at the beginning of each training cycle and adds the interval hard negative samples to the NCE, finally obtaining the training representation of the model. The loss of combined noise contrastive estimation is defined as follows:

[0111] Forward loss:

[0112]

[0113] Reverse loss:

[0114]

[0115] in, The cosine similarity function; This represents the number of negative sample pairs. For hyperparameters; The number of hard negative samples. For dialogue embedding vectors, For the intended embedding vector, To embed vectors with positive intent, Negative intention embedding vector, Negative dialogue embedding vectors This is for exponential operations.

[0116] Total loss:

[0117]

[0118] The experimental scenarios of the present invention are described in detail below, and the implementation results are analyzed in light of the advantages of the present invention.

[0119] The hardware environment used in the experiment was as follows: CPU, Intel(R) Core(TM) i7-14700KF, 3.40GHz, 20 cores and 28 threads; GPU, NVIDIA GeForce RTX 4080 SUPER, 16GB video memory; memory, 128GB; hard drive, 932GB.

[0120] Two datasets will be used for the experiment: IPCD and CHIP-CDN. Details of the datasets are as follows:

[0121] IPCD: The dataset consists of approximately 40,000 online medical dialogues collected from an intelligent pre-consultation system. Researchers labeled each dialogue to determine if it contained medical intent or other medical needs. For dialogues containing medical intent, the researchers used the intents appearing in the dialogue for labeling. Ultimately, approximately 22,000 medical dialogues containing descriptions of medical intents were obtained. The average number of intents in these dialogues was 1.67, and the average length was 31.17. To accommodate a dual-encoder structure, i.e., encoding one intent at a time, dialogues containing multiple intents in the training and validation sets were paired with each intent to obtain a separate positive sample, resulting in a positive sample set. This invention randomly selects 1,000 positive samples containing multiple intents as test data, selects 8,500 positive samples as training data, and includes a total of 2,500 positive and negative samples in the validation data. The negative samples in the validation data are generated by randomly selecting irrelevant intents from a given dialogue.

[0122] CHIP-CDN: The public dataset of the Chinese Biomedical Language Understanding Evaluation Benchmark (CBLUE), used for semantic standardization of real diagnostic entities mined from Chinese electronic medical records. First, the data format needs to be processed to meet the requirements of the model in this invention. For each real diagnosis in the dataset, one or more intents are standardized, and each standardized intent is paired with a separate positive sample. This invention extracts 4401 intents from the public dataset CHIP-CDN. The training set contains 7000 data points (only positive samples), the validation set contains 2500 data points (including both positive and negative samples), and the test set contains 2000 data points.

[0123] To measure the effectiveness and efficiency of the method proposed in this invention, this experiment will select Precision, Recall, F1 score, and NDCG as evaluation metrics. These evaluation metrics will be introduced below:

[0124] Assuming the user provides the following dialogue: The goal is to retrieve Corresponding intent set Assuming for The result set retrieved by the proposed UIMTH method is The actual result set is .

[0125] Precision represents the proportion of truly relevant documents among the search results. Specifically, precision reflects the proportion of correct intents among all retrieved intents. Precision is calculated as follows:

[0126]

[0127] Recall represents the proportion of relevant documents retrieved out of all relevant documents. Specifically, recall reflects the proportion of retrieved correct intents out of all correct intents. Recall is calculated as follows:

[0128]

[0129] The F1 score (F-Measure) is a comprehensive evaluation metric that combines precision and recall. It balances the relationship between the two and is the harmonic mean of precision and recall. It provides a more comprehensive assessment of a retrieval system's performance and serves as an overall evaluation of the search results. The F1 score is calculated as follows:

[0130]

[0131] The NDCG (Normalized Discounted Cumulative Gain) evaluation metric combines the relevance of results with their ranking order, weighting the ranking results to reflect users' actual preferences. The NDCG value is obtained by calculating the cumulative gain and discounted cumulative gain for each item and then standardizing them. NDCG values ​​range from 0 to 1; higher values ​​indicate better model performance in ranking results.

[0132] Gain refers to the relevance score of elements in the sorted list, which is also the predicted probability value. Assuming... This represents the relevance score of the i-th element in the sorted list. The cumulative gain is a weighted sum based on the relevance of the search results, without considering the influence of position. The formula for calculating the cumulative gain at a specified position k is shown below:

[0133]

[0134] Loss of cumulative gain This refers to reducing the CG value of each result to give greater weight to the impact of higher-ranked results on the final outcome. The cumulative gain calculation method is as follows:

[0135]

[0136] Since different search terms return different search results, and DCG is calculated cumulatively, it's not possible to directly compare the DCG values ​​of two different search results. To solve this problem, the DCG value needs to be normalized, which is achieved by dividing it by the ideal DCG, i.e., the IDCG value. Here, IDCG represents the maximum DCG value of the ideal, standard answer. The NDCG calculation formula is shown below:

[0137]

[0138] The comparison method is selected as follows:

[0139] Bi-hardNCE: This model employs a Siamese-BERT architecture with bidirectional NCE loss, training the model by maximizing the similarity difference between positive and negative samples. In Bi-hardNCE, for each sample, its hard negative sample data is mined for model training.

[0140] Sent-BERT: This model uses a Siamese-BERT architecture, incorporating two BERT branches and adding a softmax classifier based on their cascaded features. When constructing negative sample pairs, an intent is randomly selected from a list of intents in a given dialogue. The loss function uses cross-entropy to measure the difference between the model's predictions and the true labels.

[0141] TRIPLET: This model receives a triple (dialogue, true intent, negative sample intent), where the negative sample intent is randomly selected from a list of intents. Using this triple, the model aims to minimize the distance between the dialogue and the true intent, while maximizing the distance between the dialogue and the negative sample intent.

[0142] CONTRASTIVE: For a given dialogue and intent pair, the distance between them is first calculated using cosine similarity. If it is a positive pair, the distance between them needs to be maximized; otherwise, the distance between them needs to be minimized. When constructing negative sample pairs, an intent is randomly selected from the list of intents of the given dialogue as a negative sample.

[0143] NCE-forward: This model is an architecture that uses the Siamese-BERT structure and features forward NCE loss. Negative samples are taken from negative samples within the batch, and the model is trained by maximizing the similarity difference between positive and negative samples.

[0144] To fully evaluate the algorithm from various perspectives, this invention will design four sets of experiments based on the characteristics of the model and dataset:

[0145] Experiment 1: Performance Comparison Experiment. This experiment will compare the intent retrieval results of the UIMTH model with those of models based on Bi-hardNCE, Sent-BERT, TRIPLET, CONTRASTIVE, and NCE-forward, using the same dataset. NDCG, Precision, Recall, and F1 scores will be calculated for each intent retrieval result.

[0146] Experiment 2: Parameter Sensitivity Analysis Experiment. This experiment will investigate the impact of the interval threshold k of the interval sampling strategy on the retrieval results.

[0147] Experiment 3: Comparative Experiment of Negative Sample Mining Methods. This experiment compares the proposed interval negative sample sampling strategy with TopK and HTT methods to demonstrate its feasibility and superiority.

[0148] Experiment 4: Ablation Experiment. The following ablation experiments were conducted on the user intent mining algorithm proposed in this paper: (1) The performance of the text graph neural network-enhanced embedding method with and without the text graph neural network was compared and analyzed to demonstrate the importance of the text graph neural network-enhanced embedding method. (2) The impact of using or not using intra-graph propagation or inter-graph propagation on the user intent mining results was compared and analyzed. (3) The performance of the method with and without semantic graph, syntactic graph, sequence graph representation and other cases was compared and analyzed to verify the impact of different graphs on the user intent mining results.

[0149] 1. Performance Comparison Experiment

[0150] The results of this invention on dataset 1 are shown in Table 1. It can be seen that UIMTH significantly outperforms all baseline models across all metrics. Specifically, the UIMTH model improves upon the best baseline Bi-hardNCE model by 3.52%, 1.62%, 4.6%, and 2.45% in NDCG@5, Precision@5, Recall@5, and F1@5, respectively. The UIMTH model using bidirectional graph augmentation with interval negative samples outperforms the Bi-hardNCE model using bidirectional hard negative samples, as well as the Sent-BERT, CONTRASTIVE, and TRIPLET models using randomly generated negative samples. This indicates that when training intent retrieval models, our method, through bidirectional graph augmentation and interval negative sample sampling, is more effective than the Bi-hardNCE model using only hard negative samples, unidirectional NCE-forward, and other models using randomly generated negative samples.

[0151] Table 1: Performance Comparison of IPCD

[0152]

[0153] Performance comparison results of this invention with other baseline methods are as follows: Figure 4 As shown, the horizontal axis represents the training epoch, and the vertical axis represents the evaluation metrics of Recall, Precision, NDCG, and F1. Observing the experimental results, it can be seen that for all evaluation metrics, the UIMTH model consistently outperforms all baseline models from the first epoch to the last epoch of training. This indicates that the proposed method has a sustained performance advantage, maintaining better performance than the baseline model throughout the entire training process.

[0154] The experimental results on CHIP-CDN are shown in Table 2. It can be seen that the UIMTH model proposed in this study performs best, achieving improvements of 0.4%, 0.5%, 1.41%, and 0.74% in NDCG@5, Precision@5, Recall@5, and F1@5 compared to the best baseline Bi-hardNCE model, respectively. All methods perform better on dataset 2 than on dataset 1. This is because CHIP-CDN has higher data quality, and analysis reveals that the dialogues and intents in its test data are more relevant.

[0155] Table 2: Performance Comparison of CHIP-CDN

[0156]

[0157] In summary, based on the patterns and comparison results shown by the evaluation indicators calculated from the results in Experiment 1, it can be seen that UIMTH has a better intention mining effect overall compared to the baseline model, which proves the effectiveness of the UIMTH model.

[0158] 2. Parameter sensitivity analysis experiment

[0159] This section of the experiment will adjust the interval threshold parameter k of the interval negative sample mining method to explore its impact on the performance of the UIMTH method. Retrieval tasks will be performed on different k values ​​(taken from the set {0.10, 0.15, 0.20, 0.25, 0.30}), and the retrieval results will be analyzed using four evaluation metrics.

[0160] Table 3: Performance Comparison under Different Threshold Parameters k

[0161]

[0162] Table 3 shows the performance results of user intent mining. It was observed that Precision@5, Recall@5, and F1@5 reached their maximum values ​​when k was set to 0.15. Although k=0.15 performed worse than the maximum value for NDCG@5, overall, k=0.15 showed the best performance. The reasons are as follows: choosing too small a k value may lead to scarce or missing data within the interval, resulting in random selection of negative samples and affecting data quality; while too large a k value may lead to the use of false negative samples, thus affecting the accuracy of the experimental results.

[0163] 3. Comparative Experiment of Negative Sample Mining Methods

[0164] To compare the performance of interval hard negative sample mining methods with other hard negative sample mining methods in intent detection, this section presents a comparative experiment, primarily comparing the following two mining methods:

[0165] 1. The TopK method selects the top k intents with the highest similarity scores as hard negative samples.

[0166] 2. Higher Than True (HTT): Select the intent that ranks higher than the true intent as a hard negative sample.

[0167] In addition, this paper adds an experimental design for eliminating spurious negative samples (-FN) for each method to evaluate the effectiveness of eliminating spurious negative samples.

[0168] Table 4: Comparison of Different Negative Sample Mining Methods

[0169]

[0170] Table 4 shows the performance of different hard negative sample mining methods on the Recall@5, Precision@5, NDCG@5, and F1@5 evaluation metrics without text graph augmentation on IPCD, where rangeHNSE represents the interval hard negative sample mining method proposed in this study. Analysis of the results in the table shows that the negative sample mining method of this invention outperforms the other two methods. Even after incorporating a false negative sample elimination strategy, the performance of the first two methods is still inferior to our method. This is because this invention not only considers the influence of false negative samples, but also makes the hard negative sample data closer to the correct data in the feature space, thereby improving the quality of the training data. With the introduction of the false negative sample elimination strategy, the performance of both TopK and HTT mining methods shows significant improvement; for example, NDCG@5 is improved by 1.77% and 0.89%, respectively. This indicates that hard negative sample mining strategies that heavily rely on high similarity scores are more susceptible to the influence of false negative sample data, as the top-ranked samples are likely to be false negative samples. This further demonstrates the importance of eliminating false negative sample data.

[0171] 4. Ablation test

[0172] The purpose of this ablation experiment is to gain a deeper understanding of the impact of its different components on overall performance by adjusting and modifying parts of the model. This experiment mainly consists of two parts:

[0173] (1) Prove the importance of text graph neural network-enhanced embedding methods:

[0174] To explore the importance of text graph neural network embedding enhancement methods in improving the quality of user intent mining results, this study removed interval negative sample mining methods and adopted the general HTT negative sample mining technique to compare and analyze the differences between applying and not applying text graph neural network embedding enhancement methods.

[0175] Experimental results are as follows Figure 5 As shown, the horizontal axis represents different evaluation metrics, and the vertical axis represents the specific values ​​of the metrics. Analysis of the four evaluation metrics reveals that the effect of introducing the text graph augmentation method is significantly better than that of not using this method. For example, Recall@5 is improved by 2.97%. This also verifies that this study, by constructing semantic graphs, syntactic graphs, and sequence graphs, achieves deep connections between dialogues and candidate sets, thereby tightly linking each sentence node. Graph propagation learning significantly enhances the quality of node embedding representations, especially playing a crucial role in the process of mining negative samples. This experiment highlights the important role of the graph augmentation method of this invention in improving model performance and verifies the importance of graph structure and propagation methods in text graph neural networks.

[0176] To verify the broad applicability of the graph augmentation strategy proposed in this invention, the Sent-BERT model was selected as the experimental platform. Our graph augmentation strategy was compared with two other different negative sample extraction strategies: a random negative sample strategy, a graph-augmented HTT strategy, and a graph-augmented HTT strategy. Experimental results are as follows: Figure 6 As shown, the horizontal axis represents different evaluation indicators, and the vertical axis represents the specific values ​​of the indicators. The graph augmentation strategy first enhances the text table embedding through a text graph neural network, and then performs negative sample mining. The HTT strategy without graph augmentation refers to directly mining negative samples based on similarity. The HTT strategy without graph augmentation and the HTT strategy based on graph augmentation maintain the same negative sample mining method; the only difference is whether or not a text graph neural network is introduced to enhance the embedding technique. Figure 6 Analysis shows that, across the four evaluation metrics examined, the graph-augmented HTT strategy outperforms both the non-graph-augmented HTT strategy and the random negative sample strategy, while the non-graph-augmented HTT strategy also outperforms the random negative sample strategy. This result demonstrates that the non-graph-augmented HTT strategy is more effective than the random negative sample strategy. Furthermore, the method of negative sample mining using text graph neural network augmentation embedding technology surpasses the effectiveness of directly mining negative samples based on similarity. These findings not only validate the effectiveness of the text graph neural network augmentation embedding technology of this invention in improving retrieval performance but also confirm the broad applicability of the proposed method.

[0177] (2) Comparative analysis of the impact of using or not using intra-graph propagation or inter-graph propagation on user intent mining results:

[0178] To investigate the role of inter-graph propagation, a comparative experiment was designed to evaluate the differences between propagation limited to within the graph and propagation both inside and outside the graph. The experiment particularly focused on the performance changes of the algorithm without inter-graph propagation, thus clarifying the impact of inter-graph propagation on model performance. Experimental results are as follows: Figure 7As shown, the horizontal axis represents different evaluation indicators, and the vertical axis represents the specific values ​​of the indicators. From Figure 7 The analysis of the four evaluation metrics provided shows that a strategy combining inter-graph propagation and intra-graph propagation significantly outperforms methods relying solely on intra-graph propagation. If, after obtaining semantic graphs, syntactic graphs, and sequence graphs, only intra-graph propagation is performed while inter-graph propagation is ignored, the model's ability to capture potential relationships between different graph structures may be limited. This is because semantic graphs, syntactic graphs, and sequence graphs may contain complementary information; effective propagation of this information between graphs would help improve model performance. Therefore, even with effective information propagation and aggregation operations within the graphs, optimal results may not be achieved.

[0179] (3) Comparative analysis of the performance of methods using and not using semantic graphs, syntactic graphs, sequence graphs, etc., to verify the impact of different graphs on the results of user intent mining:

[0180] To evaluate the contributions and importance of semantic graphs, syntactic graphs, and sequence graphs in the model, this invention designed a series of comparative experiments. Specifically, three combination methods were constructed: semantic graphs combined with syntactic graphs, semantic graphs combined with sequence graphs, and syntactic graphs combined with sequence graphs. The results of these combinations were compared and analyzed with those of methods that simultaneously included all three types of graphs. The purpose of these experiments was to explore the impact of different graph combinations on model performance, thereby verifying the unique value and necessity of each graph in the model. Experimental results are as follows: Figure 8 As shown, the horizontal axis represents different combinations of text graphs, and the vertical axis represents the NDCG, Precision, Recall, and F1 evaluation metrics. Figure 8 The four evaluation metrics showed that the experimental results of combining semantic graphs with syntactic graphs, semantic graphs with sequence graphs, and syntactic graphs with sequence graphs were all inferior to those of combining all three types of graphs simultaneously. This observation highlights the importance of each type of graph in text representation. The absence of any one type of graph leads to incomplete information, thus affecting the final experimental results. Therefore, the comprehensive application of the three types of graphs provides a more comprehensive method for text feature extraction, thereby improving the performance of user intent mining.

[0181] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A user intent mining method based on a dual-tower model, characterized in that: The specific steps are as follows: Step 1: Obtain raw network data from the network data source, set interval thresholds and hard negative sample thresholds; process and clean the raw network data to obtain user dialogue text and intent document text; Step 2: Construct a dual-encoder retrieval model that includes encoding user dialogue text and encoding intent document text, and output a fixed-dimensional set of dialogue embedding vectors and intent embedding vectors. Step 3: Based on the user dialogue text, intent document text, and the embedding vectors in the fixed-dimensional dialogue embedding vector set and intent embedding vector set, construct three types of fine-grained graphs: semantic graph, syntactic graph, and temporal graph. Generate enhanced dialogue node embedding representations and enhanced intent node embedding representations through intra-graph propagation and inter-graph propagation. Step 4: Using the similarity score of positive sample pairs as anchor points, and through interval thresholds, mine the enhanced dialogue node embedding representation and the enhanced intent node embedding representation to obtain hard negative samples. Step 5: Using the noise contrast estimation method, the dual encoder retrieval model is trained by combining the hard negative samples. The model parameters are optimized by minimizing the forward loss and backward loss to obtain the trained dual tower model. Step 6: Input the user dialogue to be mined into the trained dual-tower model, calculate the similarity with the candidate intents, sort the similarity and return the intent mining results.

2. The user intent mining method based on a dual-tower model according to claim 1, characterized in that, Step 1 specifically includes the following steps: Step 1.1: Collect raw network data containing dialogue and intent from the data source; Step 1.2: Process and clean the collected raw data to extract valid text information; Step 1.3: Obtain the user dialogue text and intent document text.

3. The user intent mining method based on a dual-tower model according to claim 1, characterized in that: The dual encoder retrieval model described in step 2 is built based on Sentence-BERT (SBERT) and adopts a twin network structure. The two encoders share weights and both include the BERT infrastructure and pooling layer.

4. The user intent mining method based on a dual-tower model according to claim 1, characterized in that, Step 3 specifically includes the following steps: Step 3.1: Calculate the edge weights between nodes in the semantic graph based on the cosine similarity of the embedded vectors; Step 3.2: Use CoreNLP tools to extract syntactic dependencies between words, construct a syntactic graph in the form of undirected relations, and calculate edge weights; Step 3.3: Calculate the edge weights of word co-occurrence in the time series graph using point mutual information and a sliding window strategy; Step 3.4: Fuse the heterogeneous information of the semantic graph, syntactic graph and temporal graph through intra-graph propagation and inter-graph propagation to generate enhanced dialogue node embedding representations and enhanced intent node embedding representations.

5. The user intent mining method based on a dual-tower model according to claim 1, characterized in that, Step 4 specifically includes: for each positive sample pair ( It can extract M negative dialogue samples. and N intentional negative samples When mining hard negative samples, select positive sample pairs ( The similarity score between the two points is used as the anchor point anch; With the anchor point anch as the center, define the sampling interval [anch-range, anch+range] according to the interval threshold range; Traverse the intent embedding vector set and filter out non-target intents whose similarity scores with the user dialogue fall within the sampling interval, as hard negative samples of the intent corresponding to the dialogue. Traverse the set of dialogue embedding vectors and filter out non-matching dialogues whose similarity scores with the target intent fall within the sampling interval, as hard negative samples of the dialogue corresponding to the intent. If the number of hard negative samples is less than the hard negative sample threshold, then they are randomly supplemented from the fixed-dimensional embedding vectors that have been traversed.

6. The user intent mining method based on a dual-tower model according to claim 1, characterized in that, The forward loss function described in step 5 and inverse loss function The specific calculation methods include: in, The cosine similarity function; This represents the number of negative sample pairs. For hyperparameters; The number of hard negative samples. For dialogue embedding vectors, For the intended embedding vector, To embed vectors with positive intent, Negative intention embedding vector, Negative dialogue embedding vectors For exponential operations, This represents the total loss.

7. The user intent mining method based on a dual-tower model according to claim 1, characterized in that, Step 6 specifically includes the following steps: Step 6.1: Input the user dialogue to be mined into the trained dual encoder retrieval model to obtain the dialogue embedding vector; Step 6.2: Calculate the cosine similarity between the dialogue embedding vector and all intent embedding vectors in the intent embedding vector set; Step 6.3: Sort the results in descending order of similarity scores to obtain the most likely intent as the user search intent mining results.

8. A user intent mining system based on a dual-tower model according to any one of claims 1 to 7, characterized in that: It includes a data acquisition device, a model building device, a text graph enhancement and representation device, an interval sampling device, a model training device, and an intent mining device; The data acquisition device collects raw data from the network platform and processes it into logical relationships of dialogue intent, thus preparing for the subsequent construction of a user search intent mining system based on a dual-tower model. The model building device constructs a dual-encoder retrieval model. Both encoders adopt the Sentence-BERT structure and share weights. The dialogue encoder takes user dialogue text as input and outputs dialogue embedding. The intent encoder takes intent description text as input and outputs intent embedding. The encoder output is transformed into a fixed-dimensional vector through pooling operations, which provides the basis for subsequent similarity calculation. The text graph enhancement representation device constructs three types of fine-grained graphs: semantic graph, syntactic graph, and temporal graph. It fuses the heterogeneous information of the three types of graphs through intra-graph propagation and inter-graph propagation to generate enhanced node embeddings. The interval negative sample sampling device calculates the cosine similarity score for each positive sample pair, which serves as the sampling anchor point. The optimal threshold is used to define the sampling interval centered on the anchor point, ensuring that the selected negative samples have a high similarity to the positive samples, which can effectively improve the model's discrimination ability. The model training device is configured with appropriate model parameters and combines forward loss and backward loss to comprehensively optimize the model's learning of the relationship between dialogue and intent matching. The intent mining device receives relevant dialogue input from the user, automatically performs format standardization processing, inputs the standardized dialogue into the model to obtain a dialogue embedding vector, calculates the cosine similarity between this vector and all intent embedding vectors in the intent set, and outputs the intent based on the similarity score.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that: When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1 to 7.

Images