Intention recognition method and device based on user clustering and theme sequence

Abstract

The invention provides an intention recognition method and device based on user clustering and a theme sequence, and relates to the technical field of user intention recognition, and the method comprises the steps: obtaining and standardizing user feature data, determining an optimal cluster number through an elbow rule, and carrying out the clustering through a k-means algorithm to generate a user feature tag; extracting dialogue text topics, determining an optimal topic number according to standardized point mutual information, classifying the optimal topic number into a topic sequence, and integrating the topic sequence to generate an observation state sequence with user characteristics; determining a hidden state number of the hidden Markov model according to a Bayesian information criterion, and inputting an observation sequence to train and generate a transfer and observation probability matrix; and identifying potential intentions of the user by combining the matrix and the theme sequence, and generating a corresponding intention transition probability matrix. According to the method, the user features and the dialogue theme sequence are combined, the intention recognition accuracy and the personalized adaptation capability of the low-data-volume scene are improved, and the computing resource consumption is reduced.

Description

Technical Field

[0001] This invention relates to the field of user intent recognition technology, and in particular to an intent recognition method and apparatus based on user clustering and topic sequences. Background Technology

[0002] User intent recognition, a core technology of Natural Language Processing (NLP), is widely used in dialogue systems such as intelligent customer service and human-computer interaction. With the development of deep learning and pre-trained language models, related technologies have constructed a complete processing chain from semantic parsing to intent classification through collaborative work involving large-scale corpus training and fine-tuning. Specifically, this technology system covers the entire process from text feature extraction and context modeling to intent decision-making, including key steps such as rule-based label matching, semantic encoding of deep neural networks, and sequential modeling using Hidden Markov Models (HMMs). Among these, HMMs characterize the hidden state transition probability matrix... With the emission probability matrix It demonstrates computational efficiency advantages in time-dependent dialogue scenarios, with its model parameters being only 1 / 100 of traditional large models and memory usage reduced by 85%.

[0003] However, existing Hidden Matrix (HMM) intent recognition methods directly use the label sequence of the dialogue text as the observed state without incorporating user feature data for differentiated modeling. This may lead to insufficient generalization ability of intent judgment or failure to capture the expressive differences among user groups, thus affecting the accuracy of personalized services. Specifically, related technologies split the text into label sequences and input them into an HMM for unsupervised learning, but the generated hidden state transition probability matrix... and emission probability matrix This approach only reflects general dialogue patterns and does not consider the impact of user characteristics such as age and interests on intent expression. In low-data-volume scenarios, this single-dimensional modeling is prone to overfitting and cannot improve training stability by sharing data within user clusters. Therefore, existing technologies face systemic shortcomings in vertical applications (such as healthcare and education), including a sharp drop in accuracy and low resource utilization, necessitating the development of a joint modeling framework that integrates user features and text sequences. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose an intent recognition method based on user clustering and topic sequences.

[0006] The second objective of this invention is to propose an intent recognition device based on user clustering and topic sequence.

[0007] The third objective of this invention is to provide an electronic device.

[0008] The fourth objective of this invention is to provide a computer-readable storage medium.

[0009] The fifth objective of this invention is to provide a computer program product.

[0010] To achieve the above objectives, a first aspect of the present invention proposes an intent recognition method based on user clustering and topic sequences, comprising: S1. Obtain user feature data and perform standardization processing. Determine the optimal number of clusters based on the elbow rule. Use the k-means algorithm to perform unsupervised clustering of users and generate user feature clustering labels. S2, extract topics from the dialogue text, determine the optimal number of topics based on standardized point mutual information, classify the dialogue text sentence by sentence into the corresponding topic sequence, and integrate with user feature clustering labels to generate an observation state sequence with user features; S3. Determine the number of hidden states of the Hidden Markov Model based on the Bayesian information criterion, input the observed state sequence into the Hidden Markov Model, and generate the hidden state transition probability matrix and the observation probability matrix through multiple rounds of iterative training. S4. Based on the hidden state transition probability matrix and the observation probability matrix, and combined with the topic sequence information, identify the user's potential intentions and generate an intention transition probability matrix corresponding to the user feature clustering.

[0011] Optionally, in S11, the user feature data is standardized using a min-max normalization method, mapping features of different scales to the [0,1] interval. The specific calculation formula is as follows:

[0012] S12, calculate the sum of squares (SSE) for different cluster numbers using the elbow rule, plot the SSE-cluster number curve, and select the inflection point as the optimal cluster number. The specific calculation formula is as follows:

[0013] in For the first Clusters, For the first The centroid of a cluster.

[0014] Optionally, in S21, the normalized point mutual information (NPMI) formula is used to calculate word co-occurrence relations. The specific calculation formula is as follows:

[0015] in For point-to-point information, For words and The joint probability.

[0016] Optionally, in S31, the model evaluation value for different numbers of hidden states is calculated using the Bayesian Information Criterion (BIC) formula. The specific calculation formula is as follows:

[0017] in This refers to the number of model parameters (i.e., the number of topics). The total number of dialogue texts. This represents the model likelihood function value.

[0018] Optionally, S41, based on the topic-hidden state correspondence in the observation probability matrix, the hidden state is mapped to the intent using the maximum probability matching rule. The specific calculation formula is as follows:

[0019] in For the first Sentence text in theme The probability of it, Hidden state To the topic The probability of emission.

[0020] Optionally, S5 dynamically updates the generated intent transition probability matrix, adjusts the clustering results according to the real-time changes in user feature data, and retrains the Hidden Markov Model based on the updated clustering labels to adapt to the evolution of user group characteristics.

[0021] To achieve the above objectives, a second aspect of the present invention provides an intent recognition device based on user clustering and topic sequences, comprising: The user feature standardization and clustering module is used to acquire user feature data and perform standardization processing. It determines the optimal number of clusters based on the elbow rule, uses the k-means algorithm to perform unsupervised clustering of users, and generates user feature clustering labels. The dialogue text topic integration module is used to extract topics from dialogue text, determine the optimal number of topics based on standardized point mutual information, classify dialogue text sentence by sentence into corresponding topic sequences, and integrate them with user feature clustering labels to generate observation state sequences with user features. The Hidden Markov Model Training Module is used to determine the number of hidden states of the Hidden Markov Model based on the Bayesian information criterion, input the observed state sequence into the Hidden Markov Model, and generate the hidden state transition probability matrix and the observation probability matrix through multiple rounds of iterative training. The intent recognition and probability matrix generation module is used to identify the user's potential intent based on the hidden state transition probability matrix and the observation probability matrix, combined with topic sequence information, and generate an intent transition probability matrix corresponding to the user feature clustering.

[0022] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of the first aspects.

[0023] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of the first aspects.

[0024] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product that, when executed by a processor, implements the method described in any one of the first aspects.

[0025] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects: The methods, apparatus, electronic devices, and computer-readable storage media of this invention improve the accuracy of intent recognition and personalized adaptation capabilities based on Hidden Markov Models by combining user feature clustering with dialogue text topic sequence modeling in low-data-volume scenarios.

[0026] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0027] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating an intent recognition method based on user clustering and topic sequence provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a specific process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an intent recognition device based on user clustering and topic sequence provided in an embodiment of the present invention. Detailed Implementation

[0028] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0029] Example 1 To address the issues of insufficient user feature data for intent recognition and the need for small-scale scenarios with low data volume but significant serialization, this invention provides an intent recognition method based on user clustering and topic sequences. Figure 1 This is a flowchart illustrating an intent recognition method based on user clustering and topic sequences, provided as an embodiment of the present invention. Figure 1 As shown, the method includes the following steps: S1. Obtain user feature data and perform standardization processing. Determine the optimal number of clusters based on the elbow rule. Use the k-means algorithm to perform unsupervised clustering of users and generate user feature cluster labels.

[0030] Specifically, in step S1, the system first acquires the user's historical feature data, including but not limited to unstructured or structured information such as age, gender, and interests. This feature data is typically represented in multi-dimensional vector form, for example: ,in Indicates the first The user in the first The system takes values ​​across each feature dimension. Due to the significant differences in the units and numerical ranges of different features, a standardization method is employed to map each feature dimension to a uniform numerical range, typically [0, 1] or [-1, 1], to eliminate the influence of scale bias on the clustering results. Z-score standardization can be optionally used, with the formula:

[0031] in For the first The mean of each feature, denoted as the standard deviation. The standardized data will be used as input for subsequent clustering algorithms.

[0032] Furthermore, the system determines the optimal number of clusters based on the Elbow Method. This method calculates different Plotting the Within-Cluster Sum of Squares (WCSS) under the given values. The relationship curve between WCSS and the target number of clusters is used, with the inflection point of the curve representing the optimal number of clusters. In some implementations, the formula for calculating WCSS is:

[0033] in Indicates the first Clusters, This is the center vector of the cluster. Through visual analysis, the system can determine... A reasonable value should be chosen to avoid overfitting or underfitting.

[0034] Subsequently, the system employs the k-means algorithm to perform unsupervised clustering on the standardized user feature data. k-means iteratively optimizes cluster centers, minimizing the distance between samples within a cluster and the cluster center. In practical applications, this step can be deployed in a distributed computing environment to handle large-scale user data. After clustering, each user is assigned a cluster label. This is used to distinguish user categories in the subsequent intent recognition module.

[0035] This step plays a crucial role in the entire technical solution. By dividing users into groups with similar characteristics, it provides user category context for subsequent intent recognition based on Hidden Markov Models, thereby improving the accuracy and robustness of intent recognition in low-data-volume scenarios.

[0036] Furthermore, S1 includes: S11, uses the minimum-maximum normalization method to standardize user feature data, mapping features of different scales to the [0,1] interval. The specific calculation formula is as follows:

[0037] Specifically, in this application proposal, step S1-1 involves standardizing the user feature data using the Min-Max Normalization method. The core principle of this method is to linearly map the original feature data to the [0,1] interval, thereby eliminating the dimensional differences between different feature dimensions and improving the stability and accuracy of subsequent clustering algorithms.

[0038] In one embodiment of the present invention, for a certain user feature Its normalized value It can be calculated using the following formula:

[0039] in, and These represent the minimum and maximum values ​​of the feature in the entire user dataset, respectively. This formula compresses all feature values ​​into a uniform numerical range, preventing certain features from dominating the clustering results due to excessively large numerical ranges.

[0040] In this embodiment, to enhance robustness, a sliding window mechanism can be introduced, that is, when dynamically updating user data, only the most recent window is used. Data calculation within a time window and This allows the system to adapt to scenarios where user characteristics change over time. Furthermore, to prevent division by zero, a minimum value can be added to the denominator. ,For example To ensure numerical stability.

[0041] This step is typically applied during the user profile building phase. Feature data includes, but is not limited to, age, gender, interest tags, spending level, and usage frequency. In actual systems, these features may come from multiple heterogeneous data sources such as user registration information, behavior logs, and device usage records, and need to be standardized after data cleaning.

[0042] By using min-max normalization, user feature data can participate more fairly in distance calculations during subsequent k-means clustering, improving the accuracy and interpretability of clustering. This step provides standardized input for user profiles for the entire intent recognition system and is a crucial preprocessing step for achieving personalized intent recognition.

[0043] S12, calculate the sum of squares (SSE) for different cluster numbers using the elbow rule, plot the SSE-cluster number curve, and select the inflection point as the optimal cluster number. The specific calculation formula is as follows:

[0044] in For the first Clusters, For the first The centroid of a cluster.

[0045] Specifically, in step S12, this application employs the Elbow Method to perform unsupervised clustering of user feature data to determine the optimal number of clusters. The core of this step lies in calculating the sum of squared errors (SSE) under different numbers of clusters, plotting the SSE-cluster number curve, and selecting the "inflection point" of the curve as the optimal number of clusters, thereby providing a basis for user category classification for subsequent intent recognition.

[0046] In this embodiment, the elbow rule is evaluated based on the SSE metric of clustering algorithms (such as K-means). SSE is defined as the sum of the squares of the Euclidean distances between all samples and the centers of their respective clusters, and its mathematical expression is:

[0047] in, This indicates the currently set number of clusters. Indicates the first The sample set in each cluster Let be the center vector of this cluster. Indicates sample The Euclidean distance between the cluster center and the cluster center. In practice, this application iterates through different cluster numbers. (Usually from 2 to 10), calculate each The corresponding SSE values ​​are plotted, and the SSE-cluster number curve is drawn. When the cluster number increases to a certain critical point, the rate of decrease in SSE slows down significantly; this inflection point is the optimal cluster number.

[0048] In one embodiment of the present invention, user feature data includes structured fields such as age, gender, and interests, which need to be standardized, for example, using the Z-score standardization method to make the various feature dimensions comparable. The number of iterations of the K-means algorithm is usually set to 100, and the initial centroids are randomly selected or the K-means++ strategy is used to improve clustering stability. The calculation precision of SSE can be set to floating-point numbers (such as 32-bit or 64-bit) to adapt to datasets of different sizes.

[0049] In this embodiment, this step is applicable to the preprocessing stage of user intent recognition, especially in scenarios with low data volume but obvious sequence characteristics, such as customer service dialogues and intelligent question-answering systems. By clustering users into groups with similar characteristics, more accurate observation sequence inputs can be provided for subsequent Hidden Markov Models, thereby improving the accuracy and generalization ability of intent recognition.

[0050] The technical advantage of this step lies in its ability to effectively avoid biases caused by subjectively setting the number of clusters through mathematical modeling and visualization analysis of the elbow rule, thereby improving the objectivity and stability of the clustering results. Furthermore, by combining the clustering results with user characteristics, it provides a more representative user category classification for subsequent intent recognition models, thus enhancing the model's ability to discern potential user intents. This has significant practical value and innovative significance, especially in resource-constrained or data-limited scenarios.

[0051] S2 extracts topics from the dialogue text, determines the optimal number of topics based on standardized point mutual information, classifies the dialogue text sentence by sentence into the corresponding topic sequence, and integrates user feature clustering labels to generate an observation state sequence with user features.

[0052] Specifically, this step aims to extract topics from the dialogue text and determine the optimal number of topics based on Normalized Pointwise Mutual Information (NPMI). Each sentence is then categorized into its corresponding topic, and finally, user feature clustering labels are combined to generate an observation state sequence with user characteristics. This process is performed before Hidden Markov Model (HMM) modeling and is a crucial preliminary step for achieving personalized intent recognition.

[0053] In this embodiment, the dialogue text is first segmented into sentences, and a Bag of Words (BBO) or Term Frequency-Inverse Document Frequency (TF-IDF) representation is constructed. Subsequently, unsupervised learning methods (such as LDA or NMF) are used for topic modeling. To determine the optimal number of topics, Normalized Point Mutual Information (NPMI) is introduced as an evaluation metric, and its calculation formula is as follows:

[0054] PMI is used to measure words and The co-occurrence relationship is determined by NPMI, which normalizes word pairs with different co-occurrence intensities to make them comparable. In some implementations, an NPMI threshold can be set (e.g., ...). This is used to filter word pairs with strong semantic connections, thereby helping to determine the rationality of the word distribution in the topic model and ultimately determining the optimal number of topics. .

[0055] Furthermore, each sentence is input into the trained topic model, and its probability distribution under each topic is calculated. The topic with the highest probability is selected as the observation state of that sentence, forming a topic sequence. This sequence serves as the observation input to the HMM, and its length is consistent with the number of sentences in the dialogue text. Each observation state corresponds to a topic label.

[0056] In one embodiment of the present invention, the number of iterations of the topic model needs to be set during the topic extraction process (e.g., ...). ), learning rate (e.g.) ), sparsity constraints (such as Regularization coefficient Methods such as […] are used to improve model convergence speed and topic discrimination. NPMI calculation relies on word co-occurrence frequency statistics, typically employing a sliding window (e.g., window size...). ) Perform context modeling.

[0057] In this embodiment, this step is applicable to scenarios such as customer service dialogues, user feedback analysis, and intelligent question-and-answer systems. It is particularly effective in improving the accuracy of intent recognition when data volume is limited but dialogues exhibit clear temporal characteristics. This is achieved by combining user clustering tags (such as...) This allows for the construction of differentiated observation state sequences for different user groups, thereby enhancing the model's ability to understand user intentions in a personalized way.

[0058] The technical effect of this step is that by introducing NPMI to optimize the topic extraction process, the semantic relevance of topic segmentation is improved; at the same time, by combining user clustering information, the observed state sequence is endowed with the context awareness of user features, providing higher quality input data for subsequent HMM modeling and intent recognition, and significantly enhancing the model's discriminative performance and generalization ability.

[0059] Furthermore, S2 includes: S21, use the Normalized Point Mutual Information (NPMI) formula to calculate word co-occurrence relations. The specific calculation formula is as follows:

[0060] in For point-to-point information, For words and The joint probability.

[0061] Specifically, in step S21, this application uses Normalized Pointwise Mutual Information (NPMI) as the core indicator to calculate the co-occurrence relationship between words, thereby assisting the topic extraction process. NPMI is an improved form based on Pointwise Mutual Information (PMI), which can more effectively measure the semantic or contextual association strength between two words, and is especially suitable for co-occurrence analysis in sparse data scenarios.

[0062] In this embodiment, the formula for calculating NPMI is:

[0063] in, Words and The mutual information between points is defined as:

[0064] This formula uses joint probability. With marginal probability and The ratio of the co-occurrence of two words measures the non-independence of their co-occurrence. NPMI introduces a normalization factor. By limiting the PMI value range to [-1, 1], words with different co-occurrence intensities are comparable, avoiding the problem that the PMI value may be negative or infinitely large.

[0065] In this embodiment, the application first constructs a word co-occurrence matrix, counts the co-occurrence frequency of word pairs in the dialogue text, and then calculates their NPMI values. This is done by setting a threshold (e.g., ...). This process filters out word pairs with strong semantic connections, which are then used as input for subsequent topic models, thereby improving the accuracy and semantic consistency of topic extraction.

[0066] In one embodiment of the invention, the calculation of NPMI depends on the accuracy of word frequency statistics, and a sliding window mechanism (e.g., a window size of 5) is typically used to capture contextual co-occurrence relationships. Furthermore, to avoid interference from low-frequency words in the results, a minimum number of co-occurrences can be set (e.g., ...). () as a filtering condition.

[0067] This step plays a crucial role in this application, and its technical effect is as follows: by quantifying the semantic association between words through NPMI, it provides high-quality co-occurrence features for subsequent topic models, thereby enhancing the ability to identify potential topics in user dialogue texts, providing a solid foundation for the construction of observation state sequences in Hidden Markov Models, and ultimately improving the accuracy and robustness of intent recognition.

[0068] S3. Based on the Bayesian information criterion, determine the number of hidden states in the Hidden Markov Model, input the observed state sequence into the Hidden Markov Model, and generate the hidden state transition probability matrix and the observation probability matrix through multiple rounds of iterative training.

[0069] Specifically, in step S31, this application determines the number of hidden states in a Hidden Markov Model (HMM) using the Bayesian Information Criterion (BIC), which is a crucial step in model structure design. BIC is a statistical model selection criterion whose core idea is to introduce a penalty term for model complexity into the log-likelihood function, thereby achieving a balance between model fit and the number of parameters and avoiding overfitting.

[0070] In this embodiment, the formula for calculating BIC is:

[0071] in, This represents the total number of estimable parameters in the model. This indicates the number of observed samples (i.e., the total number of dialogue texts). This represents the maximum likelihood function value of the model under the current parameters. In a Hidden Markov Model, Typically related to the number of hidden states and the number of observed states Related.

[0072] In this embodiment, the present application enumerates different numbers of hidden states. Each HMM model is constructed separately, and its BIC value is calculated. The model with the largest BIC value is selected as the final model structure, thereby determining the optimal number of hidden states. This process is typically performed independently in subgroups after user feature clustering to accommodate the intent recognition needs of different user categories.

[0073] Furthermore, this step is particularly crucial in scenarios with low data volume but significant sequence characteristics. By employing the BIC criterion, the most suitable model complexity can be selected under limited sample conditions, thereby improving the model's generalization ability and the accuracy of intent recognition. This method avoids the subjectivity of traditional empirical setting of the number of hidden states, enhancing the model's adaptability and scientific rigor, and is a significant innovation of this application in terms of structural optimization and improved intent recognition accuracy.

[0074] Furthermore, S3 includes: S31, calculate the model evaluation value for different numbers of hidden states using the Bayesian Information Criterion (BIC) formula. The specific calculation formula is as follows:

[0075] in This refers to the number of model parameters (i.e., the number of topics). The total number of dialogue texts. This represents the model likelihood function value.

[0076] Specifically, in step S31, this application evaluates and selects the number of hidden states in the Hidden Markov Model using the Bayesian Information Criterion (BIC) formula, thereby achieving an optimal balance between model complexity and fitting ability. BIC is a criterion based on statistical model selection. Its core idea is to prevent overfitting by introducing a penalty term for the number of model parameters, which is particularly suitable for model optimization in scenarios with small samples or low data volume.

[0077] In this embodiment, the formula for calculating BIC is:

[0078] in, This indicates the number of parameters to be estimated in the model. Represents the total number of samples. This represents the maximum likelihood function value of the model under the current parameters. In this application, This corresponds to the number of hidden states in a Hidden Markov Model, while This represents the total number of user dialogue texts. By enumerating different numbers of hidden states, the corresponding BIC value is calculated, and the number of hidden states with the largest BIC value is selected as the final number of hidden states in the model.

[0079] Furthermore, the penalty items in BIC It is positively correlated with model complexity, while This reflects the model's fit to the data. In scenarios with low data volume, this penalty term can effectively suppress the risk of overfitting caused by too many model parameters, thereby improving the model's generalization ability. Optionally, the calculation of BIC can be combined with the EM algorithm to iteratively optimize the model parameters to obtain a more accurate likelihood estimate.

[0080] In practical applications, this step is typically deployed during the model initialization phase of a user intent recognition system. After user feature clustering is completed, the BIC value corresponding to the dialogue text sequence is calculated for each user group to determine the most suitable number of hidden states for that user group. For example, in a cluster where the user interest category is "education," 3-5 hidden states may be needed to capture the transition relationships of intents such as "course consultation," "exam arrangement," and "learning feedback."

[0081] Through this step, this application achieves adaptive selection of the number of hidden states, improves the modeling accuracy of the hidden Markov model in different user categories, provides a more reasonable state space structure for subsequent intent recognition, and thus enhances the system's recognition capability and stability in scenarios with low data volume and high sequence dependency.

[0082] S4. Based on the hidden state transition probability matrix and the observation probability matrix, and combined with the topic sequence information, identify the user's potential intentions and generate an intention transition probability matrix corresponding to the user feature clustering.

[0083] Specifically, this step aims to identify potential user intentions by combining the hidden state transition probability matrix and the observation probability matrix, and by incorporating user feature clustering information, ultimately generating an intention transition probability matrix corresponding to the user feature clusters. This process is based on the dual Markov property of Hidden Markov Models (HMMs), namely the probabilistic relationship between the hidden state sequence and the observed state sequence, and models user intentions through unsupervised learning.

[0084] In one embodiment of the present invention, the topic sequence extracted in S2 is first input into the HMM model as the observation state sequence. The hidden states represent the user's potential intention states, such as "purchase consultation," "technical support," and "complaint feedback." The observation states are composed of topic labels output by the topic model, with each topic label corresponding to a set of high-frequency semantic words. The HMM model constructs a dynamic evolution model of user intention by learning the emission probabilities (observation probability matrix) between the observation states and hidden states, as well as the transition probabilities (hidden state transition probability matrix) between hidden states. In some implementations, the number of hidden states is determined using the Bayesian Information Criterion (BIC), with the following formula:

[0085] in, This represents the number of model parameters (i.e., the number of hidden states). This represents the total number of samples (number of dialogue texts). This represents the log-likelihood value of the model. A larger BIC value indicates a better model fit, thus allowing determination of the optimal number of hidden states.

[0086] Furthermore, based on user feature clustering, each cluster category will have independent HMM model parameters. For example, if users are clustered into... If there are 10 categories, then each category will generate an independent set of hidden state transition probability matrices. and observation probability matrix ,in In this way, the model can capture the differentiated behavioral patterns of different user groups in shifting their intentions during conversations.

[0087] In this embodiment, this step is applicable to scenarios such as customer service dialogue systems and intelligent assistants that require dynamic recognition of user intent. It is particularly effective in situations where data volume is limited but dialogue exhibits clear temporal characteristics, significantly improving the accuracy of intent recognition. By guiding intent modeling through clustering, the system can provide more personalized intent response strategies for different user groups.

[0088] The technical effect of this step is that by introducing user feature clustering information, the context awareness capability of intent recognition is enhanced, enabling the model to more accurately capture the evolution path of user intent, thereby improving the robustness and adaptability of intent recognition.

[0089] The intent recognition method based on user clustering and topic sequence in this invention can combine user feature data with the topic sequence of dialogue text and achieve efficient intent recognition in low-data-volume scenarios through a hidden Markov model, thereby improving the accuracy and adaptability of intent judgment.

[0090] Furthermore, S4 includes: S41, based on the topic-hidden state correspondence in the observation probability matrix, the hidden state is mapped to the intent using the maximum probability matching rule. The specific calculation formula is as follows:

[0091] in For the first Sentence text in theme The probability of it, Hidden state To the topic The probability of emission.

[0092] Specifically, in this application, the step of "mapping the hidden states to intents using the maximum probability matching rule based on the topic-hidden state correspondence in the observation probability matrix" is a key step in realizing the identification of potential user intents. Technically, this step relies on the pre-trained observation probability matrix in a Hidden Markov Model (HMM) and uses the maximum probability matching rule (Viterbi algorithm or maximum likelihood estimation) to map each hidden state to a specific intent category, thereby achieving the conversion from serialized topics to user intents.

[0093] Specifically, the observation probability matrix Each hidden state is described With observation status The probability relationship between (i.e., topics) is in the form of: ,in Represents hidden state Generate observation state The probability of [o_1, o_2, ..., o_T]. In this step, the system first constructs an observation state sequence O = [o_1, o_2, ..., o_T] based on the topic sequence output by the topic extraction module, where [o_1, o_2, ..., o_T]. The length of the dialogue text. Then, through the observation probability matrix... For each hidden state The observed states are probabilistically matched, and the topic with the highest probability is selected as the observation input for that hidden state.

[0094] In this embodiment, this step involves the mapping relationship between hidden states and intent categories, typically employing a one-to-one or one-to-many mapping strategy. In a one-to-one mapping, each hidden state corresponds to a specific intent category, and the mapping rule is as follows:

[0095] in Indicates the first The intent category corresponding to each hidden state. Represents hidden state In observation state The posterior probability under this condition. In a one-to-many mapping, a threshold can be introduced. ,when Intent mapping is only performed at this time to avoid misjudgments due to low confidence.

[0096] In one embodiment of the present invention, this step is applicable to the dynamic recognition of user intent in multi-turn dialogue systems, particularly in scenarios with low data volume and high sequence dependency, such as customer service dialogues, smart home voice interaction, and educational question-and-answer systems. By combining user clustering information, the system can generate differentiated intent recognition results for different user groups (such as young users, elderly users, and users with specific interests), thereby improving the accuracy of personalized services.

[0097] The technical advantage of this step lies in its ability to effectively improve the accuracy and robustness of intent recognition by precisely mapping latent states to intents. Compared to existing technologies that rely solely on text labels or explicit states for intent determination, this method utilizes the contextual information of latent states and the observation probability matrix to more comprehensively capture the evolutionary path of user intents. It exhibits stronger adaptability and recognition capabilities, especially in situations with long dialogue sequences and frequent intent changes.

[0098] The intent recognition method based on user clustering and topic sequence in this invention can combine user feature data with the topic sequence of dialogue text and achieve efficient intent recognition in low-data-volume scenarios through a hidden Markov model, thereby improving the accuracy and adaptability of intent judgment.

[0099] S5 dynamically updates the generated intent transfer probability matrix, adjusts the clustering results according to the real-time changes in user feature data, and retrains the Hidden Markov Model based on the updated clustering labels to adapt to the evolution of user group characteristics.

[0100] Specifically, this step involves dynamically updating the intent transition probability matrix in the Hidden Markov Model (HMM) to adapt to the evolution of user group characteristics over time. In some implementations, this process first performs unsupervised clustering based on user feature data (such as age, gender, interest tags, etc.), dividing users into several categories, each representing a group of users with similar behaviors or semantic preferences. The clustering results serve as prior information for user intent recognition, guiding subsequent intent modeling.

[0101] In this embodiment, the clustering process typically employs the k-means algorithm, and the optimal number of clusters is determined using the elbow method. After clustering is complete, the system continuously receives real-time updates of user feature data, such as changes in user interest tags and migrations of historical conversational behavior. When the distribution of user feature data changes significantly, the system will recalculate cluster centers and update user cluster labels. This process can be controlled by setting a cluster update threshold (e.g., cluster center drift exceeding a certain threshold). Or the cluster profile coefficient decreases by more than This is triggered to ensure the timeliness and accuracy of clustering results.

[0102] In one embodiment of the invention, the system associates the updated clustering labels with the topic sequence of the dialogue text, thus reclassifying the category distribution of the observation sequence. Subsequently, based on the new observation sequence, the system retrains the HMM model, updating the transition probability matrix between its hidden states. And the emission probability matrix between the hidden state and the observed state. Specifically, the transition probability matrix The update uses the Expectation-Maximization (EM) algorithm or the Baum-Welch algorithm to iteratively optimize the model parameters, enabling the model to more accurately reflect the evolution path of the current user group's intent.

[0103] In this embodiment, this step is particularly suitable for scenarios with low data volume but obvious sequence characteristics, such as customer service dialogues, intelligent question-and-answer systems, and personalized recommendations. By dynamically updating the HMM model, the system can adapt to the shift in user behavior patterns, improving the accuracy and response speed of intent recognition.

[0104] The technical effect of this step is that by combining the dynamic changes of user feature data with the semantic evolution of dialogue text, the intent recognition model can be adaptively updated, thereby improving the robustness and generalization ability of the model in non-static user environments.

[0105] The intent recognition method based on user clustering and topic sequence in this invention improves the adaptability of the Hidden Markov Model to the evolution of user group characteristics by dynamically updating the intent transition probability matrix and adjusting the clustering results according to the real-time changes in user feature data, thereby maintaining high accuracy and stability of intent recognition in continuously changing interaction scenarios.

[0106] Example 2 This invention relates to another intent recognition method based on user clustering and topic sequences, such as... Figure 2 As shown, the method includes the following steps: Step 1: User clustering processing, including: Step 11: Data preprocessing.

[0107] Collect users' historical characteristics data, including age, gender, interests and hobbies, and standardize the data to ensure that features at different scales have the same impact on the clustering results.

[0108] Step 12: Clustering.

[0109] The elbow rule was used to plot the inner sum of squares curves for different cluster values ​​to determine the optimal number of clusters, and k-means was used to cluster the users.

[0110] Step 2: Extraction of topic information from dialogue text includes: For conversation texts from different user categories, perform the following operations: Step 21: Topic extraction and classification.

[0111] The optimal number of topics is determined using standardized point-to-point mutual information. Standardized point-to-point mutual information is developed based on the Point-to-Point Mutual Information (PMI) model and is a type of information entropy. It can determine the consistency between words based on their density. The specific calculation formula is as follows: =

[0112]

[0113] The topic count is input into the topic model, and unsupervised information extraction is performed on the dialogue text to obtain the corresponding topic sequences. Each topic sequence contains multiple words distributed according to their probability. Based on the characteristics of the topic words under each topic, they are categorized, such as collaboration-related topics.

[0114] Step 22: Data integration.

[0115] The topic model is used to classify each sentence in the dialogue text into different topics. Combined with the topic classification obtained earlier, the topic classification corresponding to each dialogue text is obtained.

[0116] S3: Hidden state extraction includes: Step 31: Determine the number of hidden states.

[0117] The number of hidden states is determined using the Bayesian Information Criterion (BIC). BIC balances model complexity and accuracy by adding a penalty term; its specific calculation formula is as follows:

[0118] Where k is the number of model parameters, i.e. the number of topics, n is the total number of samples, i.e. the number of dialogue texts, and L is the likelihood function value. The larger the BIC value, the better the model performance.

[0119] Step 32: Hidden state transition probability matrix.

[0120] The previously obtained topic sequenced data is input into the Hidden Markov Model as the initial observation state sequence, and the initial probability distribution vector and state transition matrix are randomly initialized. After multiple iterations, the model reaches stationarity and generates the hidden state probability transition matrix, as well as the observation probability matrices corresponding to the topic and the hidden state.

[0121] Step 4: Identifying potential user intent, including: Based on the observation probability matrix and combined with topic information, the latent states are classified into different intentions, thus obtaining the intention transition probability matrix, which is used to determine the potential intentions of different categories of users in a certain dialogue.

[0122] This invention improves the Hidden Markov Model's adaptability to the evolution of user group characteristics by dynamically updating the intent transfer probability matrix and adjusting the clustering results according to real-time changes in user feature data, thereby maintaining high accuracy and stability of intent recognition in continuously changing interaction scenarios.

[0123] Example 3 To achieve the above embodiments, the present invention also proposes an intent recognition device based on user clustering and topic sequence. Figure 3 This is a schematic diagram of an intent recognition device based on user clustering and topic sequence, provided as an embodiment of the present invention. Figure 3 As shown, the device includes: The user feature standardization and clustering module 100 is used to acquire user feature data and perform standardization processing. It determines the optimal number of clusters based on the elbow rule, uses the k-means algorithm to perform unsupervised clustering of users, and generates user feature clustering labels. The dialogue text topic integration module 200 is used to extract topics from the dialogue text, determine the optimal number of topics based on standardized point mutual information, classify the dialogue text sentence by sentence into the corresponding topic sequence, and integrate and generate an observation state sequence with user features by combining user feature clustering labels. The Hidden Markov Model Training Module 300 is used to determine the number of hidden states of the Hidden Markov Model based on the Bayesian information criterion, input the observed state sequence into the Hidden Markov Model, and generate the hidden state transition probability matrix and the observation probability matrix through multiple rounds of iterative training. The intent recognition and probability matrix generation module 400 is used to identify the user's potential intent based on the hidden state transition probability matrix and the observation probability matrix, combined with topic sequence information, and generate an intent transition probability matrix corresponding to the user feature clustering.

[0124] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0125] Example 4 To implement the above embodiments, the present invention also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0126] Example 5 To implement the above embodiments, the present invention also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0127] Example 6 To implement the above embodiments, the present invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0128] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0129] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0130] This invention is intended to provide implementation schemes for users to selectively prevent the use or access to personal information data. That is, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information can be de-identified to protect user privacy.

[0131] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0132] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0133] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0134] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0135] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0136] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0137] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0138] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An intent recognition method based on user clustering and topic sequences, characterized in that, include: S1. Obtain user feature data and perform standardization processing. Determine the optimal number of clusters based on the elbow rule. Use the k-means algorithm to perform unsupervised clustering of users and generate user feature clustering labels. S2, extract topics from the dialogue text, determine the optimal number of topics based on standardized point mutual information, classify the dialogue text sentence by sentence into the corresponding topic sequence, and integrate with user feature clustering labels to generate observation state sequence with user features; S3. Determine the number of hidden states of the Hidden Markov Model based on the Bayesian information criterion, input the observed state sequence into the Hidden Markov Model, and generate the hidden state transition probability matrix and the observation probability matrix through multiple rounds of iterative training. S4. Based on the hidden state transition probability matrix and the observation probability matrix, and combined with the topic sequence information, identify the user's potential intentions and generate an intention transition probability matrix corresponding to the user feature clustering.

2. The method as described in claim 1, characterized in that, The process of acquiring user feature data and standardizing it, determining the optimal number of clusters based on the elbow rule, performing unsupervised clustering of users using the k-means algorithm, and generating user feature clustering labels also includes: S11, uses the minimum-maximum normalization method to standardize user feature data, mapping features of different scales to the [0,1] interval. The specific calculation formula is as follows: S12, calculate the inner sum of squares (SSE) for different cluster numbers using the elbow rule, plot the SSE-cluster number curve, and select the inflection point as the optimal cluster number. The specific calculation formula is as follows: in For the first Clusters, For the first The centroid of a cluster.

3. The method as described in claim 1, characterized in that, The process of extracting topics from the dialogue text, determining the optimal number of topics based on standardized point mutual information, classifying the dialogue text sentence by sentence into corresponding topic sequences, and integrating user feature clustering labels to generate observation state sequences with user features also includes: S21, use the Normalized Point Mutual Information (NPMI) formula to calculate word co-occurrence relations. The specific calculation formula is as follows: in For point-to-point information, For words and The joint probability.

4. The method as described in claim 1, characterized in that, The process of determining the number of hidden states in the Hidden Markov Model based on the Bayesian information criterion, inputting the observed state sequence into the Hidden Markov Model, and generating the hidden state transition probability matrix and observation probability matrix through multiple rounds of iterative training further includes: S31, calculate the model evaluation value for different numbers of hidden states using the Bayesian Information Criterion (BIC) formula. The specific calculation formula is as follows: in This refers to the number of model parameters, i.e., the number of topics. The total number of dialogue texts. This represents the model likelihood function value.

5. The method as described in claim 1, characterized in that, The step of identifying the user's latent intent based on the hidden state transition probability matrix and the observation probability matrix, combined with topic sequence information, and generating an intent transition probability matrix corresponding to the user feature clustering further includes: S41, based on the topic-hidden state correspondence in the observation probability matrix, the hidden state is mapped to the intent using the maximum probability matching rule. The specific calculation formula is as follows: in For the first Sentence text in theme The probability of it, Hidden state To the topic The probability of emission.

6. The method as described in claim 1, characterized in that, Also includes: S5 dynamically updates the generated intent transition probability matrix, adjusts the clustering results based on real-time changes in user feature data, and retrains the Hidden Markov Model based on the updated clustering labels to adapt to the evolution of user group characteristics.

7. An intent recognition device based on user clustering and topic sequence, characterized in that, include: The user feature standardization and clustering module is used to acquire user feature data and perform standardization processing. It determines the optimal number of clusters based on the elbow rule, uses the k-means algorithm to perform unsupervised clustering of users, and generates user feature clustering labels. The dialogue text topic integration module is used to extract topics from the dialogue text, determine the optimal number of topics based on standardized point mutual information, classify the dialogue text sentence by sentence into the corresponding topic sequence, and integrate the user feature clustering labels to generate an observation state sequence with user features. The Hidden Markov Model Training Module is used to determine the number of hidden states of the Hidden Markov Model based on the Bayesian information criterion, input the observed state sequence into the Hidden Markov Model, and generate the hidden state transition probability matrix and the observation probability matrix through multiple rounds of iterative training. The intent recognition and probability matrix generation module is used to identify the user's potential intent based on the hidden state transition probability matrix and the observation probability matrix, combined with topic sequence information, and generate an intent transition probability matrix corresponding to the user feature clustering.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images