Dialogue knowledge base adaptive construction method based on Bayesian nonparametric model

Abstract

The invention discloses a dialogue knowledge base adaptive construction method based on a Bayesian nonparametric model, and belongs to the field of data processing and machine learning. The method comprises the following steps: continuously obtaining customer dialogue text data and carrying out feature extraction to generate a text feature vector for clustering; inputting the text feature vectors into a Bayesian nonparametric model for adaptive clustering, dynamically determining a clustering number according to data distribution, and forming an initial clustering cluster; performing incremental clustering processing on the text data in the initial clustering cluster, and locally updating model parameters based on a newly input dialogue text feature vector to generate a dynamically adjusted clustering result; and extracting descriptive theme keywords from the clustering result by adopting a keyword extraction algorithm, generating an automatic tag corresponding to a cluster, and constructing a dialogue knowledge base. According to the method, self-adaption and dynamic construction of the dialogue knowledge base are realized, and the accuracy, the real-time performance and the expandability of the knowledge base are improved.

Description

Technical Field

[0001] This invention relates to the fields of data processing and machine learning, and in particular to an adaptive construction method, apparatus, device, and storage medium for a dialogue knowledge base based on a Bayesian nonparametric model. Background Technology

[0002] In fields such as customer service, knowledge management, and intelligent dialogue systems, building a structured knowledge base of dialogue or question-and-answer scripts is crucial. Traditional methods mainly rely on manual classification and organization of massive amounts of dialogue text. This approach not only consumes a lot of human resources and time, but also struggles to adapt to the rapid growth of business data volume and the dynamic changes in market demands.

[0003] To improve build efficiency, the industry has begun to adopt automation technologies. Existing automated build technologies typically involve two core steps: data classification and data labeling. However, both of these technologies have significant drawbacks.

[0004] First, existing clustering algorithms have inherent limitations in the data classification stage. For example, the widely used K-means algorithm and some incremental clustering methods based on centroid vectors both require manually pre-setting the number of clusters before the model runs. The K-means algorithm requires explicitly specifying the number of clusters K, while related incremental clustering methods require pre-setting the number of centroids and their initial vectors. However, in real-world business scenarios, especially in dynamic environments where the business scope is constantly evolving and the topics of conversation are becoming increasingly diverse, the range and number of topics covered by the conversation data are constantly changing and difficult to accurately estimate in advance. This strong dependence on a fixed number of clusters results in existing methods lacking the necessary flexibility to adaptively adjust the cluster size according to the actual distribution characteristics of the input data. Consequently, they struggle to cope with the continuous influx of new conversation data and emerging topics, leading to poor clustering results and limited coverage and timeliness of the knowledge base.

[0005] Secondly, in the data labeling stage, current technologies have a low degree of automation and heavily rely on manual intervention. Currently, the generation of tags for various question-and-answer texts in the discourse knowledge base is typically done by experts based on experience before the clustering process, or by manual labeling after the clustering results are produced, involving reviewing and summarizing the content of each cluster. This approach severs the inherent connection between data classification and tag generation, failing to create an end-to-end automated process. It not only introduces subjective bias and additional manpower costs but also delays the knowledge base construction process, making knowledge base updates and maintenance costly and hindering true real-time adaptability.

[0006] In summary, the shortcomings of existing automated construction technologies in dynamic data classification and automated tag generation have become bottlenecks restricting the intelligent development and efficient application of discourse knowledge bases. Summary of the Invention

[0007] The present invention aims to at least partially solve one of the technical problems in the related art.

[0008] To address this, this invention discloses an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model. This method continuously acquires customer dialogue text data and extracts features to generate text feature vectors. It then uses a Bayesian nonparametric model to adaptively cluster the feature vectors, dynamically determining the number of clusters based on data distribution. Furthermore, it performs incremental learning and local parameter updates on the clusters to dynamically optimize the clustering results. Finally, it employs keyword extraction technology to automatically generate cluster topic tags, thereby completing the adaptive construction of the dialogue knowledge base and effectively improving its automation level and dynamic adaptability.

[0009] Another objective of this invention is to propose an adaptive construction device for dialogue knowledge base based on a Bayesian nonparametric model.

[0010] The third objective of this invention is to provide a computer device.

[0011] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0012] To achieve the above objectives, this invention proposes an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model, comprising: S1, continuously acquire customer conversation text data and extract features to generate text feature vectors for clustering; S2, input the text feature vector into a Bayesian nonparametric model for adaptive clustering, dynamically determine the number of clusters based on the data distribution, and form an initial cluster; S3 performs incremental clustering on the text data in the initial clusters, and locally updates the model parameters based on the new input dialogue text feature vectors to generate dynamically adjusted clustering results; S4. A keyword extraction algorithm is used to extract descriptive topic keywords from the clustering results, generate automatic labels for the corresponding clusters, and construct a dialogue knowledge base.

[0013] An adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model according to an embodiment of the present invention may also have the following additional technical features: In one embodiment of the present invention, the step of continuously acquiring customer dialogue text data and extracting features to generate text feature vectors for clustering includes: S11 uses the bag-of-words model and TF-IDF algorithm to extract statistical features from the text and generate a numerical feature vector of the text. S12 uses a BERT pre-trained language model to embed semantic features into the text, generating a high-dimensional feature vector containing contextual semantic information.

[0014] In one embodiment of the present invention, the step of inputting the text feature vector into a Bayesian nonparametric model for adaptive clustering, dynamically determining the number of clusters based on the data distribution, and forming initial clusters includes: S21 uses the DP-GMM model, by setting prior distribution parameters. and basis distribution The optimal number of clusters is dynamically determined based on the distribution characteristics of the text feature vectors. S22, parameter estimation of the DP-GMM model is performed based on the variational inference method, and model convergence is achieved by maximizing the lower bound of evidence.

[0015] In one embodiment of the present invention, the step of performing incremental clustering processing on the text data in the initial cluster, and locally updating the model parameters based on the new input dialogue text feature vector to generate dynamically adjusted clustering results, includes: S31, when the similarity threshold between the newly input text feature vector and the existing clusters... satisfy When the time comes, update the parameters of the corresponding cluster; S32, when the similarity between the newly input text feature vector and all clusters is less than... When needed, create new clusters and initialize the corresponding parameters.

[0016] In one embodiment of the present invention, the step of extracting descriptive topic keywords from the clustering results using a keyword extraction algorithm, generating automatic labels for corresponding clusters, and constructing a dialogue knowledge base includes: S41, using the KeyBERT algorithm, calculates the semantic similarity between keywords and text based on the embedding vectors of the BERT model, and selects the top keywords with the highest similarity. Use one keyword as a tag; S42 uses the TextRank algorithm to model the graph structure of text in clusters and generates topic keyword tags by iteratively calculating node weights.

[0017] In one embodiment of the present invention, it further includes: S5, based on the change in the number of clusters in the dynamically adjusted clustering results. ,when Exceeding the preset threshold When this happens, the structured update process of the knowledge base is triggered, the topic keyword tags of all clusters are recalculated and the knowledge base index is updated synchronously.

[0018] To achieve the above objectives, another aspect of the present invention proposes an adaptive construction device for a dialogue knowledge base based on a Bayesian nonparametric model, comprising: The data acquisition and feature extraction module is used to continuously acquire customer dialogue text data and extract features to generate text feature vectors for clustering. The Bayesian nonparametric adaptive clustering module is used to input the text feature vector into the Bayesian nonparametric model for adaptive clustering, dynamically determine the number of clusters based on the data distribution, and form initial clusters. The incremental clustering parameter update module is used to perform incremental clustering processing on the text data in the initial clusters, and locally update the model parameters based on the feature vector of the new input dialogue text to generate dynamically adjusted clustering results. The keyword extraction and knowledge base construction module is used to extract descriptive topic keywords from the clustering results using a keyword extraction algorithm, generate automatic labels for the corresponding clusters, and construct a dialogue knowledge base.

[0019] In one embodiment of the present invention, it further includes: The knowledge base structure update module is used to update the number of clusters in the dynamically adjusted clustering results. ,when Exceeding the preset threshold When this happens, the structured update process of the knowledge base is triggered, the topic keyword tags of all clusters are recalculated and the knowledge base index is updated synchronously.

[0020] This invention discloses an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model. By introducing a Bayesian nonparametric model, the method dynamically determines and adaptively adjusts the number of clusters, effectively overcoming the limitation of pre-setting the number of clusters in existing technologies. It achieves full automation from feature extraction, dynamic clustering, incremental updates to automatic tag generation, significantly improving the adaptive capability and construction efficiency of the dialogue knowledge base in dynamic business scenarios. Simultaneously, it ensures the flexibility of the knowledge base structure and the accuracy of topic descriptions, enhancing the knowledge management system's ability to process continuously growing and changing dialogue data and its practical value.

[0021] To achieve the above objectives, a third aspect of this application provides a computer device, including a processor and a memory; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, for implementing an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model as described in the first aspect embodiment.

[0022] To achieve the above objectives, a fourth aspect of this application proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model as described in the first aspect embodiment.

[0023] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the specific steps of an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of an adaptive construction device for a dialogue knowledge base based on a Bayesian nonparametric model according to an embodiment of the present invention. Figure 4 It is a computer device according to an embodiment of the present invention. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] The following description, with reference to the accompanying drawings, describes an adaptive construction method, apparatus, device, and storage medium for a dialogue knowledge base based on a Bayesian nonparametric model, according to an embodiment of the present invention.

[0028] The core idea of ​​this invention is to construct a dynamic dialogue knowledge base construction system that can adapt to data distribution by introducing a Bayesian nonparametric model. First, multi-level feature extraction is performed on continuously acquired customer dialogue text to generate text feature vectors containing statistical properties and deep semantics. Then, utilizing the adaptive clustering capability of the Bayesian nonparametric model, the optimal number of clusters is dynamically determined based on the inherent distribution of the data, forming an initial cluster structure. Through an incremental learning mechanism, the system can locally update model parameters based on newly added dialogue feature vectors, achieving dynamic optimization and expansion of the clustering results. Finally, multiple keyword extraction algorithms are integrated to automatically generate descriptive topic tags from multiple dimensions such as semantic similarity and graph structure weights. This technical approach transforms the traditional knowledge base generation method, which relies on preset parameters and static construction, into an intelligent construction closed loop that can autonomously perceive data changes and continuously evolve and update, significantly improving the adaptability, accuracy, and maintainability of the dialogue knowledge base in dynamic business scenarios.

[0029] Example 1 To achieve the above invention, embodiments of the present invention provide an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model, such as... Figure 1 As shown, it includes: S1 continuously acquires customer conversation text data and extracts features to generate text feature vectors for clustering.

[0030] Specifically, the core objective of this step is to transform the original customer dialogue text into numerical feature vectors that can be used for subsequent Bayesian nonparametric model clustering, thereby achieving a structured representation and semantic modeling of the text content.

[0031] Specifically, this step begins by continuously acquiring customer conversation text data from multiple business channels (such as customer service systems, sales platforms, and customer communication records) through a data acquisition module. This text data is typically in unstructured form and requires preprocessing, including natural language processing (NLP) techniques such as word segmentation, stop word filtering, stemming, or lemmatization to remove noise and standardize text representation. Subsequently, word embedding models (such as Word2Vec, GloVe, or BERT) are used to semantically encode the text, representing each text as a high-dimensional vector. Further, optionally, TF-IDF weighting or attention mechanisms are used to weight and combine the word vectors to generate a global feature vector representation of the text.

[0032] Furthermore, the dimension of the text feature vector is usually set between 256 and 768, specifically depending on the word embedding model adopted. For example, the text vector output by the BERT-base model has a dimension of 768, while the Word2Vec model can be set to 300 dimensions. In addition, a sliding window mechanism can be introduced during the feature extraction process, with the window size set to 5 to 10 words to capture context semantic information. After the feature vector is generated, normalization is usually performed to unify the vector length, facilitating the calculation of similarity by subsequent clustering algorithms. The commonly used normalization method is L2 normalization, that is, for the vector perform processing to make it satisfy .

[0033] Specifically, this step is widely applied in scenarios such as customer service, sales support, and product consultation in the insurance industry. For example, when a customer consults the auto insurance terms, the system can obtain the conversation text in real time, extract its semantic feature vector, and input it into the subsequent clustering model to identify whether the conversation belongs to categories such as "claim settlement process", "premium calculation", or "scope of liability". This process supports online incremental processing, that is, based on the existing knowledge base, continuously receiving new conversation data and performing feature extraction, so as to achieve the dynamic update of the knowledge base.

[0034] Specifically, this step provides high-quality input data for the adaptive clustering of the subsequent Bayesian nonparametric model by converting unstructured text into structured feature vectors. Its key value lies in enhancing the semantic perception ability of the clustering algorithm, avoiding the limitation of presetting the number of clusters in traditional methods such as K-Means, and at the same time, through the normalization processing of the feature vector, enhancing the robustness and generalization ability of the model, laying a solid foundation for constructing a dynamic and adaptive conversation knowledge base.

[0035] Furthermore, S1 includes: S11, using the bag-of-words model and TF-IDF algorithm to extract statistical features from the text and generate a numerical feature vector of the text.

[0036] Specifically, this step is a key preprocessing link in the entire automatic construction process of the conversation knowledge base. Its technical implementation is based on the text vectorization method in natural language processing, aiming to convert unstructured text data into a structured vector representation that can be used for subsequent clustering and topic summarization.

[0037] Specifically, the bag-of-words model first performs word segmentation on the text to construct a dictionary consisting of all unique words. Each text is represented as a vector, whose dimension is equal to the size of the dictionary, and each element in the vector represents the frequency of the corresponding word in the text. Furthermore, to reduce the interference of high-frequency common words (such as "of", "is", etc.) on the clustering result, the TF-IDF algorithm is used to weight the word frequency. The calculation formula of TF-IDF is: ; in, To indicate a certain word, Indicates the current document. This represents the total number of documents. Indicates the inclusion word The number of documents. TF-IDF captures the thematic features of a text more effectively by multiplying the term frequency (TF) by the logarithm of the document frequency (DF), giving higher weights to words that appear frequently in the current document but are rare in the entire corpus.

[0038] Furthermore, during dictionary construction, a maximum vocabulary size (e.g., 10,000) and a minimum word frequency threshold (e.g., 5 times) can be set to filter low-frequency noise words. In the TF-IDF model, smoothing (e.g., adding 1 for smoothing) is typically used to avoid zero-value problems, and a stopword list can be used to remove meaningless words. Additionally, an n-gram model (e.g., bigram or trigram) can be optionally used to preserve some contextual information and improve semantic expressiveness.

[0039] Specifically, this step is applicable to the preprocessing stage of large amounts of customer dialogue text in insurance business, such as text data in scenarios like customer consultation, claims communication, and product recommendation. By converting the dialogue text into TF-IDF feature vectors, high-quality input can be provided for subsequent Bayesian nonparametric model clustering, thereby achieving adaptive classification without pre-setting the number of clusters.

[0040] Specifically, this step effectively extracts the semantic features of the text, providing a measurable numerical representation for the clustering algorithm and improving the accuracy and interpretability of clustering. Simultaneously, the TF-IDF weighting mechanism enhances sensitivity to key business terms, facilitating the generation of topic summaries for subsequent clustering results, thereby enabling the automated construction and dynamic updating of the knowledge base.

[0041] S12 uses a BERT pre-trained language model to embed semantic features into the text, generating a high-dimensional feature vector containing contextual semantic information.

[0042] Specifically, the technical implementation of this step is based on a deep learning model with a Transformer architecture, which performs context-aware semantic encoding on the input text through a multi-layer self-attention mechanism and a feedforward neural network.

[0043] Furthermore, the dialogue text is first segmented and converted into the BERT input format, including token_ids, attention_mask, and token_type_ids. Subsequently, these inputs are fed into the BERT model, which outputs the hidden state vector for each token. Typically, the final hidden state of the [CLS] tag is taken as the semantic embedding vector for the entire sentence, with a dimension of 768 (for BERT-base) or 1024 (for BERT-large).

[0044] Furthermore, to enhance the robustness and generalization ability of semantic representations, multilingual BERT (mBERT) or domain-adapted BERT variants (such as FinBERT) can be optionally employed to adapt to the semantic feature extraction needs of different languages ​​or insurance business scenarios. In addition, fine-tuning can be used to further optimize the model on specific business corpora, thereby improving the discriminative ability of the embedded vectors in clustering tasks.

[0045] Specifically, it is typically deployed in the text preprocessing module of a unified intelligent platform to process customer dialogue text from insurance business scenarios, such as insurance consultations, claims communication, and product recommendations. Through semantic embedding, it can effectively capture the implicit intentions and contextual relationships in the dialogue, providing a solid foundation for subsequent adaptive clustering and label generation.

[0046] Specifically, this step achieves an efficient conversion from raw text to structured semantic vectors, significantly improving the accuracy and stability of the clustering algorithm. Through context-aware embedding, it effectively distinguishes semantically similar but differently intentional dialogue texts, thereby enhancing the intelligence and practicality of knowledge base construction.

[0047] S2, input the text feature vector into a Bayesian nonparametric model for adaptive clustering, dynamically determine the number of clusters based on the data distribution, and form an initial cluster.

[0048] Specifically, this step overcomes the need to pre-define the number of clusters required in traditional clustering methods (such as K-Means) by introducing a Bayesian nonparametric model. This overcomes the limitations of traditional methods, thereby enabling adaptive clustering of customer dialogue text.

[0049] Specifically, this step first takes the text feature vectors (such as TF-IDF vectors, word embedding vectors, or output vectors of pre-trained models like BERT) after text preprocessing and feature extraction as input and feeds them into a Bayesian nonparametric model. This model typically employs nonparametric Bayesian methods such as the Dirichlet Process Mixture Model (DPMM) or the Chinese Restaurant Process (CRP). Its core idea is to fit the data using an infinite mixture model of probability distributions, thereby automatically inferring the optimal number of clusters based on the data distribution without pre-setting the number of clusters. During model training, each text feature vector is assigned to a potential cluster, and the number of clusters dynamically expands as the data increases.

[0050] Furthermore, key parameters of the Bayesian nonparametric model include concentration parameters. It controls the probability of new cluster generation; and the base distribution. This is used to define the prior distribution of cluster centers. In practical applications, Typically set to The specific value is adjusted based on the sparsity of the dataset and the required clustering granularity. Furthermore, the convergence of the model can be assessed through the number of iterations. Controls are applied to posterior sampling methods (such as Gibbs sampling or variational inference) to ensure the stability and accuracy of clustering results.

[0051] Specifically, this step applies to the ever-growing customer conversation datasets in the insurance business, such as online or offline communication records between insurance agents and customers. Through adaptive clustering, the system can identify emerging business topics or customer question types in real time and automatically form initial clusters, providing a structured data foundation for subsequent label generation and knowledge base updates.

[0052] Specifically, it achieves adaptive adjustment of the number of clusters, avoiding the limitations of traditional methods that rely on pre-set parameters. This addresses the issue of decreased clustering quality due to unreasonable clustering methods. Simultaneously, its incremental clustering capability eliminates the need to retrain the entire model when faced with new data, significantly reducing computational resource consumption and improving the system's real-time responsiveness and scalability. Furthermore, this step provides high-quality initial clustering results for subsequent keyword extraction and tag generation, enhancing the automation and practicality of the entire knowledge base construction process.

[0053] Furthermore, S2 includes: S21 uses the DP-GMM model, by setting prior distribution parameters. and basis distribution The optimal number of clusters is dynamically determined based on the distribution characteristics of the text feature vectors.

[0054] Specifically, this step is the core component of the entire automatic construction method of the dialogue knowledge base. Its technical principle is based on the Bayesian nonparametric modeling idea, which can effectively address the limitation of traditional clustering methods that require pre-setting the number of clusters.

[0055] Specifically, DP-GMM is an infinite mixture model based on the Dirichlet Process. Its core idea is to generate potential clustering structures by introducing a stochastic process. In particular, the model assumes that the data follows a base distribution. and concentration parameters Defined random distribution ,Right now In this invention, It is usually set to a multivariate Gaussian distribution. , used to describe the prior distribution characteristics of text feature vectors. Text feature vectors are usually extracted using methods such as word embedding (e.g., Word2Vec, BERT) or TF-IDF to form a high-dimensional vector representation.

[0056] Furthermore, This is a key parameter controlling the number of clusters; the higher its value, the more clusters the model tends to generate. In practical applications, This can be adjusted based on the size and complexity of the dataset, and is typically set to... In addition, the basic distribution mean Covariance Initialization can also be performed based on domain knowledge. For example, in the insurance business scenario, it can be set based on the semantic distribution of common problems.

[0057] Specifically, this step is applicable to continuously growing customer dialogue text datasets, especially when business scenarios frequently change and new problems constantly emerge. DP-GMM can adaptively increase the number of clusters, thereby dynamically building and updating the knowledge base. This method eliminates the need for manual intervention in setting the number of categories, reducing maintenance costs and improving the system's scalability and robustness.

[0058] Furthermore, the technical benefits of this step are reflected in the following: through the DP-GMM model, the system can automatically identify the latent topic structure in text data and achieve adaptive adjustment of the number of clusters, thereby improving the accuracy and generalization ability of clustering. At the same time, this method supports incremental learning, allowing updates to only the local structure when new data arrives, significantly reducing the consumption of computational resources.

[0059] S22, parameter estimation of the DP-GMM model is performed based on the variational inference method, and model convergence is achieved by maximizing the lower bound of evidence.

[0060] Specifically, this step first transforms the customer dialogue text into a high-dimensional semantic vector representation using a word embedding model (such as BERT, Word2Vec, etc.), and then inputs these vectors into the DP-GMM model for clustering. The model approximates the posterior distribution through a variational inference method, which introduces a variational distribution. To approximate the true posterior The model parameters are estimated by optimizing the ELBO function. The ELBO function is defined as follows: ; The first term represents the expectation of the joint distribution, and the second term is the KL divergence, which measures the difference between the variational distribution and the true posterior. During the optimization process, the model iteratively updates the parameters using the EM algorithm or gradient ascent method to maximize the ELBO, thereby approximating the true posterior distribution and achieving convergence of the clustering results.

[0061] Specifically, the key parameters of the DP-GMM model include the concentration parameters of the Dirichlet process. This parameter controls the probability of generating new clusters, and is usually set to [value missing]. The specific value is adjusted based on the sparsity of the dataset. Furthermore, the covariance matrix of the Gaussian distribution... and mean vector It is also an optimization objective, and its updates depend on the current clustering assignment. and data points The statistical characteristics of the data are analyzed. In practical applications, this step is suitable for dynamic clustering scenarios of customer dialogue text, especially in fields such as insurance where the knowledge base needs to be continuously updated. Using this method, the system can automatically identify new dialogue topics and generate corresponding clusters without manual intervention, thereby achieving incremental construction and dynamic updates of the knowledge base.

[0062] Furthermore, the technical effect of this step is to significantly improve the flexibility and scalability of the clustering process, solve the technical bottleneck of needing to pre-determine the number of clusters in traditional clustering methods, reduce the cost of manual intervention, and improve the automation and adaptability of knowledge base construction.

[0063] S3 performs incremental clustering on the text data in the initial clusters, locally updating the model parameters based on the new input dialogue text feature vectors to generate dynamically adjusted clustering results.

[0064] Specifically, this step introduces an incremental learning mechanism, enabling the model to update only the local clustering structure when new input dialogue text feature vectors arrive, thereby avoiding the high time complexity and resource consumption caused by re-clustering the entire dataset.

[0065] Specifically, this incremental clustering process is implemented based on a Bayesian nonparametric model (such as the Chinese Restaurant Process, CRP, or Hierarchical Dirichlet Process, HDP). In the initial stage, the model has already clustered the historical dialogue text, generating several clusters. When new dialogue text feature vectors... Upon arrival, the model first calculates its similarity to the centers of existing clusters, typically using cosine similarity or Euclidean distance as the metric. If the similarity exceeds a preset threshold... If so, the text is assigned to the most similar cluster, and the cluster's center vector and intra-cluster statistics (such as the number of texts within the cluster) are updated. Mean of eigenvectors within a cluster (etc.); if the threshold is not exceeded, a new cluster is created, with the text feature vector as the initial center.

[0066] Specifically, this incremental clustering process involves several key parameters, including a similarity threshold. Learning rate updated at cluster centers and the minimum number of members in a cluster. .in, Typically set to to This ensures semantic consistency between the new text and existing clusters; Used to control the impact of new data on the cluster center, typically set to a value of [value to be filled in]. to ; To prevent clusters from becoming too sparse, it is usually set to or .

[0067] Furthermore, this step applies to the continuously growing customer dialogue datasets in insurance operations, such as customer service systems and intelligent question-and-answer platforms. By dynamically adjusting the clustering structure, the system can respond in real time to the emergence of new business scenarios, such as inquiries about new insurance products or changes in claims processes, thereby maintaining the timeliness and accuracy of the knowledge base.

[0068] Furthermore, the technical advantage of this incremental clustering process is that it significantly reduces the computational overhead and storage pressure on the model, while improving the adaptability and stability of the clustering results. By using partial updates instead of full re-clustering, the system can maintain efficient operation on large-scale, frequently updated dialogue data, providing a high-quality clustering foundation for subsequent keyword extraction and tag generation.

[0069] Furthermore, S3 includes: S31, when the similarity threshold between the newly input text feature vector and the existing clusters... satisfy When the time comes, update the parameters of the corresponding cluster.

[0070] Specifically, the technical implementation of this step is based on the adaptive clustering mechanism of the Bayesian nonparametric model, combined with the similarity measurement of text feature vectors, to ensure that the system can efficiently and accurately classify new data into existing categories without pre-setting the number of clusters, and update parameters when necessary.

[0071] Specifically, this step first converts the newly input dialogue text into a high-dimensional feature vector using a text embedding model (such as BERT, Sentence-BERT, etc.). ,in This represents the embedding dimension. Subsequently, the system calculates the relationship between this vector and the center vectors of all existing clusters. Similarity between Cosine similarity or Euclidean distance is typically used as a metric. If a cluster exists... satisfy If the text belongs to the cluster, then only the parameters of the cluster (such as cluster center, number of members, keyword distribution, etc.) are updated, rather than recalculating the global clustering structure, thus significantly reducing computational complexity.

[0072] Furthermore, similarity threshold This is a key hyperparameter, typically ranging from [0, 1], with the specific setting depending on the semantic density of the text and the required granularity of clustering. In practical applications, The results can be dynamically adjusted through cross-validation or semantic similarity assessment based on business scenarios. For example, in an insurance business scenario, if high semantic consistency of the clustering results is required, then... Can be set to Or higher.

[0073] Specifically, this step is applicable to business systems that continuously receive customer dialogue texts, such as insurance customer service platforms and intelligent question-answering robots. Whenever new dialogue text enters the system, this mechanism can quickly determine its affiliation and update only the parameters of the corresponding cluster when the similarity condition is met, thereby achieving dynamic maintenance and real-time expansion of the knowledge base.

[0074] Furthermore, the technical advantage of this step lies in avoiding the high computational overhead of re-clustering the entire dataset with each new data entry, as required by traditional clustering methods, through a local update strategy. This improves the system's real-time responsiveness and resource utilization. Simultaneously, this mechanism enhances the robustness of the clustering model, enabling it to maintain the stability of the clustering structure even when faced with semantically similar but differently expressed texts, thus providing high-quality input for subsequent label generation and knowledge base construction.

[0075] S32, when the similarity between the newly input text feature vector and all clusters is less than... When needed, create new clusters and initialize the corresponding parameters.

[0076] Specifically, when the similarity between the newly input text feature vector and all clusters is less than 10, At this point, the system will create new clusters and initialize their parameters. This step is a key incremental clustering mechanism in the automatic construction method of dialogue knowledge base based on Bayesian nonparametric models, used to cope with dynamically changing customer dialogue text data and ensure the continuous expansion and updating of the knowledge base.

[0077] Specifically, this step first transforms the new input text into a feature vector using a text embedding model (such as BERT, Sentence-BERT, etc.). ,in This is the embedding dimension. Subsequently, the system calculates the vector and its relationship to the center vectors of all current clusters. ( Similarity between Cosine similarity is typically used as a metric, and its calculation formula is as follows: ; If all If the semantic similarity between the text and all existing clusters is below the set threshold, it means the text cannot be reasonably classified. In this case, the system will create a new cluster and... As the initial center vector of this cluster In some implementations, cluster initialization strategies can also be introduced, such as setting a minimum number of cluster members. Only when the number of new cluster members reaches this threshold will they be formally incorporated into the knowledge base structure, in order to avoid noisy text causing the number of clusters to swell.

[0078] Furthermore, threshold The value is typically set within the range of [0.1, 0.5], with the specific value depending on the output distribution of the text embedding model and the semantic granularity requirements of the business scenario. The cluster center vector can be updated using a mean update strategy, i.e.: ; in, Indicates the first The set of all text feature vectors in each cluster The number of its members.

[0079] Specifically, this step applies to the dynamic classification of customer dialogue text in insurance business. Especially when new business types or new questions arise, the system can automatically identify and establish new knowledge categories without manual intervention. For example, when a customer first raises a question about the "new energy vehicle claims process," if the topic is not yet covered by existing clusters, the system will automatically create a new cluster and initialize relevant parameters, providing a foundation for the classification and tag generation of similar questions in the future.

[0080] Specifically, the technical effect of this step is that it enables adaptive growth of the number of clusters, enhances the model's ability to adapt to unknown semantic patterns, improves the automation level and scalability of knowledge base construction, thereby significantly reducing manual maintenance costs and improving the system's practicality and robustness in dynamic business environments.

[0081] S4. A keyword extraction algorithm is used to extract descriptive topic keywords from the clustering results, generate automatic labels for the corresponding clusters, and construct a dialogue knowledge base.

[0082] Specifically, at the technical implementation level, this step first relies on the clustering results output by Bayesian nonparametric models (such as DP-Means, Chinese Restaurant Process, etc.). These clusters have the ability to adaptively adjust their number, dynamically expanding or merging cluster structures based on newly added dialogue text. Within each cluster, the system uses semantic-based keyword extraction algorithms (such as KeyBERT, TextRank, LDA, etc.) to perform topic modeling on the text content, extracting the most representative keywords as the labels for that cluster.

[0083] Furthermore, the performance and effectiveness of keyword extraction algorithms depend on several key parameters. For example, in KeyBERT, keyword extraction relies on the embedding dimension of the pre-trained language model (such as BERT, RoBERTa). and the length of the keyword candidate set , usually set , In the TextRank algorithm, keyword extraction depends on how the graph is constructed, including the window size. Number of iterations and threshold Typical settings , , The LDA model requires setting the number of topics. Sparsity parameters of the document-term matrix and In this invention, since the number of clusters has been adaptively determined by the Bayesian nonparametric model, the number of topics in LDA is... It can be directly aligned with the number of clusters. and Typically set to and This is to enhance the sparsity of the topic distribution.

[0084] Specifically, this step is widely applicable in business scenarios such as insurance, finance, and customer service, and is used to automatically classify and tag the dialogue text between customers and business personnel. For example, in insurance business, customers may ask questions on multiple topics such as claims process, product terms, and service consultation. The system automatically identifies the topic tags of each cluster through keyword extraction algorithms, such as "claims process", "car insurance terms", and "policy cancellation consultation", thereby building a structured question-and-answer knowledge base, which is convenient for business personnel to quickly retrieve and use later.

[0085] Specifically, this step effectively solves the problem of manual tag generation in traditional methods, significantly improving the automation and efficiency of knowledge base construction. Through keyword extraction algorithms, the system can accurately capture the topic features of clusters semantically without human intervention, thereby achieving automatic tag generation and updating. Furthermore, this method supports incremental updates; when new dialogue text is clustered, the system can immediately generate corresponding tags and update the knowledge base, ensuring the timeliness and accuracy of the knowledge system.

[0086] Furthermore, S4 includes: S41, using the KeyBERT algorithm, calculates the semantic similarity between keywords and text based on the embedding vectors of the BERT model, and selects the top keywords with the highest similarity. Use keywords as tags.

[0087] Specifically, this step plays a crucial role in the automatic construction of the dialogue knowledge base. Its technical implementation is based on semantic embedding and keyword extraction techniques in natural language processing, and it has a high degree of automation and semantic understanding capabilities.

[0088] Furthermore, BERT models (such as bert-base-uncased or bert-base-chinese) map the input text into a vector representation of length 768. KeyBERT further extracts the embedding vectors of all candidate keywords in the text and calculates the cosine similarity between each keyword vector and the original text vector, using the following formula: ; in, For the text embedding vector, This is the embedding vector of the keyword. Using this formula, the system can quantify the semantic relevance between the keyword and the text.

[0089] Furthermore, in this step The preset number of keywords is typically set to 5 to 10, depending on business needs. The BERT model has an embedding dimension of 768, and keyword vectors can be generated using either max-pooling or mean-pooling during keyword extraction. Furthermore, KeyBERT supports custom stop word lists and keyword length limits (e.g., minimum 2 words, maximum 5 words) to improve the semantic representativeness and readability of keywords.

[0090] Specifically, this step is mainly used to automatically generate topic tags for customer dialogue text after clustering using a Bayesian nonparametric model. For example, in an insurance business scenario, the system can automatically extract keywords such as "claims," ​​"policy," and "premium" from clusters such as "claims process," "policy changes," and "premium calculation" as tags for subsequent retrieval and classification in the knowledge base.

[0091] Specifically, through semantic embedding and similarity calculation, automatic identification and tag generation of text topics are achieved, avoiding the inefficiency and subjectivity of traditional manual tagging. Simultaneously, combined with the context-aware capabilities of the BERT model, KeyBERT can extract more semantically representative keywords, thereby improving the structure and retrieval efficiency of the knowledge base.

[0092] S42 uses the TextRank algorithm to model the graph structure of text in clusters and generates topic keyword tags by iteratively calculating node weights.

[0093] Specifically, this step is a key step in building a dialogue knowledge base. It is used to automatically summarize and tag the text content within each cluster after clustering is completed, thereby improving the searchability and semantic expressiveness of the knowledge base.

[0094] Furthermore, TextRank is a graph-based keyword extraction method. Its core idea is to treat words in the text as nodes in a graph, with co-occurrence relationships between words as edges, constructing a directed graph. Then, a variant of the PageRank algorithm is used to iteratively calculate the nodes in the graph to evaluate the importance of each word in the entire text. Specifically, the weight update formula for each word node is: ; in, Indicates the first Each word node Indicates pointing to The set of word nodes, Words The number of other word nodes it points to. The damping coefficient is usually set to... Through multiple iterative calculations, a stable weight for each word is eventually obtained through convergence. Words with higher weights are then used as the topic keywords for that cluster.

[0095] Specifically, the key parameters of the TextRank algorithm include window size, co-occurrence threshold, and iteration count. In this invention, the window size is typically set to 2 to 5 words to construct co-occurrence relationships between words; the co-occurrence threshold is used to filter low-frequency co-occurrence relationships and is generally set to 3 or 5; the iteration count is typically set to 100 to 200 times to ensure the convergence of weight calculation. Furthermore, a stop word filtering mechanism can optionally be introduced to remove meaningless common words and improve the semantic representativeness of keywords.

[0096] Specifically, this step is mainly applied to the label generation stage after clustering customer dialogue text. For example, in an insurance business scenario, a cluster may contain multiple dialogue texts about the "claims process." Keywords such as "claims," ​​"materials," "review," and "timeliness" are extracted using the TextRank algorithm and used as labels for the cluster, facilitating quick retrieval and use of relevant language by business personnel. This method is suitable for large-scale, dynamically updated dialogue datasets, especially in scenarios where business needs change frequently, enabling automatic label updates and optimization.

[0097] Specifically, this step, through graph structure modeling and iterative weight calculation, enables automatic keyword extraction from text within clusters without manual intervention, significantly improving the automation and efficiency of knowledge base construction. Simultaneously, the generated tags possess high semantic relevance and representativeness, contributing to enhanced semantic organization of the knowledge base and improving the accuracy and response speed of question-and-answer matching. Furthermore, this method, combined with the incremental clustering mechanism of Bayesian nonparametric models, enables dynamic updates and adaptive expansion of the knowledge base, adapting to the continuous evolution of business scenarios.

[0098] S5, based on the change in the number of clusters in the dynamically adjusted clustering results. ,when Exceeding the preset threshold When this happens, the structured update process of the knowledge base is triggered, the topic keyword tags of all clusters are recalculated and the knowledge base index is updated synchronously.

[0099] Specifically, the system continuously monitors the changes in the number of dynamic clusters generated by the incremental clustering process and accurately calculates the amount of change. When this change exceeds a preset stability threshold, it is determined that a significant shift has occurred in the macro-topic structure of the knowledge base, and a global structured update process is automatically initiated. This process is not a local adjustment, but rather a unified recalculation and annotation of the topic keywords for all clusters within the knowledge base, and a synchronous update of their index structure, thereby ensuring the consistency between the internal representation of the knowledge base and the external data distribution at the technical level.

[0100] Furthermore, in real-world dialogue and interaction scenarios, the evolution of user interests and business hotspots is often continuous and non-stationary. By setting reasonable thresholds, this solution can effectively distinguish between daily micro-topic fluctuations and macro-level thematic structural evolution. It avoids the waste of computing resources caused by frequent global reconstruction, and also prevents the decline in accuracy and timeliness of the knowledge base due to update lag when substantial changes occur in the thematic structure, thus achieving an effective balance between maintenance costs and knowledge freshness.

[0101] Specifically, the change in the number of clusters is a key macro-monitoring indicator, and its preset threshold is set based on the stability requirements of the knowledge base in specific business scenarios. This threshold can be used as a configurable parameter of the system, allowing operations and maintenance personnel to dynamically adjust it according to business cycles, data inflow rates, or domain characteristics. A higher threshold means that the system is more conservative in the face of structural changes and tends to maintain the existing state; while a lower threshold makes the system more sensitive to changes and can quickly respond to the emergence of new topics or the decline of old topics.

[0102] Specifically, this technological characteristic is particularly important when processing streaming data or in business areas undergoing rapid development. For example, in intelligent customer service systems, the launch of new products or seasonal events can quickly generate a large number of new inquiry topics; in public opinion analysis systems, the outbreak of public events can trigger dramatic changes in the structure of discourse topics. In such scenarios, this mechanism can automatically detect these macro-level structural changes and drive the knowledge base to complete a thorough "version iteration," thereby always maintaining its synchronization with the real-world topic ecosystem.

[0103] Specifically, this invention, by introducing this intelligent triggering and global update mechanism, ultimately achieves the transformation of the knowledge base from a "static archive" to a "dynamic organism." It ensures that the evolution of the knowledge base is not only unsupervised and adaptive, but also structured and controlled. This not only significantly improves the automation level of knowledge base management, but also fundamentally guarantees the long-term accuracy, consistency, and reliability of the knowledge base as the core support for downstream applications (such as intelligent question answering and recommendation systems).

[0104] This invention discloses an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model. By introducing a Bayesian nonparametric model, the method achieves adaptive determination and dynamic adjustment of the number of clusters, effectively overcoming the core shortcomings of existing technologies that require pre-setting the number of clusters and rely on manual annotation. This method automates the entire process from feature extraction, dynamic clustering, incremental updates to automatic label generation, significantly improving the intelligence level of knowledge base construction and the system's adaptive capabilities. While ensuring the accuracy of the knowledge structure, it greatly reduces maintenance costs and enhances the knowledge coverage and long-term practical value of the dialogue system in dynamic business scenarios.

[0105] Example 2 To achieve the above invention, embodiments of the present invention also provide specific steps for an adaptive construction method of a dialogue knowledge base based on a Bayesian nonparametric model, such as... Figure 2 As shown, it includes: S101, continuously acquire relevant dialogue text data: acquire dialogue text data that needs to be categorized in the knowledge base from various sources, and continuously add new text data.

[0106] S102, Bayesian nonparametric model text clustering: Input text data into a Bayesian nonparametric model for clustering. This method does not require pre-setting the number of clusters or centroids. The model can automatically classify data according to the data, adaptively increase the number of clusters, and has incremental clustering capability for new input data, avoiding the waste of multiple clustering operations on the full text data.

[0107] S103, Text Clustering Topic Summary Generation: Run keyword extraction algorithms such as Keybert, TextRank, and LDA in each text cluster output by the Bayesian nonparametric model to generate descriptive labels for each cluster.

[0108] S104, Conversation Knowledge Base Storage: Conversation knowledge base is built and stored based on various text clusters and their corresponding tags, and new dialogue texts are continuously acquired to dynamically update the knowledge base.

[0109] This invention discloses a specific step-by-step method for adaptively constructing a dialogue knowledge base based on a Bayesian nonparametric model. By introducing a Bayesian nonparametric model, the method dynamically determines and adaptively adjusts the number of clusters, effectively overcoming the core shortcomings of existing technologies that require pre-setting the number of clusters and rely on manual annotation. It achieves full automation from data acquisition, adaptive clustering, topic summarization to knowledge base construction, significantly improving the efficiency of knowledge base construction and the system's adaptability. While ensuring the accuracy of the knowledge structure, it greatly reduces maintenance costs and enhances the knowledge coverage and long-term practical value of the dialogue system in dynamic business scenarios.

[0110] Example 3 To achieve the above invention, such as Figure 3 As shown, this embodiment also provides an adaptive construction device 10 for a dialogue knowledge base based on a Bayesian nonparametric model. The device 10 includes: The data acquisition and feature extraction module 100 is used to continuously acquire customer dialogue text data and perform feature extraction to generate text feature vectors for clustering. The Bayesian nonparametric adaptive clustering module 200 is used to input the text feature vector into the Bayesian nonparametric model for adaptive clustering, dynamically determine the number of clusters according to the data distribution, and form an initial cluster. The incremental clustering parameter update module 300 is used to perform incremental clustering processing on the text data in the initial clusters, and locally update the model parameters based on the new input dialogue text feature vector to generate dynamically adjusted clustering results. The keyword extraction and knowledge base construction module 400 is used to extract descriptive topic keywords from the clustering results using a keyword extraction algorithm, generate automatic labels for the corresponding clusters, and construct a dialogue knowledge base.

[0111] In one embodiment of the present invention, it further includes: a knowledge base structured update module, used to update the knowledge base structure based on the change in the number of clusters in the dynamically adjusted clustering results. ,when Exceeding the preset threshold When this happens, the structured update process of the knowledge base is triggered, the topic keyword tags of all clusters are recalculated and the knowledge base index is updated synchronously.

[0112] This invention discloses an adaptive knowledge base construction device based on a Bayesian nonparametric model. Through the coordinated operation of various functional modules, it achieves dynamic determination of the number of clusters and adaptive adjustment of the knowledge base structure, effectively overcoming the core defects of existing technologies that require pre-setting the number of clusters and rely on manual intervention. This device automates the entire process from feature extraction, dynamic clustering, incremental updates to automatic tag generation, significantly improving the efficiency of knowledge base construction and the system's adaptability. While ensuring the accuracy and timeliness of the knowledge structure, it greatly reduces operation and maintenance costs, enhancing the knowledge management efficiency and long-term practical value of the dialogue system in dynamic business scenarios.

[0113] To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 4 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads the executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the adaptive construction method of dialogue knowledge base based on Bayesian nonparametric model described above.

[0114] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model as described in the foregoing embodiments.

[0115] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0116] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. An adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model, characterized in that, include: S1, continuously acquire customer conversation text data and extract features to generate text feature vectors for clustering; S2, input the text feature vector into a Bayesian nonparametric model for adaptive clustering, dynamically determine the number of clusters based on the data distribution, and form an initial cluster; S3 performs incremental clustering on the text data in the initial clusters, and locally updates the model parameters based on the new input dialogue text feature vectors to generate dynamically adjusted clustering results; S4. A keyword extraction algorithm is used to extract descriptive topic keywords from the clustering results, generate automatic labels for the corresponding clusters, and construct a dialogue knowledge base.

2. The method as described in claim 1, characterized in that, The process of continuously acquiring customer dialogue text data and extracting features to generate text feature vectors for clustering includes: S11 uses the bag-of-words model and TF-IDF algorithm to extract statistical features from the text and generate a numerical feature vector of the text. S12 uses a BERT pre-trained language model to embed semantic features into the text, generating a high-dimensional feature vector containing contextual semantic information.

3. The method as described in claim 1, characterized in that, The step of inputting the text feature vector into a Bayesian nonparametric model for adaptive clustering, dynamically determining the number of clusters based on the data distribution, and forming initial clusters includes: S21 uses the DP-GMM model, by setting prior distribution parameters. and basis distribution The optimal number of clusters is dynamically determined based on the distribution characteristics of the text feature vectors. S22, parameter estimation of the DP-GMM model is performed based on the variational inference method, and model convergence is achieved by maximizing the lower bound of evidence.

4. The method as described in claim 1, characterized in that, The incremental clustering process performed on the text data in the initial clusters, and the local updating of model parameters based on the new input dialogue text feature vectors to generate dynamically adjusted clustering results, includes: S31, when the similarity threshold between the newly input text feature vector and the existing clusters... satisfy When the time comes, update the parameters of the corresponding cluster; S32, when the similarity between the newly input text feature vector and all clusters is less than... When needed, create new clusters and initialize the corresponding parameters.

5. The method as described in claim 1, characterized in that, The step of using a keyword extraction algorithm to extract descriptive topic keywords from the clustering results, generating automatic labels for corresponding clusters, and constructing a dialogue knowledge base includes: S41, using the KeyBERT algorithm, calculates the semantic similarity between keywords and text based on the embedding vectors of the BERT model, and selects the top keywords with the highest similarity. Use one keyword as a tag; S42 uses the TextRank algorithm to model the graph structure of text in clusters and generates topic keyword tags by iteratively calculating node weights.

6. The method as described in claim 1, characterized in that, Also includes: S5, based on the change in the number of clusters in the dynamically adjusted clustering results. ,when Exceeding the preset threshold When this happens, the structured update process of the knowledge base is triggered, the topic keyword tags of all clusters are recalculated and the knowledge base index is updated synchronously.

7. An adaptive construction device for a dialogue knowledge base based on a Bayesian nonparametric model, characterized in that, include: The data acquisition and feature extraction module is used to continuously acquire customer dialogue text data and extract features to generate text feature vectors for clustering. The Bayesian nonparametric adaptive clustering module is used to input the text feature vector into the Bayesian nonparametric model for adaptive clustering, dynamically determine the number of clusters based on the data distribution, and form initial clusters. The incremental clustering parameter update module is used to perform incremental clustering processing on the text data in the initial clusters, and locally update the model parameters based on the feature vector of the new input dialogue text to generate dynamically adjusted clustering results. The keyword extraction and knowledge base construction module is used to extract descriptive topic keywords from the clustering results using a keyword extraction algorithm, generate automatic labels for the corresponding clusters, and construct a dialogue knowledge base.

8. The apparatus as claimed in claim 7, characterized in that, Also includes: The knowledge base structure update module is used to update the number of clusters in the dynamically adjusted clustering results. ,when Exceeding the preset threshold When this happens, the structured update process of the knowledge base is triggered, the topic keyword tags of all clusters are recalculated and the knowledge base index is updated synchronously.

9. An electronic device, comprising: processor; Memory, which stores executable instructions; When the processor executes instructions, it implements the adaptive construction method of dialogue knowledge base based on Bayesian nonparametric model as described in any one of claims 1-6.

10. A computer-readable storage medium storing a computer program, which, when executed by a processor, implements an adaptive construction method for a dialogue knowledge base based on a Bayesian nonparametric model as described in any one of claims 1-6.

Images