Rapid incremental clustering method for domain question-answering system consultations

Abstract

The invention discloses a rapid incremental clustering method for domain question-answering system consultations. The method comprises the following steps of performing semantic pretreatment on user consultations by utilizing a semantic irrelevancy dictionary and a word class dictionary through combination of consultation history offline clustering algorithm and consultation repeat removal on the basis of a clustering frame combined with offline clustering and online clustering, so that semantic normalization is realized; furthermore, calculating and constructing a similarity degree pattern on the basis of the similarity degree of multiple characteristics; performing offline clustering on consultation history of a user on the basis of the similarity degree pattern; and furthermore, performing online clustering on the consultations of the user by utilizing the offline clustering result as a clustering characteristic, and combining the offline clustering result with the online clustering result, so as to generate a clustering result. The clustering method provided by the invention is rapid in system response, is capable of achieving the accuracy conforming to the practical application demand, and is high in effectiveness and accuracy.

Description

technical field [0001] The invention relates to data mining and natural language processing in the field of artificial intelligence computers, in particular to a user consultation clustering method for text customer service consultation systems such as domain question answering systems. Background technique [0002] In a large number of natural language applications such as domain question answering systems, there is a basic and common problem: there are a large number of user consultation histories in the system, and user consultation consists of a short text (hereinafter referred to as short text corpus or user consultation) Corpus), how to cluster the consultation history into different classes according to a certain degree of similarity, and use the clustering results to identify and help the Q&A system understand the user’s consultation in the user domain question answering system. [0003] In the field of search engines, Baidu Zhizhi, domain question answering systems,...

AI Technical Summary

Problems solved by technology

[0005] 1) The amount of data is large, and the efficiency of the algorithm is very low by directly using clustering, which cannot meet the needs of the application;

[0006] 2) There are a lot of semantic noise in user consultation questions, which is a main reason for the poor clustering effect;

[0009] 5) Since the efficiency of the clustering algorithm is relatively low, it takes a lot of time to cluster all the data, which cannot meet the requirements of the application;

Images