Method for mapping Chinese problems on basis of LDA (latent Dirichlet allocation)

Abstract

The invention discloses a method for mapping Chinese problems on the basis of LDA (latent Dirichlet allocation). The method includes classifying document libraries by the aid of LDA theme models; classifying word characteristics for the problems by the aid of Softmax regression models; assigning high weights to notional words according to difference of categories of the word characteristics, assigning low weights to functional words according to the difference of the categories of the word characteristics and allowing the weights of different word characteristics of the notional words to be different from one another; finding dependency relations of terms in sentences by means of syntactic analysis on the basis of dependency grammar; assigning different weights according to the difference of components of the terms in the sentences; multiplying two portions to obtain a weight of each word in each problem; establishing relationships by the aid of weighted distribution of terms in the problems and distribution of themes and lexical terms in documents according to Bayesian rules. The method has the advantages that the documents are classified on the basis of the LDA theme models, the different weights are distributed on the reference of the word characteristics of the lexical terms in the interrogative sentences and the components in the sentences, accordingly, effects of important lexical terms can be improved during classification, and the Chinese problem mapping accuracy can be improved.

Description

technical field [0001] The invention relates to a Chinese question mapping method, in particular to an LDA-based Chinese question mapping method, and belongs to the field of computer software. Background technique [0002] With the rapid development of Internet technology, search engines can provide people with various online information quickly and conveniently. Early search engines required users to submit keywords for query, and then the system returned to the user a list of web documents related to the query. Such limited keywords sometimes cannot fully express the user's query intention, and even not all users can accurately Give all kinds of keywords you want to query. Therefore, people urgently need a more efficient and convenient way to obtain information from the Internet. The question answering system was born under such a background. It allows users to ask questions in natural language and then directly returns accurate answers. [0003] Automatic question answe...

AI Technical Summary

Problems solved by technology

Now the machine learning method based on statistics occupies a dominant position. The more representative one is to use the SVM (Support Vector Machine) algorithm to classify problems. The analysis determines that using this method to classify Chinese questions will bring a large error

Images