Text annotation method and system based on mixed topics

Abstract

The invention provides a mixed-topic-based text marking method and a mixed-topic-based text marking system. The method comprises the following steps of learning about an acquired concept text by adopting an LDA (latent dirichlet allocation) algorithm, namely setting a first target explicit topic, and learning about the first target explicit topic to obtain the probability distribution of first target explicit topic-words; learning about an acquired resource text by adopting the LDA algorithm, namely setting a second target explicit topic and a target implicit topic, and learning about an initialization result of the second target explicit topic and the target implicit topic to obtain the probability distribution of second target explicit topic-words and the probability distribution of target implicit topic-words; marking a text to be marked according to the probability distribution of the second target explicit topic-words and the probability distribution of the target implicit topic-words. According to the method and the system, the explicit topics and the implicit topic are mixed, so that the problem of expandability of a field body can be solved in a low-cost way, and the text marking quality can be improved.

Description

technical field [0001] The present invention relates to the technical field of text annotation, and more specifically, to a method and system for text annotation based on mixed topics. Background technique [0002] With the promotion of mobile Internet and social networks, a large amount of User Generated Content (UGC for short) has been produced, but because people tend to use different words and expressions to express similar content, the widely used in traditional search engines The method of managing UGC content based on the inverted index of words cannot reveal the inherent relevance of UGC, and cannot effectively maintain, retrieve and recommend these texts. Therefore, it is very necessary to understand the meaning of the text at the semantic level. [0003] UGC can be deeply understood using Natural Language Processing (NLP for short) technology, but due to the complexity of human natural language, precise understanding is impossible and often unnecessary. In fact, if...

AI Technical Summary

Problems solved by technology

[0006] However, neither of these two ideas is perfect. Although the former can produce semantic annotations that are easy to understand and conform to human intuition, it requires a huge amount of manual editing work, which is simply impossible in many environments; Potential topics can be automatically learned through machine learning methods, but the meaning of the learned topics is often difficult to interpret, and its granularity is often too large, and there is a lack of methods to control the granularity

Images