Text Type Recognition via Keyword Probability and Topic Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text type recognition methods, such as Naive Bayes and Support Vector Machine, face accuracy issues due to polysemous words and synonyms, leading to low classification accuracy.
Innovation Solution
A method involving preprocessing to extract a keyword set, calculating occurrence probability values, and using a document topic generation model to determine the text type based on the relationship between keywords and topics, improving classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing text type recognition methods (Naive Bayes, Support Vector Machine) are used, then the recognition process is simple, but the accuracy of recognizing text type is low due to polysemous words and synonyms
Solution Approach 1:
The patent segments the text analysis process into multiple stages: first extracting keywords from the text, then calculating occurrence probabilities of these keywords, and finally using a document topic generation model to determine text type. This segmentation allows the system to handle polysemous words and synonyms by analyzing keyword contexts rather than relying on simple word-based classification.
Solution Approach 2:
The patent introduces a document topic generation model as an intermediary between keyword extraction and text type recognition. This model uses occurrence probability values of keywords to determine topic probabilities, which then feed into the final text type classification. This intermediary layer helps resolve ambiguities caused by polysemous words and synonyms by considering the overall topic context.
2Measurement precision
If a document topic generation model is used to improve recognition accuracy, then the accuracy of recognizing text type is improved, but the computational complexity increases
Solution Approach 1:
The patent performs preliminary keyword extraction and occurrence probability calculation before applying the document topic generation model. By pre-processing the text to identify and quantify keywords, the system reduces the dimensionality of the input data for the topic model, thereby lowering computational requirements while maintaining or improving recognition accuracy.
Solution Approach 2:
The patent transforms the text recognition problem from direct word-based classification to a multi-stage process involving keyword extraction, occurrence probability calculation, and topic probability determination. This parameter transformation allows the system to capture semantic relationships more effectively while managing computational complexity through structured processing stages.
Data Source
AI summary
The present disclosure discloses a method and apparatus for recognizing a text type, and a device. A specific embodiment of the method comprises: preprocessing a pre-acquired text to obtain a keyword set of the text; calculating an occurrence probability value of each keyword in the keyword set in the text; importing, for the each keyword in the keyword set, the keyword and the occurrence probability value corresponding to the keyword into a pre-established document topic generation model, and determining an occurrence probability value of each topic in the text preset in the document topic generation model; and recognizing a type to which the text belongs based on the occurrence probability value of the each topic in the text. The embodiment improves an accuracy of recognizing the type of the text.