Text Categorization Model Using Predicate Extraction and PMI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Unsupervised learning methods result in low inference accuracy for learned models, necessitating improved methods to enhance model performance.
Innovation Solution
An information processing device that employs supervised learning techniques, including morphological analysis, extraction of predicate words, and calculation of pointwise mutual information to generate a learned model with higher inference accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If unsupervised learning is used to generate a learned model, then the model can be generated without labeled data, but the inference accuracy is low
Solution Approach 1:
The patent segments the learning process into two distinct phases: a learning data acquisition phase where labeled data is collected and preprocessed, and a learning model generation phase where the model is trained using the prepared data. This segmentation allows the system to leverage the high accuracy of supervised learning while managing data requirements through structured acquisition and preparation processes.
Solution Approach 2:
The patent performs preliminary actions by acquiring and preprocessing learning data before model generation. The system pre-processes the data to ensure it is ready for training, which enables the use of supervised learning methods that require clean, prepared data, thereby achieving high inference accuracy without needing complex data processing during the training phase.
2Reliability
If supervised learning is used to generate a learned model, then the inference accuracy is high, but learning data must be acquired and prepared
Solution Approach 1:
The system performs preliminary data acquisition and preparation actions to create a ready-to-use learning dataset. By pre-acquiring and pre-processing the data, the system can efficiently train the model without needing to collect and process large amounts of data during the training phase, thus reducing the effective data quantity requirement while maintaining high inference accuracy.
Solution Approach 2:
The patent creates a learned model that copies and generalizes patterns from the learning data. The model learns representative features and relationships from the training data, enabling it to accurately categorize new input data without requiring the actual input data to be processed in detail during inference, thereby reducing the effective data quantity needed for accurate predictions.
3Reliability
If morphological analysis and predicate extraction are performed, then the learned model can accurately categorize data, but the processing complexity increases
Solution Approach 1:
The patent segments the text processing into distinct modules: morphological analysis unit that performs word segmentation and POS tagging, and predicate extraction unit that identifies predicate words from the morphological analysis results. This segmentation allows each module to specialize in specific tasks, improving categorization accuracy while making the overall processing complexity more manageable through modular design.
Solution Approach 2:
The system extracts only the necessary information from the text data through morphological analysis and predicate extraction. By taking out and focusing on predicate words and their relationships with nouns and adjectives, the system achieves accurate categorization without processing every detail of the input text, thus reducing unnecessary processing complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An information processing device (100) includes an acquisition unit (120) that acquires multiple pieces of learning data in each of which a document and a category have been associated with each other, a morphological analysis performance unit (130) that performs morphological analysis on each of the multiple pieces of learning data, an extraction unit (140) that extracts words being predicates from among a plurality of words obtained by the morphological analysis, and a calculation generation unit (150) that generates a learned model by calculating pointwise mutual information based on the plurality of words obtained by the morphological analysis, a plurality of extracted words, and a plurality of categories, the learned model being a learned model which outputs a category corresponding to data when the data is inputted.