Morpheme Importance Analysis Model Using Logistic Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current morpheme importance analysis models, such as decision tree models, suffer from low computing speed and accuracy in analyzing morpheme importance in queries, especially in continuous fields and text mining applications, due to their hierarchical structures and reliance on manual annotation and limited training data.
Innovation Solution
A method and apparatus for determining a morpheme importance analysis model by acquiring training data, determining feature values for preset features like morpheme self-explanation and cross features, and calculating model parameters to create a more efficient and accurate analysis model, which improves computing speed and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If decision tree models are used for morpheme importance analysis, then the analysis can be performed, but the computing speed is low and accuracy is insufficient
Solution Approach 1:
The patent changes the fundamental parameters of the analysis model by transitioning from decision tree models to logistic regression models. This parameter change enables the system to achieve both high accuracy in morpheme importance analysis and improved computing speed, as logistic regression can process continuous fields more efficiently while maintaining measurement precision through statistical optimization
Solution Approach 2:
The patent replaces the mechanical hierarchical structure of decision trees with a statistical mathematical model (logistic regression). This substitution allows for more efficient computation while improving accuracy, particularly in continuous fields, by using statistical optimization rather than hierarchical rule-based processing
2Reliability
If manual annotation is used to create training data, then the model can be trained, but the process is time-consuming and limited in scope
Solution Approach 1:
The patent implements self-service by enabling the system to automatically generate training data from search query logs without requiring manual annotation. The system extracts queries, segments them into morphemes, and automatically determines importance scores based on user behavior patterns, thereby eliminating the time-consuming manual annotation process while maintaining data quality through automated statistical methods
Solution Approach 2:
The patent performs preliminary action by pre-processing and storing search query logs in advance. The training data is generated from historical query information that has been collected and organized beforehand, allowing the model to be trained efficiently without requiring real-time manual annotation during the training process
3Ease of manufacture
If limited training data is used, then the model can be developed, but the accuracy and applicability are restricted
Solution Approach 1:
The patent enables the system to self-generate extensive training data from search query logs, eliminating the need for manual data collection. The system automatically extracts and processes historical query information, generating large-scale training datasets that enhance model applicability while keeping the development process straightforward through automated procedures
Solution Approach 2:
The patent achieves universality by creating a training data generation system that can process various types of search queries and morphemes automatically. The model can be trained on diverse query data from different domains and contexts, significantly expanding its applicability while maintaining ease of development through a unified automated approach
Data Source
AI summary
A method and an apparatus for determining a morpheme importance analysis model is provided, which belongs to the field of computers. The method includes: acquiring at least two pieces of training data, each piece of training data including a query, any morpheme in the query, and an importance score of the any morpheme in the query; determining a feature value of each preset feature of each piece of training data; and determining a model parameter according to the feature value of each preset feature of all training data and importance scores of morphemes included in all training data, and determining a morpheme importance analysis model according to the determined model parameter.