Hybrid Natural Language Processor for Query Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Natural language processing systems face challenges in efficiently classifying queries due to high computing resource requirements and difficulty in correctly classifying queries with varying word meanings, leading to incorrect domain classification.
Innovation Solution
A hybrid natural language processing system that combines statistical models with rule-based classifiers and named entity recognition, using pre- and post-processing rules to refine classifications and extract relevant entities, allowing for more accurate domain and task identification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If statistical models are used to classify natural language queries into categories, then classification coverage is improved, but computing resource requirements increase significantly
Solution Approach 1:
The patent segments the classification process into multiple stages: first using rule-based classification for quick filtering, then applying statistical models only to queries that don't match initial rules. This hierarchical segmentation reduces overall computing resource usage while maintaining comprehensive classification coverage.
Solution Approach 2:
The patent applies preliminary rule-based classification before invoking statistical models. By performing preliminary filtering with low-cost rules, the system prepares the input in advance, allowing statistical models to process fewer queries and thus reducing total computing resource requirements.
2Adaptability or versatility
If statistical models are used for natural language classification, then classification capability is improved, but accuracy deteriorates for queries where word variations significantly alter meaning
Solution Approach 1:
The patent merges rule-based classification with statistical model classification into a hybrid system. The rule-based component captures precise word-meaning relationships, while the statistical model provides general classification capability. Together, they achieve both high accuracy for word-sensitive queries and broad classification capability.
Solution Approach 2:
The patent introduces rule-based classification as an intermediary layer between the input query and the statistical model. This intermediary processes queries first, applying linguistic rules that capture subtle word meaning variations, thereby improving the accuracy of subsequent statistical model classification for ambiguous queries.
3Measurement precision
If rule-based classifiers are added to restrict statistical model output, then classification accuracy is improved, but device complexity increases
Solution Approach 1:
The patent implements a dynamic classification system where the interaction between rule-based and statistical model components adapts to each query. Rules act as dynamic constraints that are applied selectively based on query characteristics, improving accuracy without requiring a static, overly complex system architecture.
Data Source
AI summary
Methods and a natural language processor for processing a natural language query are provided. The processor includes a classifier, a rule-based pre-processor, a rule-based post-processor, a named entity recognizer, and an output module. The method involves receiving a text representation of the natural language query, pre-processing the text representation, applying a classification statistical model to the text representation when pre-processing fails, applying a post-processing rule, and performing name entity recognition.