NL-KL Classification Model for Query Data Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional machine learning techniques for annotating training data are labor-intensive and inefficient, and automated methods using query click log data often produce low-quality training data.
Innovation Solution
A classification model, referred to as the NL-KL classification model, is developed to distinguish between natural language (NL) and keyword language (KL) linguistic items, which improves the quality of training data by filtering out NL items and using them to produce a natural language interpretation model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated techniques leverage query click log data to generate training data, then productivity is improved, but manufacturing precision deteriorates
Solution Approach 1:
The patent segments training data into two distinct categories: natural language data (from queries with multiple words expressing intent) and keyword data (from single-word or short-phrase queries). By separating these types, the system can apply different processing strategies - using natural language queries directly as training examples while using keyword queries to generate candidate answers through search result analysis, thereby maintaining high productivity while improving manufacturing precision.
Solution Approach 2:
The patent introduces search result data as an intermediary element between keyword queries and training data generation. When a keyword query is submitted, the system uses the search results (titles, snippets, URLs) as intermediate material to construct candidate answers, which are then paired with the keyword query to form training examples. This intermediary approach enables automated generation of high-quality training data from keyword queries that would otherwise be too ambiguous.
2Manufacturing precision
If manual annotation techniques are used, then manufacturing precision is improved, but productivity deteriorates
Solution Approach 1:
The patent implements self-service by enabling the system to automatically generate training data from user interactions with the search engine. Query logs containing user-submitted queries and corresponding click behavior are automatically processed to extract training examples. The system uses its own operational data (search queries and user clicks) to train itself, eliminating the need for external manual annotation while maintaining high productivity through automated processing of large volumes of interaction data.
3Productivity
If keyword language items are used for training, then productivity is improved, but manufacturing precision deteriorates
Solution Approach 1:
The patent applies inversion by reversing the traditional approach to keyword query processing. Instead of manually annotating keyword queries or discarding them as low-quality, the system inverts the process by using keyword queries to generate candidate answers through search result analysis, then pairing these generated answers with the original keyword queries to create training examples. This inverted approach transforms potentially low-quality keyword data into valuable training material, maintaining productivity while improving manufacturing precision through automated answer generation.
Data Source
AI summary
This disclosure pertains to a classification model, and to functionality for producing and applying the classification model. The classification model is configured to discriminate whether an input linguistic item (such as a query) corresponding to either a natural language (NL) linguistic item or a keyword language (KL) linguistic item. An NL linguistic item expresses an intent using a natural language, while a KL linguistic item expresses the intent using one or more keywords. In a training phase, the functionality produces the classification model based on query click log data or the like. In an application phase, the functionality may, among other uses, use the classification model to filter a subset of NL linguistic items from a larger set of items, and then use the subset of NL linguistic items to train a natural language interpretation model, such as a spoken language understanding model.