Data Dictionary Generation via Token Pair Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search mechanisms in databases and information resources face inefficiencies due to poor quality of catalog information, leading to either too few or unrelated data items when users enter search queries, as the selection of words, terms, and phrases to describe data items is often manual and time-consuming, and automated methods require manual evaluation.
Innovation Solution
A data dictionary is generated using token pairs, where each keyword is associated with synonyms, and statistical analysis is applied to evaluate and validate these pairs to ensure they produce similar search results, enhancing the search mechanism's relevance and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual selection of words, terms, and phrases is used to build catalog information, then the quality of catalog information can be maintained, but the time and effort required increases significantly
Solution Approach 1:
The system automatically generates catalog information by extracting terms, phrases, and concepts directly from the data stored in the information resource. The search mechanism performs self-service by using the data itself to create the catalog, eliminating the need for manual vocabulary selection while maintaining quality through automated analysis of actual data content
Solution Approach 2:
The manual mechanical process of selecting and categorizing vocabulary terms is replaced with an automated computational system that uses text analysis, frequency counting, and pattern recognition to generate catalog information from the data repository, significantly reducing time while preserving quality
2Loss of time
If automated methods are used to generate catalog information, then the time required is reduced, but the quality and relevance of catalog information deteriorates
Solution Approach 1:
The system incorporates feedback loops where search queries and results are analyzed to continuously refine and improve the catalog information. The automated method uses feedback from actual search patterns and data relationships to enhance the quality and relevance of generated catalog entries over time
Solution Approach 2:
The automated system adjusts parameters such as term frequency thresholds, relevance weighting factors, and categorization criteria based on analysis of the data repository characteristics, allowing the quality of generated catalog information to be optimized for specific data types and search requirements
3Device complexity
If simple search mechanisms are used, then the system complexity is reduced, but the ability to return relevant results decreases
Solution Approach 1:
The search mechanism is segmented into distinct functional components: a catalog generation module that creates structured information from the data repository, and a search execution module that queries the catalog. This segmentation allows the system to maintain moderate complexity while achieving high result relevance through specialized processing in each segment
Data Source
AI summary
There is provided an ecommerce method and system to generate a data dictionary for searching data items stored in a database. In one embodiment, the system comprises a candidate list generator module to generate a list of keywords from search query information and generate a set of token pairs including a keyword from the list of keywords and a token, the token being a synonym of the keyword. Demand information retrieved from query logs maintained for user-provided query entries is used to apply candidate selection rules to token pairs. The system also comprises a validation module and a data dictionary module to receive validated token pairs as entries in a vocabulary.