Taxonomy-Based Information Retrieval for Non-Verbatim Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The exponential growth of data has made it challenging for users to retrieve relevant documents accurately, as they often struggle to recall specific details, and existing technologies fail to effectively map user queries to documents even when query terms are not present verbatim in the input corpus.
Innovation Solution
A method and device for information retrieval that determines top n-grams from an input corpus, generates document clusters, identifies an external corpus, merges it with the input corpus to create an enriched corpus, and builds a taxonomy to retrieve documents based on n-gram clusters and relatedness, enabling retrieval of relevant documents even when query terms are not present verbatim.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If users perform search using verbatim terms from the input corpus, then retrieval accuracy improves, but the system cannot retrieve documents when query terms are not present verbatim in the corpus
Solution Approach 1:
The system transforms the search approach by changing from exact term matching to n-gram based clustering and taxonomy mapping. Instead of requiring verbatim query terms, the system clusters documents by n-grams and creates a taxonomy that allows flexible query interpretation, thereby maintaining retrieval accuracy while improving query flexibility
Solution Approach 2:
The patent introduces an intermediary taxonomy structure that maps between user queries and document content. This taxonomy acts as a mediator that connects query terms to relevant documents even when exact term matches are absent, resolving the contradiction between precision and flexibility
2Reliability
If users recollect specific facts to reform queries, then retrieval relevance improves, but users struggle to accurately recollect specific details
Solution Approach 1:
The system segments documents into n-gram clusters and organizes them in a taxonomy structure. This segmentation allows users to query at different levels of granularity without needing to recollect specific details, as the segmented structure provides multiple access points to relevant information
Solution Approach 2:
The system performs preliminary action by pre-clustering documents and building the taxonomy structure before user queries are submitted. This pre-processing creates an organized framework that guides users in formulating queries without requiring them to recollect specific factual details
3Measurement precision
If the search scope is limited to the input corpus, then retrieval precision improves, but the scope of information retrieval is restricted
Solution Approach 1:
The patent creates a taxonomy that serves multiple functions: it organizes the input corpus for precise retrieval, maps external domain terms to the corpus, and enables both narrow and broad search scopes. This multi-functional taxonomy resolves the contradiction between precision and scope
Data Source
AI summary
This disclosure relates generally to an information retrieval technology and more particularly to a creation of a taxonomy to facilitate subsequent search and retrieval of information. In one embodiment, an information retrieval device is disclosed, that comprises a processor and a memory that stores instructions, which, on execution, causes the processor to receive an input corpus. Thereafter, input document clusters are generated from top input n-grams associated with the input corpus. Further, top-ranked input n-grams are determined from the top input n-grams. Thereafter, an external corpus is identified based on the top-ranked input n-grams. An enriched corpus (external and input corpus), is clustered based on top enriched n-grams associated with the enriched corpus to generate enriched document clusters. Further, for each n-gram of the enriched corpus, corresponding n-gram clusters are determined. Finally, creates a taxonomy based on the input document clusters, the enriched document clusters, the n-gram clusters and the top-ranked input n-grams.