Heterogeneous Search Index Subset Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current web browsers are unintelligent and require manual searching, leading to inefficiencies in finding relevant information, and MetaSearch engines face challenges in merging and ranking results from heterogeneous search indices, often returning excessive documents that overwhelm users.
Innovation Solution
A system and method that estimates the number of documents for each entity in a search scope, forming a subset of entities to send to search engines, using historical caches and machine learning techniques to refine queries and minimize the number of documents retrieved, thereby improving recall and reducing search costs without compromising precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If MetaSearch engines merge results from multiple search engines, then comprehensive search coverage is improved, but the number of returned documents becomes excessive and overwhelms users
Solution Approach 1:
The patent extracts only the most relevant documents from the merged search results by using a selection process that evaluates document quality and relevance. Instead of returning all documents from multiple search engines, the system selectively extracts a manageable subset that maintains comprehensive coverage while avoiding information overload.
Solution Approach 2:
The system changes the parameter of document selection by introducing quality metrics and relevance thresholds. By adjusting these parameters, the system can control the number and quality of returned documents, transforming the raw volume of search results into a refined, manageable set that balances comprehensiveness with usability.
2Ease of operation
If web browsers perform manual searching, then user control over search process is maintained, but search efficiency and time to find relevant information deteriorates
Solution Approach 1:
The system performs preliminary actions by pre-processing and organizing search results from multiple engines before presenting them to the user. Documents are pre-evaluated, ranked, and filtered based on relevance and quality metrics, so that when users receive results, they are already optimized for quick access to relevant information, saving significant time while maintaining user control over the search process.
3Measurement precision
If search engines return top ranked results from each engine, then search precision is improved, but recall of comprehensive results deteriorates
Solution Approach 1:
The patent merges results from multiple search engines and then applies a unified ranking and selection process. By combining the strengths of different engines and using comprehensive evaluation criteria, the system achieves both high precision (through quality filtering) and high recall (through diverse source coverage), resolving the trade-off between returning only top results and maintaining comprehensive coverage.
Data Source
AI summary
A method, system and computer program product are disclosed for searching for data. In one embodiment, the invention provides a method comprising identifying a query and a search scope including a set of specified entities; and for each of these entities, estimating a number of documents that would be identified in a search through the entity to answer the query. On the basis of this estimating, a subset of the entities is formed. The query and this subset of entities are sent to a search engine to search the subset of entities to answer the query. In one embodiment, the estimating includes collecting statistical information from queries to build up a historical cache using heuristics or machine learning techniques, wherein the query includes a key word and a scope, and the historical cache contains a maximum number of returned results for an entity given the queries executed.