Dataset Search Optimization via Statistical Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large and complex datasets in product databases, such as those used by search providers, lead to slow search result delivery times, which is detrimental to providers as users prioritize speed, and existing methods struggle to efficiently process and deliver results quickly.
Innovation Solution
Generating and utilizing count statistics for characteristic-value pairs in datasets to reduce the number of records searched, identifying a 'driving search criterion' for logically related groups of criteria, and creating a reduced set of records for search requests, thereby minimizing the records to be accessed during searches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If large and complex datasets are stored in searchable databases, then comprehensive product information is available for analysis, but search result delivery time increases
Solution Approach 1:
The patent segments the large dataset into multiple partitions or chunks that can be searched independently. By dividing the comprehensive product database into smaller manageable segments, the system can process search queries more efficiently without sacrificing data completeness, thus reducing search result delivery time while maintaining comprehensive product information availability.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing statistical information (such as count statistics for characteristic-value pairs) before actual search queries are executed. This advance preparation allows the system to quickly filter and navigate large datasets during search operations, significantly reducing the time required to deliver search results while maintaining comprehensive data coverage.
2Measurement precision
If comprehensive datasets are searched to ensure accurate results, then search accuracy is improved, but computational resources are consumed
Solution Approach 1:
The patent applies partial action by searching only a subset or portion of the comprehensive dataset that is most relevant to the query, rather than exhaustively searching all records. By using statistical information to identify and focus on the most promising segments of the database, the system maintains search accuracy while significantly reducing computational resource consumption.
Solution Approach 2:
The patent extracts and utilizes statistical information (such as count statistics) from the comprehensive dataset to guide the search process. By taking out this metadata and using it to filter and prioritize search operations, the system achieves accurate results without the need to process the entire dataset, thereby reducing computational resource consumption.
3Quantity of substance
If traditional search methods are used on large datasets, then complete search coverage is achieved, but search speed decreases
Solution Approach 1:
The patent introduces statistical information (count statistics for characteristic-value pairs) as an intermediary layer between the query and the comprehensive dataset. This intermediary enables the system to quickly assess and navigate the data, identifying relevant segments without exhaustive searching, thus maintaining complete search coverage while dramatically improving search speed.
Solution Approach 2:
The patent performs preliminary computation of statistical metrics that characterize the dataset structure and content. This advance preparation creates a framework that guides subsequent search operations, allowing the system to achieve complete search coverage by focusing computational effort only on relevant portions of the data, thereby significantly improving search speed.
Data Source
AI summary
Methods and apparatus to search datasets are disclosed. An example disclosed method includes receiving a search request having at least two criteria and assigning the criteria to a first group according to a logical relationship between the criteria. The example method further includes determining which of the criteria in the first group is satisfied by a least amount of records in a database based on a plurality of counts, the counts respectively indicative of a number of corresponding records in the database satisfying a respective one of criteria exhibited by the database, and identifying a reduced set of records in the database to be searched, the reduced set of records corresponding to the first or second criteria that is satisfied by the least amount of records in the database, and reducing a search time associated with the search request by searching the reduced set of records from the database.