Multi-layer search index for reducing computing resource costs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search engines face high computing resource costs when processing large networks of documents due to the structure and size of their indexes, making it inefficient to analyze and serve search queries effectively.
Innovation Solution
A multi-layer search-engine index is implemented, comprising a term index and a document index, where the term index organizes searchable terms and the document index organizes document-specific information, both served using computing components like solid-state drives, optimizing memory utilization and CPU usage by omitting certain categories of information and using efficient encoding mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a comprehensive search-engine index is used to cover all documents in a vast network, then the search engine can identify relevant documents, but the computing resource costs become prohibitively high
Solution Approach 1:
The patent divides the search-engine index into multiple segments or layers, where each segment covers a subset of documents. This segmentation allows the system to load only relevant index segments into memory based on query characteristics, reducing the amount of data that needs to be processed at any given time while maintaining comprehensive search coverage across all documents.
2Reliability
If the index includes all categories of information for every document, then complete search coverage is achieved, but memory utilization decreases and CPU usage increases
Solution Approach 1:
The patent implements local quality by storing different types of information at different levels of detail in the index structure. Frequently accessed or important document attributes are stored in full detail, while less critical attributes are stored in compressed or summarized form. This allows the system to optimize memory usage and CPU processing by applying different storage strategies to different parts of the index based on their access patterns and importance.
Solution Approach 2:
The patent employs parameter changes by dynamically adjusting the level of detail and compression applied to index data based on query characteristics, document importance, and system resource availability. This allows the system to transform the index representation from a static comprehensive structure to a dynamic adaptive structure that optimizes for current processing needs.
3Reliability
If traditional indexing structures are used, then the system can handle current query loads, but it cannot efficiently support parallel query processing or scale to larger document networks
Solution Approach 1:
The patent introduces a new dimensional organization to the index structure, arranging index data in a multi-dimensional space that allows for efficient parallel access patterns. This dimensional reorganization enables multiple queries to be processed simultaneously by directing them to different segments or layers of the index, thereby supporting parallel query processing while maintaining the ability to handle current query loads.
Data Source
AI summary
Subject matter described herein includes a multi-layer search-engine index. Accordingly, the search-engine index is divided into multiple indexes, each of which includes a respective set of information used to serve (i.e., respond to) a query. One index includes a term index, which organizes a set of terms that are found among a collection of documents. Another index includes a document index, which organizes a set of documents that are searchable. A computing device is used to serve the search-engine index (i.e., to analyze the index when identifying documents relevant to a search query). For example, a solid-state device might be used to serve the multi-layer search-engine index.