Hierarchical Clusters and To Operator for Inverted Index Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inverted Index Databases (IIDBs) like Lucene are limited in representing structural relationships between records beyond single-level linear collections, making it difficult to perform precise searches for ambiguous terms such as consumer brands, which can have multiple meanings, while maintaining high recall.
Innovation Solution
The introduction of a 'To' operator and bit maps allows for efficient representation and querying of hierarchical clusters within IIDBs, enabling the translation of query results across different levels of a cluster hierarchy and improving the precision of searches by using Exclude Terms and frame extraction rules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If inverted index databases use single-level linear collection structure, then device complexity is low, but manufacturing precision of search results deteriorates due to inability to represent complex structural relationships
Solution Approach 1:
The patent segments the single-level linear collection into hierarchical clusters with multiple levels. Each cluster divides records into sub-clusters, creating a tree-like structure that preserves complex relationships while maintaining manageable complexity at each level. This segmentation enables precise search results by capturing structural context without overwhelming system complexity.
Solution Approach 2:
The patent adds hierarchical dimensionality to the traditional single-level structure by introducing multiple cluster levels (parent clusters, child clusters, sub-clusters). This dimensional transformation allows the database to represent complex structural relationships vertically through hierarchy levels while maintaining horizontal scalability, thereby improving search precision without linearly increasing overall complexity.
2Measurement precision
If inverted index databases represent complex structural relationships, then search precision for ambiguous terms improves, but device complexity increases
Solution Approach 1:
The patent introduces dynamic cluster expansion and contraction capabilities where hierarchical structures can be expanded or collapsed based on query needs. This dynamic approach allows the system to represent complex relationships only when necessary for ambiguous terms, improving search precision while avoiding unnecessary complexity for simple queries. The hierarchical structure adapts its complexity level dynamically.
Solution Approach 2:
The patent uses cluster headers and intermediate indexing structures as mediators between the raw data and query processing. These intermediaries organize complex hierarchical relationships into manageable segments that can be processed efficiently. The intermediate structures act as buffers that simplify the complexity burden while preserving the ability to represent complex structural relationships for improved search precision.
3Loss of information
If inverted index databases maintain simple linear collection structure, then ease of operation is high, but loss of information about structural relationships increases
Solution Approach 1:
The patent implements nested cluster structures where parent clusters contain child clusters, which in turn contain sub-clusters, creating a nested hierarchy. This nesting preserves structural relationship information at multiple levels of abstraction. The nested structure maintains ease of operation by allowing queries to work with aggregated parent clusters when detailed structural information is not needed, while still preserving the ability to access fine-grained structural relationships when necessary.
Data Source
AI summary
To the standard operations of an inverted index database, a new “To” operator is added. The “To” operator treats the standard single-level linear collection of records as being organized into localized clusters. Techniques for hierarchical clusters are presented. During indexing, hierarchical clusters are serialized according to a uniform visitation procedure. Serialization produces bit maps, one for each hierarchical level, that preserve the hierarchical level of each record and its location in the serialization sequence. For non-hierarchical clusters, byte maps can be produced. The “To” operator accepts a list of records, each at a same hierarchical level in a cluster, and a specification of a hierarchical level that all the input records should be converted into. The “To” operator outputs a list of records, representing a conversion of the input records to the specified new level. Logical Forms, against which frame extraction rules match, can be stored in hierarchical clusters. A database query can be formulated, that matches against such Logical Forms, that provides the same or similar functionality to a frame extraction rule. Such queries extend the range of applications of a particular inverted index database and/or can aid in the development of new frame extraction rules.