Query Server Inverted Index for Complex Data Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Query efficiency is low when dealing with large amounts of data stored across different tables and databases, particularly for complex combination queries in relational databases.
Innovation Solution
A data processing system comprising a data storage layer, a data loading layer, and a data index layer, where the query server creates an inverted index for updated data from multiple tables and databases, and includes workload balancing to dynamically allocate query tasks across multiple query servers, improving query efficiency and handling complex queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If data is divided into different tables and databases according to a predetermined rule, then data storage organization is improved, but query efficiency deteriorates
Solution Approach 1:
The patent introduces an inverted index as an intermediary structure between the divided tables/databases and the query operation. The inverted index maps query terms to their corresponding table and row locations, enabling efficient retrieval without scanning multiple divided tables directly. This mediator resolves the contradiction by providing a streamlined access path that bypasses the complexity of querying divided data structures.
Solution Approach 2:
The patent creates a copied representation of the data distribution information in the inverted index. Instead of directly querying the original divided tables, the system uses a copied index structure that contains pre-computed location information. This copying approach allows rapid query resolution while maintaining the benefits of data division for storage and update efficiency.
2Quantity of substance
If data is stored in multiple tables and databases, then data storage scalability is improved, but complex combination query capability deteriorates
Solution Approach 1:
The inverted index structure serves multiple functions: it supports simple term queries, complex combination queries, and provides a unified access interface for divided data. By making the index structure multi-functional, the system can handle various query types including complex combinations across multiple tables and databases, resolving the contradiction between storage scalability and query versatility.
Solution Approach 2:
The inverted index acts as a universal mediator that translates diverse query requirements into efficient retrieval operations. Whether the query is simple or complex, the inverted index provides a standardized access mechanism that works across all divided tables and databases, enabling complex combination queries while maintaining storage scalability.
3Device complexity
If query tasks are processed by a single query server, then system simplicity is maintained, but concurrent processing capability deteriorates
Solution Approach 1:
The patent segments the query processing function by dividing the inverted index into multiple parts distributed across different query servers. Each server handles a portion of the index and can process queries independently. This segmentation enables concurrent processing while maintaining relative system simplicity through modular architecture, resolving the contradiction between system simplicity and concurrent processing capability.
Solution Approach 2:
The patent introduces a new dimension of parallelism by distributing the index across multiple servers in a clustered architecture. Instead of improving concurrency within a single server, the system adds a spatial dimension of distribution, allowing simultaneous query processing across multiple nodes. This dimensional change enables concurrent processing while keeping each individual server relatively simple.
Data Source
AI summary
A data processing system comprises a loading server and a query server. The loading server is configured to extract updated data from at least two table databases and insert the updated data into the query server. The query server is configured to create an inverted index for querying the updated data, and the inverted index is indexed to an index table of the query server.


