Secondary Index for Distributed Database Parallel Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current index technologies in database systems are serial and incompatible with massive parallel solutions, leading to increased workload in data access due to the growing size of data clusters, necessitating a new approach for parallel index solutions.
Innovation Solution
A computerized method for building a secondary index in a distributed database system that involves generating a first database index, sorting it, and then creating a second index with maximum and minimum key values and physical locations for each block, enabling direct access and efficient data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional serial index technology is used in database systems, then data access efficiency is improved for small datasets, but scalability deteriorates as data cluster size increases
Solution Approach 1:
The patent divides the database index into multiple segments distributed across different servers in a cluster. Each server maintains a portion of the index, allowing parallel processing of data access requests. This segmentation enables the system to handle larger datasets by distributing the workload across multiple nodes rather than concentrating it on a single serial index structure.
Solution Approach 2:
The patent introduces a hierarchical dimension to traditional indexes by creating a two-level index structure: local indexes on individual servers and a global index across the cluster. This dimensional expansion allows the system to maintain efficient access patterns while scaling horizontally, transforming the single-dimension serial access model into a multi-dimensional parallel access architecture.
2Quantity of substance
If data cluster size is increased to handle Big Data, then data storage capacity is improved, but data access workload increases due to index incompatibility
Solution Approach 1:
The index structure is segmented into manageable portions distributed across cluster nodes, with each node maintaining its own local index. This reduces the complexity burden on individual servers while collectively providing comprehensive coverage of the entire dataset, enabling scalable index management across large data clusters.
Solution Approach 2:
The patent introduces a global index as an intermediary layer that coordinates access across distributed local indexes. This intermediary structure simplifies the complexity of managing distributed indexes by providing a unified view and coordination mechanism, allowing queries to be routed efficiently across the cluster without requiring each node to manage the entire index.
3Productivity
If parallel processing is implemented to handle large datasets, then processing capacity is improved, but compatibility with traditional index solutions deteriorates
Solution Approach 1:
The patent implements a dynamic index architecture where the system can adapt its indexing strategy based on the specific query and data distribution. Local indexes enable parallel processing for partitioned queries, while the global index provides fallback capability for queries requiring full-cluster access, allowing the system to dynamically select the most appropriate access path.
Solution Approach 2:
The patent creates a universal index system that serves multiple functions: local indexes handle partitioned queries in parallel, the global index coordinates cross-partition queries, and the system can adapt to different query patterns. This multi-functional index architecture maintains compatibility with traditional indexing concepts while enabling parallel processing capabilities.
Data Source
AI summary
In one aspect, a computerized method for building a secondary index used to access index keys in a distributed database includes the step of providing a distributed database. The computerized method includes the step of providing a database file of the distributed database or on a cluster. The computerized method includes the step of building a first database index of index keys of the database file. The computerized method includes the step of organizing the first database index. The computerized method includes the step of, based on the organization of the first database index, building a second database index.