Query Processing System Shard Indexing for Scan Time Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face inefficiencies in processing queries for single user data due to large data volumes, leading to prolonged scan times as they struggle to efficiently fill memory regions and scan user data.
Innovation Solution
A computer-implemented method using shard indexing and memory region indexing techniques to store and retrieve data associated with a user, employing an increment sequence method to map data to natural numbers for efficient storage and retrieval within predefined intervals, utilizing index and directory mappers to identify memory regions and shards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If database systems store large amount of data, then data storage capacity is improved, but scan time for single user query increases
Solution Approach 1:
The database is divided into multiple shards, where each shard contains a portion of the user data. This segmentation allows the system to store large amounts of data while reducing scan time by only searching relevant shards for single user queries, rather than scanning the entire database.
Solution Approach 2:
The patent introduces a new dimension of organization by mapping user data to memory regions using increment sequence methods and modular arithmetic. This creates a multi-dimensional indexing structure (shard level + memory region level) that enables efficient location of user data without scanning all stored data.
2Reliability
If database systems scan all memory regions, then data access completeness is improved, but processing efficiency deteriorates
Solution Approach 1:
The system performs preliminary organization of data into shards and memory regions using indexing techniques before queries are executed. This preliminary action creates a structured layout where user data is pre-positioned in specific memory regions, enabling efficient retrieval without scanning all memory regions during query processing.
Solution Approach 2:
The patent introduces index structures and mappers as intermediaries between the query system and stored data. These intermediaries translate user identifiers into specific shard and memory region locations, allowing the system to access complete user data efficiently without brute-force scanning of all memory regions.
3Quantity of substance
If conventional database systems store data, then data storage is achieved, but memory region filling efficiency is poor
Solution Approach 1:
The patent applies parameter changes by using modular arithmetic operations to map user data to specific memory regions. This mathematical approach ensures uniform distribution of data across memory regions, maximizing filling efficiency and eliminating the inefficiencies of conventional storage methods.
Data Source
AI summary
The present disclosure provides a method and system to reduce scan time for single user query using a query processing system. The query processing system receives a first set of data. In addition, the query processing system obtains a second set of data. Further, the query processing system fetches a third set of data. Furthermore, the query processing system stores the first set of data, the second set of data, and the third set of data in a first shard of a plurality of shards. Moreover, the query processing system collects the first set of data, the second set of data, and the third set of data in a first memory region of a plurality of memory regions. Also, the query processing system receives a query request from an administrator. Also, the query processing system scans the first memory region of the plurality of memory regions of the first shard.