Database Record Sorting via Key Range Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In bulk data processing, accessing multiple database tables randomly increases processing time, especially when dealing with vast amounts of data, as existing techniques do not effectively utilize the sorting order of input data to enhance performance.
Innovation Solution
The method involves segmenting key values into ranges for each table, generating a unique permutation of segment values to create a sort key, and adding this sort key to each record to facilitate sorting and storage, allowing for sequential access to database tables, thereby improving processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If bulk data processing is performed with random access to multiple database tables, then data processing capability is maintained, but processing time increases significantly
Solution Approach 1:
The patent applies preliminary action by pre-sorting the input data records according to the sorting order of database table keys before bulk processing. This preliminary sorting arrangement ensures that when records are processed in batches, they access database tables in a sequential rather than random manner, significantly reducing access time while maintaining processing capability.
Solution Approach 2:
The patent segments the bulk data processing into batches of records, where each batch is sorted according to the sorting order of database table keys. This segmentation allows the system to process data in manageable portions while maintaining efficient sequential access patterns to the database, resolving the contradiction between processing capability and time loss.
2Speed
If input data is sorted according to database table key sorting order, then access speed to database tables is improved, but additional sorting processing is required
Solution Approach 1:
The patent performs sorting as a preliminary action before the main bulk processing loop. By pre-sorting the input data according to database table key ordering, the system achieves fast sequential access during processing without adding sorting complexity within the processing loop itself.
Solution Approach 2:
The patent utilizes the existing sorting capability of the data processing system to sort input records according to database table key ordering. This self-service approach leverages the system's inherent sorting functions rather than requiring external or complex specialized sorting mechanisms, achieving fast access speed with manageable processing complexity.
3Productivity
If records are processed in batches with sequential access, then processing efficiency is improved, but the ability to handle unsorted input data increases complexity
Solution Approach 1:
The patent applies preliminary sorting to unsorted input data before batch processing. This preliminary action transforms the input data into a sorted structure that enables efficient sequential access during batch processing, resolving the contradiction between processing efficiency and the complexity of handling unsorted input.
Solution Approach 2:
The patent segments the input data into batches and applies sorting within each batch according to database table key ordering. This segmentation approach allows the system to handle unsorted input data while maintaining processing efficiency through sequential access patterns in each batch.
Data Source
AI summary
A method and system for sorting data of an input file containing multiple records associated with multiple tables of a database. The multiple records include key values. The key values are segmented into ranges of key values for each table. Each range of key values for each table is a segment having a segment value. Multiple key values are selected for the multiple records. A block number, which contains a unique permutation of the segment values of the segments, is generated. The segment values denote the ranges of key values encompassing the multiple key values in each record. A sort key value for each record is ascertained, based on the generated block number for each record, and added to each record. The multiple records are sorted according to the sort key values in the multiple records. The sorted multiple records are stored in an output file.