Variable Length Key Sorting via Extraction and Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data sorting techniques for objects with variable length sort keys are inefficient due to the need for padding and unpadding, which results in significant data expansion, increased storage requirements, and prolonged processing times, especially for large datasets like databases.
Innovation Solution
The method involves extracting key information, expanding it, and storing it in key records for sorting, without storing the expanded data in intermediate storage, using sort control cards to identify key fields, and reorganizing data records based on sorted key information, reducing data movement and storage needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If variable length sort keys are padded to fixed length for sorting, then sorting can be performed using standard sort routines, but the amount of intermediate storage required increases significantly
Solution Approach 1:
The patent segments the sorting process into two distinct phases: key extraction and data reorganization. Only the sort keys (not the entire records) are padded and stored in intermediate storage during the first phase. This segmentation allows standard fixed-length sort routines to be used while minimizing the volume of data requiring intermediate storage.
Solution Approach 2:
The patent extracts only the sort key fields from each record, pads them to fixed length, and stores only these extracted keys in intermediate storage. The full records are kept in their original locations and are not moved to intermediate storage, thereby dramatically reducing the quantity of substance requiring intermediate storage while still enabling standard sorting operations.
2Productivity
If padded records are written to and read from intermediate storage, then sorting can be completed, but the processing time increases significantly
Solution Approach 1:
The patent extracts and pads only the sort keys rather than entire records, and stores only these compact key structures in intermediate storage. This extraction approach dramatically reduces the volume of data that must be written to and read from intermediate storage, thereby reducing the data transfer time while still completing the sorting operation.
Solution Approach 2:
The patent segments the data into sort keys and record data, processing only the keys through the time-consuming pad/write/read/unpad cycle. The record data remains in its original location throughout the sort operation, being reorganized only after the keys are sorted. This segmentation eliminates unnecessary data transfer time for the bulk of the data.
3Adaptability or versatility
If the maximum supported key size is used for all records, then all variable length keys can be accommodated, but the average key size increases
Solution Approach 1:
The patent applies local quality by padding each sort key only to its specific required length (up to the maximum supported size) rather than uniformly padding all keys to the maximum size. This allows the data structure to adapt locally to each key's actual requirements, accommodating variable length keys effectively while minimizing the average key size across the entire dataset.
Data Source
AI summary
Techniques for sorting data include (1) obtaining records containing variable length key fields, (2) expanding the variable length key fields into fixed length key fields in a temporary structure, (3) sorting the temporary structure based on the fixed length key fields, and (4) reorganizing the data containing the variable length key fields in accordance with the sorted temporary structure. Reorganized data can replace the original data or be saved to temporary storage where it can later be merged with other reorganized data. Temporary storage of reorganized data does not incorporate the expanded fixed length key information used in the temporary structure.