Multi-Range Scan Query Processing with Segmented Buffers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems, such as DBMS, are inefficient in processing N sort queries in multi-range scans, leading to slow processing speeds and high memory usage due to excessive scanning and sorting of records, which is exacerbated by the increasing number of messages and friends, resulting in temporal and spatial costs that are not optimized.
Innovation Solution
A method that allocates buffers based on the number of extracted records to directly obtain sorted results during the scan process, minimizing the number of records scanned and using a buffer allocator and scanner to efficiently process N sort queries in a multi-range scan by comparing data and updating the buffer to satisfy queries, thereby reducing the need for intermediate storage and sorting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all messages of friends are extracted as interim results and sorted in reverse order of creation time using conventional DBMS, then complete sorted results are obtained, but processing speed becomes very slow and huge storage space is needed
Solution Approach 1:
The patent segments the sorting process by maintaining N separate buffers (one for each friend) instead of collecting all messages first. Each buffer independently maintains sorted order for that friend's messages, avoiding the need to sort all messages together. This segmentation reduces the sorting scope from O(Total Messages) to O(N messages per friend), significantly improving processing speed while maintaining complete sorted results.
Solution Approach 2:
The patent applies partial action by extracting only the necessary N most recent messages from each friend rather than extracting and sorting all messages. The scanner stops scanning a friend's messages once N messages are collected in that friend's buffer, avoiding unnecessary processing of older messages. This partial extraction maintains sorted result completeness while dramatically reducing processing time and memory usage.
2Measurement precision
If all messages of friends are extracted as interim results using conventional DBMS, then complete data set is obtained, but huge storage space for storing interim results is needed
Solution Approach 1:
The patent divides the storage requirement into N separate buffers of size N each (one buffer per friend) instead of one large buffer storing all messages. This segmentation reduces total memory usage from O(Total Messages) to O(N²), where N is the number of messages to extract per friend. Each buffer stores only the necessary sorted messages for its associated friend, eliminating the need for huge interim result storage.
Solution Approach 2:
The patent extracts only the necessary N messages from each friend's message set rather than extracting all messages. This partial extraction ensures that each buffer contains only the messages needed for the final sorted result, minimizing memory consumption while maintaining data completeness for the query requirements.
3Measurement precision
If sorting is performed on a large number of records in conventional DBMS, then complete sorted results are obtained, but time and space waste is increased
Solution Approach 1:
The patent segments the sorting operation into N independent sorting processes (one per friend's buffer) instead of one large sorting operation on all messages. Each buffer performs sorting on only N messages from its associated friend, reducing the sorting complexity from O(Total Messages log Total Messages) to O(N² log N). This segmentation maintains accurate sorted results while dramatically reducing sorting time.
Solution Approach 2:
The patent performs sorting only on the necessary N messages from each friend rather than sorting all messages. This partial sorting approach ensures that each buffer contains accurately sorted messages in reverse chronological order, achieving sorting accuracy where needed while avoiding waste of time on sorting messages that will not be included in the final result.
Data Source
AI summary
Disclosed is a method and an apparatus for optimally processing N sort queries in a multi-range scan, including: allocating a buffer based on a first number of extracted records included in the queries; extracting first data related to first attributes in a first list included in the queries and storing the extracted first data in the buffer; extracting second data related to the first attributes that are not extracted in the first list included in the queries; and updating the buffer so as to satisfy the queries by comparing the data stored in the buffer with the second data, wherein the queries are N sort queries in a multi-range scan for at least one record sorted based on the first attributes and second attributes.