Iterative Data Pattern Detection Using Dual Buffer XOR Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems require full data comparison to achieve complete pattern detection accuracy, which is inefficient and resource-intensive, especially when detecting repetitive patterns within user data pages.
Innovation Solution
A method that iteratively loads chunks of data into buffers, performs XOR and OR operations to identify repetitive patterns, and reduces data size to metadata, allowing for efficient pattern detection and storage representation within the storage system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full data comparison is performed to achieve complete pattern detection accuracy, then pattern detection accuracy is improved, but processing time and computational resources increase
Solution Approach 1:
The patent segments data into fixed-size chunks and processes them in parallel using multiple buffers. By dividing the data stream into manageable segments that can be independently processed and compared, the system achieves comprehensive pattern detection without requiring sequential full-data comparison, thereby reducing processing time while maintaining detection accuracy.
Solution Approach 2:
The patent performs preliminary hashing on data chunks before full comparison. By pre-processing data to generate hash values and using these for initial pattern identification, the system filters out non-matching patterns early in the process, avoiding expensive full-data comparisons and reducing overall processing time while preserving detection accuracy.
2Reliability
If full data is stored to ensure complete pattern information, then pattern detection completeness is improved, but storage space consumption increases
Solution Approach 1:
The patent extracts only the essential pattern information from full data chunks and stores merely the metadata representing detected patterns. By separating the pattern identification function from the full data storage requirement, the system maintains complete pattern detection capability while storing only the compressed pattern metadata, significantly reducing storage space consumption.
Solution Approach 2:
The patent creates simplified representations (copies) of pattern data in metadata form rather than storing complete original data. These metadata copies contain sufficient information for pattern detection and reconstruction purposes, enabling the system to maintain pattern detection completeness while using minimal storage space through efficient data representation.
3Productivity
If data is processed in larger chunks to reduce iteration overhead, then processing efficiency is improved, but memory buffer requirements increase
Solution Approach 1:
The patent divides data into fixed-size chunks that fit within standard buffer configurations (e.g., 4KB pages). By segmenting data into uniform, manageable units, the system achieves high processing efficiency through consistent chunk sizes while keeping buffer memory requirements predictable and controllable, avoiding the need for excessively large buffers.
Solution Approach 2:
The patent implements a hierarchical buffer structure where multiple buffers operate at different levels of the processing pipeline. Data flows through nested buffer stages (input buffer, processing buffer, output buffer), allowing efficient data handling with moderate buffer sizes at each level rather than requiring one extremely large buffer, thus improving processing efficiency while controlling memory usage.
Data Source
AI summary
A method, computer program product, and computing system for loading a first chunk of data from a user data page into a first buffer. Each successive chunk of data from the user data page may be iteratively loaded into a second buffer. The first chunk of data in the first buffer may be iteratively compared with each successive chunk of data in the second buffer. In response to determining that the first chunk of data in the first buffer is equivalent to each successive chunk of data in the second buffer, the first chunk of data may be defined as a pattern.