Data Compression Synchronization via History Buffer Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression and decompression methods fail to efficiently identify and utilize redundant data across large volumes, leading to inefficient bandwidth utilization in data transfer over networks and I/O interfaces.
Innovation Solution
An apparatus and method that includes a memory device and a processor to manage data chunks, where redundant data is identified and represented by reference tokens, allowing efficient transmission and reconstruction of data by matching end locations and managing data chunks for compression and decompression processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If existing compression and decompression methods are used, then data transfer is performed, but redundant data is not efficiently identified and transmitted, leading to inefficient bandwidth utilization
Solution Approach 1:
The system performs preliminary actions by maintaining history data buffers at both compression and decompression sides, and pre-establishing synchronization through closing requests and acknowledgments. This allows the system to proactively identify redundant data before transmission, rather than reacting after data is sent, thereby improving bandwidth utilization by avoiding transmission of duplicate information.
Solution Approach 2:
The system uses reference tokens that copy only the essential identification information of redundant data segments rather than copying the actual data content. When redundant data is detected in the history buffer, instead of transmitting the full redundant segment, the system transmits a compact reference token that points to the existing data at the decompression side, dramatically reducing bandwidth consumption while maintaining data transfer efficiency.
2Speed
If data is compressed and transmitted without synchronization, then transmission speed may be improved, but data consistency between compression and decompression sides cannot be ensured
Solution Approach 1:
The system implements a feedback mechanism where the decompression side sends closing requests to the compression side, and the compression side responds with closing acknowledgments. This feedback loop ensures that both sides are synchronized about the state of history data buffers, allowing the system to maintain data consistency while operating at high speed. The feedback confirms that redundant data identification is valid and that reference tokens will be correctly interpreted.
Solution Approach 2:
The system performs preliminary synchronization actions by exchanging closing requests and acknowledgments before actual data transmission using reference tokens. This preliminary action establishes a consistent state between compression and decompression sides, ensuring that both parties have the necessary history data available and are synchronized in their understanding of data redundancy, thereby maintaining reliability without sacrificing transmission speed.
3Loss of energy
If redundant data is identified and represented by reference tokens, then bandwidth utilization improves, but additional processing and synchronization overhead is introduced
Solution Approach 1:
The system applies partial action by implementing reference token compression only for data segments that are actually redundant, rather than applying a complex compression algorithm to all data uniformly. The synchronization overhead (closing requests and acknowledgments) is incurred only when needed to establish or update the history data state, not continuously. This selective approach reduces the overall processing burden while maintaining high bandwidth utilization for redundant data.
Solution Approach 2:
The system changes the parameter representation of data from full data segments to compact reference tokens when redundancy is detected. Instead of transmitting actual data bytes, the system transmits reference tokens with much smaller size that encode only the necessary identification information. This parameter change dramatically improves bandwidth utilization while the synchronization overhead is minimized by using simple acknowledgment protocols rather than complex coordination mechanisms.
Data Source
AI summary
An apparatus comprises a memory device that stores first history data including a first portion associated with a first end location, and at least one processor configured to receive a closing request from a recipient device storing second history data, indicating that a second portion of the second history data is closing from accepting additional data, and comprising a second end location associated with the second portion; responsive to receiving the closing request, determine whether the first end location matches the second end location; responsive to a matching, enter a confirmation state where the first portion is closed from accepting additional data, and transmit a closing acknowledgement to the recipient device, indicating that the first portion has entered the confirmation state, and allowing the recipient device to close the second portion in response to receiving the closing acknowledgement.