Shared Dictionary Compression for Variable-Length Data Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression systems face challenges in efficiently managing dictionaries for data blocks of varying lengths, leading to increased storage requirements and reduced performance due to the large number of dictionaries needed.
Innovation Solution
A data compression system that uses shared hash tables and dictionaries across different data lengths, reducing the number of tables and dictionaries required by dividing input data into blocks of varying lengths and utilizing a shared structure for hash tables and compression dictionaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the number of patterns of data lengths is increased to improve compression performance, then compression accuracy is improved, but the number of dictionaries increases and physical storage area required increases
Solution Approach 1:
The patent creates a shared dictionary that serves multiple data length patterns simultaneously. Instead of having separate dictionaries for 2-byte, 4-byte, 8-byte, and 16-byte patterns, a single shared dictionary is used that can handle all these patterns, thereby reducing the total number of dictionaries while maintaining compression accuracy for all data lengths.
Solution Approach 2:
The patent merges multiple separate dictionaries into one shared dictionary structure. By combining the functionality of multiple dictionaries into a single unified structure, the system reduces the quantity of storage required while preserving the ability to compress data of various lengths effectively.
2Adaptability or versatility
If the number of dictionaries is increased to handle more data length patterns, then adaptability to different data patterns is improved, but device complexity increases
Solution Approach 1:
The shared dictionary is designed to be universal, handling multiple data length patterns (2-byte, 4-byte, 8-byte, 16-byte) within a single structure. This universal approach maintains adaptability to different data patterns while reducing device complexity by eliminating the need for multiple separate dictionaries.
Solution Approach 2:
The patent segments the shared dictionary into multiple areas, each dedicated to handling specific data length patterns. This segmentation allows the single shared dictionary to efficiently manage different data lengths by directing each pattern to its appropriate area, maintaining versatility without increasing overall complexity.
3Productivity
If separate dictionaries are used for each data length pattern, then compression performance for specific patterns is optimized, but storage efficiency decreases
Solution Approach 1:
The patent merges multiple separate dictionaries into one shared dictionary, improving storage efficiency by eliminating redundant storage. The shared dictionary maintains compression performance for specific patterns by providing dedicated areas within the shared structure for each data length pattern, thus achieving both storage efficiency and compression performance.
Solution Approach 2:
The shared dictionary implements local quality by creating distinct areas within the shared structure, where each area is optimized for specific data length patterns. This allows each pattern to receive specialized handling within the unified dictionary, maintaining high compression performance while using a single shared storage structure.
Data Source
AI summary
A divider divides an input data into a plurality of data blocks. A hash calculator calculates a hash value of each data block. A compression dictionary memory stores a compression dictionary that stores a previous input data and includes a shared dictionary shared by different data lengths. A hash table memory stores a hash table that stores an address representing a storage location of the data block corresponding to the hash value on the compression dictionary for each data block and includes a shared table shared by different data lengths. An address acquirer acquires the address corresponding to the data block based on the hash table. A matcher determines sameness between the previous data block indicated by the address and the new input data. An encoder generates a compressed data that includes matching information and a matched portion is converted to the address.