Compression Library Scheduling for Multi-Mode Lossless Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression technologies are limited as one acceleration device can only implement one lossless compression mode, lacking adaptability to different compression algorithms.
Innovation Solution
A data compression method and apparatus that utilize a compression system with a scheduling device and acceleration devices, where the system selects and implements specific compression algorithms from a library based on request identifiers, enabling multiple lossless compression modes to be supported by a single acceleration device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If one acceleration device is dedicated to implement one lossless compression mode, then the implementation is simple and reliable, but the adaptability to different compression algorithms is poor
Solution Approach 1:
The acceleration device is designed to support multiple lossless compression modes (lz4, zlib, zstd) within a single device by loading different compression algorithms from a compression library based on the compression mode identifier in the request, making the device universal and adaptable to different compression requirements
Solution Approach 2:
The acceleration device dynamically loads and switches between different compression algorithms from the compression library based on the received compression mode identifier, allowing the device to adapt its functionality dynamically rather than being fixed to a single compression mode
2Adaptability or versatility
If multiple compression algorithms are integrated into one acceleration device, then the adaptability improves, but the device complexity increases
Solution Approach 1:
A compression library is introduced as an intermediary component that stores multiple compression algorithms (lz4, zlib, zstandard) and provides them to the acceleration device through a unified interface, reducing the complexity burden on the acceleration device itself while still enabling support for multiple compression modes
Solution Approach 2:
The compression system is segmented into distinct components: the acceleration device that handles execution, the compression library that stores algorithms, and the scheduling device that manages requests. This segmentation allows each component to have a specific function, reducing overall system complexity while maintaining multi-mode support
Data Source
AI summary
A compression system includes at least one acceleration device, a memory of the acceleration device stores a compression library, and the compression library includes one or more compression algorithms that support implementation of a lossless compression mode. The lossless compression mode is split into one or more fine-grained compression algorithms. Therefore, when obtaining any compression request, the compression system may select, based on an identifier that is carried in the compression request and that indicates a lossless compression mode, a compression algorithm corresponding to the lossless compression mode, thereby performing compression processing on data in the specified lossless compression mode by using the compression algorithm corresponding to the identifier.