Tiered Storage Compression Selection for Unstructured Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing re-compression schemes for unstructured data require user participation and are inadequate for meeting the compression needs of diverse data types, often leading to data loss and inefficiency.
Innovation Solution
A data processing method and apparatus that autonomously determines a compression algorithm based on tiered storage features and data characteristics, allowing for efficient re-compression of unstructured data without user intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing re-compression schemes are used, then user participation is required and compression requirements can be met, but user burden increases and operation complexity worsens
Solution Approach 1:
The system automatically analyzes data features and selects appropriate compression algorithms without user intervention. The processing apparatus autonomously determines compression parameters and executes re-compression based on data characteristics, eliminating the need for users to understand compression knowledge or manually configure settings.
Solution Approach 2:
The system dynamically adjusts compression parameters based on data features such as data type, dimension, size, and content characteristics. Different compression algorithms and parameters are automatically selected according to the specific properties of the unstructured data being processed.
2Ease of manufacture
If existing re-compression schemes are used, then specific algorithm support is provided, but adaptability to diverse data types deteriorates
Solution Approach 1:
The system is designed to handle multiple types of unstructured data including office documents, HTML, images, audio, and video data. It provides a universal re-compression capability that adapts to different data formats through automatic feature analysis and algorithm selection, rather than requiring separate solutions for each data type.
Solution Approach 2:
The system changes compression parameters and algorithm selection based on data features such as data type, dimension, size, and content characteristics. This allows the same system to effectively compress diverse data formats by adapting its compression approach to match the specific properties of each data type.
3Manufacturing precision
If manual compression selection is used, then compression control is precise, but time consumption and productivity worsen
Solution Approach 1:
The system automatically analyzes data features and selects appropriate compression algorithms without user intervention. The processing apparatus autonomously determines compression parameters and executes re-compression based on data characteristics, eliminating the need for users to understand compression knowledge or manually configure settings.
Solution Approach 2:
The system performs preliminary analysis of data features including data type, dimension, size, and content characteristics before compression. This preliminary characterization enables the system to pre-select the most appropriate compression algorithm and parameters, streamlining the subsequent compression process.
Data Source
AI summary
This application provides a data processing method and apparatus. The method is applied to a storage system. The storage system includes a storage apparatus and a processing apparatus. The method is performed by the processing apparatus. The method includes: obtaining a tiered storage feature and a data feature of first data, where the tiered storage feature includes at least one of the following features: an importance, an access frequency, and a retention time, and the data feature includes at least one of the following features: a data type, a data dimension, a data size, or a data content feature; determining a first compression algorithm based on the tiered storage feature and the data feature; and compressing the first data based on the first compression algorithm, to obtain compressed data.