Storage System Encoder with Adaptive Compression Scale
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network-based encoders and decoders for data compression and decompression have high calculation processing loads due to uniformly processing all data, leading to increased resource requirements and costs, and the need for large-scale neural networks to handle diverse data types.
Innovation Solution
A storage system that determines the compression operation scale based on data features, using a selector to choose between different compression routes for each data portion, allowing for efficient compression and decompression by allocating appropriate processing loads based on data complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a neural network-based encoder and decoder are used for data compression, then data reduction ratio is improved, but calculation processing load increases
Solution Approach 1:
The patent divides the input data into multiple pieces and processes each piece independently through the neural network encoder. This segmentation allows the system to handle complex data with high compression ratios while using simple compression methods for simpler data portions, thereby reducing the overall calculation processing load while maintaining effective data reduction
Solution Approach 2:
The patent applies different compression strategies to different data pieces based on their local characteristics. Complex data pieces receive neural network-based compression with higher processing load, while simpler pieces use standard compression methods with lower processing load, optimizing the balance between compression effectiveness and computational cost
2Adaptability or versatility
If a large-scale neural network is used to handle diverse data, then adaptability is improved, but calculation processing load increases
Solution Approach 1:
By segmenting data into multiple pieces, the system can apply appropriately-sized neural networks to each segment based on its complexity, rather than using a single large-scale network for all data. This maintains adaptability to diverse data types while reducing the overall calculation processing load
Solution Approach 2:
The patent applies neural network processing selectively to data pieces that benefit most from it, rather than uniformly processing all data with a large-scale network. This partial application of complex processing maintains adaptability for diverse data while avoiding unnecessary computational overhead for simpler data portions
3Ease of operation
If uniform processing is applied to all data, then simplicity of operation is improved, but calculation processing load increases
Solution Approach 1:
The patent introduces dynamic processing where the compression method is adjusted based on data characteristics. The system automatically determines which data pieces require neural network processing and which can use standard methods, creating a flexible operational approach that reduces calculation load while maintaining ease of operation through automated decision-making
Data Source
AI summary
To reduce a calculation processing load as a whole while realizing a small amount of data loss for at least one of compression and decompression. For each of a plurality of pieces of data, a storage system determines a compression operation scale of the data based on a feature of the data, executes a lossy compression operation according to the determined compression operation scale to covert the data into encoded data, and stores the encoded data or compressed data thereof into a storage device.