Machine Learning Data Compression with Parallel Error Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional lossless data compression techniques, such as Lempel-Ziv based algorithms, suffer from sequential processing that leads to linear run time and are inefficient for data lacking repetition.
Innovation Solution
Utilizing machine learning techniques, including neural networks, to perform parallel lossy compression and error prediction/correction, resulting in a losslessly compressed data format.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Lempel-Ziv based sequential compression algorithms are used, then lossless compression is achieved, but the run time becomes linear and processing efficiency deteriorates
Solution Approach 1:
The patent divides the data stream into fixed-size blocks and processes each block independently using parallel operations. This segmentation enables simultaneous processing of multiple blocks, transforming the linear sequential processing into parallel operations that achieve logarithmic run time while maintaining lossless compression through systematic error tracking and correction mechanisms.
Solution Approach 2:
The patent replaces traditional mechanical sequential processing algorithms with machine learning models (neural networks) that can perform compression and decompression operations in parallel. These ML models process multiple data blocks simultaneously, substituting the sequential mechanical processing with intelligent parallel computation that achieves both lossless compression and improved processing efficiency.
2Loss of substance
If Lempel-Ziv based compression algorithms are used, then repeated patterns in data are exploited, but data lacking repetition yields little to no compression benefit
Solution Approach 1:
The patent changes the fundamental parameter of how data is analyzed by using machine learning models that can detect various patterns and relationships in data beyond simple repetition. These models adapt their processing based on the characteristics of each data block, enabling effective compression of diverse data types including those without obvious repeated patterns by learning from the statistical properties and structures within each block.
Solution Approach 2:
The patent creates a universal compression system using machine learning models that can handle multiple data types and patterns simultaneously. The ML-based compressor and decompressor are designed to work effectively on various data characteristics (repeated patterns, random data, structured data) making the system adaptable to diverse data types while maintaining compression efficiency through parallel block processing.
Data Source
AI summary
Disclosed herein are embodiments for compressing data. A first encoding, a decoding, and an error prediction index are received from one or more artificial neural networks. The first encoding corresponds to a lossy compression of the data. The decoding corresponds to a decompression of the first encoding. The error prediction index indicates one or more locations of predicted error in the decoding. Based on the data and the error prediction index, a first set of bits is generated to include one or more bit values of the data at the one or more locations of predicted error. Based on the error prediction index and the decoding, a second set of bits is generated to indicate one or more locations of unpredicted error in the decoding. The first encoding, the first set of bits, and the second set of bits are stored as a losslessly compressed version of the data.