Parallel Decompression of Executable Files With Data Dependencies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in accelerating the decompression of compressed executable files, particularly those using algorithms like Gzip, which are considered impossible to parallelize due to data dependencies, resulting in noticeable delays during application launch and boot times on mobile devices.
Innovation Solution
A system and method for parallel decompression of compressed files using multiple processor cores, where decompression processors handle partial decompression of blocks, and aggregation and merge processors sequence and merge data to form an uncompressed file, leveraging metadata and pipelined stages to efficiently manage data dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If compressed executable files are decompressed using traditional serial methods, then data integrity is maintained, but decompression time is excessive and noticeable to users
Solution Approach 1:
The compressed file is divided into multiple independent blocks that can be decompressed in parallel. Each block contains self-contained data with local references, allowing simultaneous processing by multiple processor cores while maintaining overall data integrity through proper sequencing of the final output.
Solution Approach 2:
Metadata including block boundaries, lengths, and reference information is extracted and prepared in advance during the decompression process. This preliminary organization of data structures enables subsequent parallel processing stages to operate independently and efficiently without data dependencies blocking progress.
2Quantity of substance
If compression algorithms like Gzip with run length encoding are used to reduce file size, then storage space is optimized, but data dependencies across blocks make parallel decompression impossible
Solution Approach 1:
The compressed file structure is segmented into blocks with explicit metadata describing each block's contents, boundaries, and internal references. This segmentation transforms the traditionally sequential Gzip format into a structure where multiple blocks can be independently processed in parallel while preserving the compression efficiency of the original algorithm.
Solution Approach 2:
Metadata acts as an intermediary layer between the compressed data blocks and the decompression processors. This metadata contains pre-calculated information about block structures, lengths, and references, eliminating the need for processors to sequentially analyze and interpret compressed data, thereby enabling parallel processing while maintaining Gzip compression integrity.
3Productivity
If multiple processor cores are used for parallel decompression, then decompression speed is increased, but system complexity increases due to coordination requirements
Solution Approach 1:
The decompression system is segmented into distinct functional stages (metadata extraction, block distribution, parallel decompression, result aggregation) that can be independently implemented and optimized. Each processor core is assigned specific blocks with clear instructions, reducing coordination overhead while maximizing parallel processing efficiency.
Solution Approach 2:
Each decompression processor core independently processes its assigned blocks using the pre-provided metadata, without requiring continuous coordination or synchronization with other cores. The metadata contains all necessary information for each processor to autonomously decompress its blocks, minimizing inter-processor communication and reducing system complexity.
Data Source
AI summary
Representative embodiments are disclosed for a rapid and highly parallel decompression of compressed executable and other files, such as executable files for operating systems and applications, having compressed blocks including run length encoded (“RLE”) data having data-dependent references. An exemplary embodiment includes a plurality of processors or processor cores to identify a start or end of each compressed block; to partially decompress, in parallel, a selected compressed block into independent data, dependent (RLE) data, and linked dependent (RLE) data; to sequence the independent data, dependent (RLE) data, and linked dependent (RLE) data from a plurality of partial decompressions of a plurality of compressed blocks, to obtain data specified by the dependent (RLE) data and linked dependent (RLE) data, and to insert the obtained data into a corresponding location in an uncompressed file. The representative embodiments are also applicable to other types of data processing for applications having data dependencies.