Hashed Delta Compression for Low-Memory Patch Generation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional delta compression systems consume significant computing power, storage bandwidth, and storage capacity, leading to increased operational costs and reduced efficiency in data storage and transmission.

Innovation Solution

A computer-implemented method for delta compression that involves hashing portions of source and target data files, generating hashing tables, and creating patch files with instructions to efficiently determine and store differences between files, using hashing functions like djb2 and CRC32 to minimize resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of substance

If conventional delta compression mechanisms are used, then data compression is achieved, but computing power consumption increases significantly

Engineering Contradiction:

Improvedata redundancyVSAvoidcomputing power consumption

Core Design Contradiction:

Loss of substanceVSUse of energy by moving object

Solution Approach 1:

The patent divides the data file into multiple portions and creates a hashing table where each portion is hashed and stored at a specific offset location. This segmentation allows for efficient comparison of only relevant portions between source and target files, reducing the overall computing power needed for delta compression while maintaining effective data redundancy reduction.

Inventive Principle:

Principle #1Segmentation

2Loss of substance

If conventional delta compression mechanisms are used, then data compression is achieved, but storage bandwidth consumption increases significantly

Engineering Contradiction:

Improvedata redundancyVSAvoidstorage bandwidth consumption

Core Design Contradiction:

Loss of substanceVSLoss of energy

Solution Approach 1:

The patent performs preliminary hashing of the source file portions and stores the hashed values in a hashing table with offset locations before the actual delta compression process. This preliminary action enables quick identification of matching portions during compression, significantly reducing the storage bandwidth consumption during the main compression operation by avoiding repeated full-file comparisons.

Inventive Principle:

Principle #10Preliminary action

3Loss of substance

If conventional delta compression mechanisms are used, then data compression is achieved, but storage capacity consumption increases significantly

Engineering Contradiction:

Improvedata redundancyVSAvoidstorage capacity

Core Design Contradiction:

Loss of substanceVSVolume of stationary object

Solution Approach 1:

The patent creates a hashing table that contains copied hashed portions of the source file at specific offset locations. This copying mechanism allows the system to reference and compare portions efficiently without storing multiple complete copies of the entire source file, thereby reducing storage capacity consumption while maintaining the ability to perform effective delta compression.

Inventive Principle:

Principle #26Copying

4Loss of substance

If conventional delta compression mechanisms are used, then data compression is achieved, but operational costs increase

Engineering Contradiction:

Improvedata redundancyVSAvoidoperational costs

Core Design Contradiction:

Loss of substanceVSObject-generated harmful factors

Solution Approach 1:

By segmenting the data into portions and creating a structured hashing table with offset locations, the patent enables more efficient processing of delta compression operations. This segmentation reduces the computational resources required, decreases storage bandwidth and capacity consumption, and ultimately lowers operational costs associated with data compression infrastructure.

Inventive Principle:

Principle #1Segmentation

5Loss of substance

If conventional delta compression mechanisms are used, then data compression is achieved, but efficiency of networks and data storage facilities is reduced

Engineering Contradiction:

Improvedata redundancyVSAvoidefficiency of networks and data storage facilities

Core Design Contradiction:

Loss of substanceVSProductivity

Solution Approach 1:

The patent performs preliminary hashing and creates a structured hashing table with offset locations before the main compression process. This preliminary action prepares the data in an optimized format that enables faster comparison and processing during delta compression, significantly improving the efficiency of networks and data storage facilities by reducing the time and resources needed for compression operations.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentEP3525352B1Delta compression

Publication Date: 2021.07.28 EXAGRID SYST

Figure 1
Figure 2a
Figure 2b

AI summary

Delta compression method, system and computer program product. Portions of source and target data files are hashed using a hashing function. A target data file is compared against the source data file to determine at least one delta difference between the files. A source data file hashing table is generated. The table includes hashed portions of the source and target data files stored in corresponding source file offset locations and corresponding target file offset locations, respectively. Portions of the source and target files are compared using corresponding source and target file offset locations. At least one common sequence of characters in the portions of the source and target files is determined based on the comparison. A patch file is generated based on the determined sequence of characters.