Archive-Specific Tokenization Engine for Minimizing Diff Archive Size
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing size of software applications and the number of computing devices lead to high bandwidth, storage, and processing costs for software updates, particularly affecting devices with limited resources like wearable computing devices and IoT devices, as existing diff update methods are inefficient with compressed files and non-deterministic data ordering.
Innovation Solution
The techniques generate minimally sized diff archives by using an archive-specific tokenization engine to identify data chunks and payload files, creating recipes for generating and decompressing files, and applying binary deltas, which reduces the size of updates by reusing data from the source archive, thus minimizing bandwidth, storage, and processing requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If block-based diff update methods are used, then software updates can be distributed, but bandwidth costs and update size increase due to inefficiency with compressed files and non-deterministic data ordering
Solution Approach 1:
The patent segments the update process into two distinct phases: (1) generating a diff archive that contains only the differences between source and target archives, and (2) applying the diff archive to reconstruct the target archive. This segmentation allows the diff archive to be much smaller than the full update, reducing bandwidth costs while maintaining update completeness.
Solution Approach 2:
The patent creates a diff archive that copies only the necessary difference information from the source archive to the target archive, rather than copying the entire target archive. The diff archive contains minimal data needed to transform the source archive into the target archive, significantly reducing the quantity of data that needs to be transmitted.
2Ease of operation
If traditional diff methods are used, then updates can be applied, but processing costs and installation times increase for devices with limited resources
Solution Approach 1:
The patent performs preliminary actions during the diff archive generation phase by pre-computing and storing the difference information in an optimized format. The diff archive is prepared in advance with all necessary transformation instructions, so that during installation, the device with limited resources only needs to apply these pre-computed differences rather than performing complex diff calculations locally.
3Quantity of substance
If archive-specific tokenization is used to identify data chunks and payload files, then diff archives can be minimized, but device complexity increases
Solution Approach 1:
The patent introduces an archive-specific tokenization engine as an intermediary component that bridges the source archive and target archive. This tokenization engine analyzes the archive structure, identifies data chunks and payload files, and generates the diff information. By concentrating the complexity in this dedicated intermediary component rather than distributing it throughout the entire update system, the patent achieves minimal diff archives while managing device complexity in a controlled manner.
Data Source
AI summary
The techniques disclosed herein generate minimally sized diff archives. As a result, bandwidth, storage, and processing costs of storing or transmitting an archive are reduced. In some configurations, a diff archive is generated utilizing specific knowledge of the structure of the source and target archives it is derived from. Specifically, an archive-specific tokenization engine identifies data chunks and payload files within each archive. Recipes for generating payload files from data chunks and data chunks from payload files are identified and stored in a manifest file, as are recipes for decompressing files, concatenating data chunks, and generating binary deltas that convert older versions of a file into a newer version. These recipes are composed by replacing recipe inputs with the outputs of other recipes. Composite recipes use inline data and data obtained from a copy of the source archive to reconstitute the target archive.