Variable-Sized Data Chunks for SSD Update Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in efficiently and securely delivering application updates over SSDs while minimizing writes and erasures, as SSDs have limited longevity due to write and erase cycles, and existing methods do not effectively balance the interests of marketplace operators and developers for efficient and secure delivery.
Innovation Solution
Implementing variable-sized data chunks for application and update data, which allows for more efficient updates by using rolling hashes, compression, and remote triggering of data coalescence to reduce fragmentation and minimize writes, while ensuring secure encryption and metadata management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If fixed-size data blocks are used for application updates, then encryption and compression can be applied effectively, but the number of writes and erasures on SSD increases, reducing SSD longevity
Solution Approach 1:
The patent divides application data into variable-sized chunks rather than fixed-size blocks. This segmentation allows the system to identify and update only the specific variable-sized chunks that have changed, rather than rewriting entire fixed-size blocks. The variable-sized chunks are then packed into fixed-size encrypted blocks only when necessary, reducing the number of write operations on SSD while maintaining security through encryption.
2Duration of action of stationary object
If variable-sized data chunks are used for updates, then the number of writes and erasures is reduced, but change detection becomes more complex
Solution Approach 1:
The patent pre-computes and stores hash values (such as CRC or MD5) for each variable-sized chunk during the initial application packaging phase. These pre-computed hashes are stored in a manifest or metadata structure. During update operations, the system only needs to compute hashes of the new data and compare them against the stored hashes to detect changes, rather than performing complex byte-by-byte comparisons of variable-sized regions.
3Productivity
If data is compressed for efficient delivery, then transmission efficiency improves, but decompression and verification overhead increases
Solution Approach 1:
The patent compresses data at the variable-sized chunk level rather than compressing entire fixed-size blocks. This granular compression approach allows the system to decompress and verify only the specific variable-sized chunks that have changed during an update, rather than decompressing entire blocks. This significantly reduces the decompression time and computational overhead while maintaining high compression ratios for efficient delivery.
Data Source
AI summary
A method, system and non-transitory computer readable instructions for data change detection includes decrypting encrypted patch data that includes compressed variable sized patch data chunks. The compressed variable sized patch data chunks are decompressed and a strong hash value is generated. The strong hash values are compared with strong hash values of corresponding variable sized application data chunks to determine a potentially non-referable patch data chunk. A bloom filter is generated for the variable sized application data chunks. A rolling hash value for a window of the potentially non-referable patch data chunk is used to determine a potentially referable area from the potentially non-referable patch data chunk. The potentially referable area is designated as referable when a weak hash and a strong hash of an application data chunk matches the weak hash and the strong hash of the potentially referable area.