Dynamic Recovery Chunk Scaling for Reliable Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data transmission methods across unreliable networks face challenges in efficiently recovering lost chunks, particularly for time-sensitive information, as complete redundancy increases bandwidth overhead and no redundancy compromises data quality.
Innovation Solution
Generating a recovery chunk based on information from a group of chunks, using a parity algorithm to recover a single lost chunk, with the group size dynamically adjusted based on historical chunk loss data to balance bandwidth usage and recovery efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complete redundancy is used to recover lost chunks, then data reliability is improved, but bandwidth overhead increases and receiver congestion occurs
Solution Approach 1:
Instead of providing complete redundancy (100% backup), the patent applies partial redundancy by generating a single recovery chunk from a group of N chunks. This partial action provides sufficient protection against single chunk loss while consuming significantly less bandwidth than complete redundancy would require.
Solution Approach 2:
The patent merges multiple chunks together through a parity algorithm to create a single recovery chunk. This combining approach allows the recovery chunk to contain distributed information from all N chunks, enabling any single lost chunk to be reconstructed when combined with the remaining N-1 chunks.
2Quantity of substance
If no redundancy is used to reduce overhead, then bandwidth efficiency is improved, but data loss recovery capability deteriorates
Solution Approach 1:
The patent applies minimal sufficient redundancy by transmitting only one recovery chunk per group of N chunks rather than complete redundancy. This partial redundancy provides just enough protection for single chunk loss scenarios while maintaining high bandwidth efficiency.
Solution Approach 2:
The system dynamically adjusts the group size N based on historical chunk loss data, changing the redundancy parameter adaptively. When loss rates are low, larger groups reduce overhead; when loss rates increase, smaller groups provide stronger protection, optimizing the balance between efficiency and reliability.
3Quantity of substance
If the group size is increased to reduce the number of recovery chunks, then bandwidth overhead decreases, but the ability to handle multiple simultaneous losses deteriorates
Solution Approach 1:
The patent implements dynamic adjustment of group size based on observed chunk loss patterns. The system transitions from static group sizing to adaptive sizing, modifying the group parameter in response to changing network conditions and historical loss data to optimize both overhead and recovery capability.
Solution Approach 2:
The system uses historical chunk loss data as feedback to continuously optimize group size. By monitoring actual loss patterns and adjusting the group parameter accordingly, the system learns from past performance and adapts to minimize overhead while maintaining adequate recovery capability for observed loss rates.
Data Source
AI summary
A recovery chunk is generated based on information found in each chunk in a group of chunks. For example, the group of chunks includes three chunks of which information in the three chunks is used to generate a recovery chunk. The recovery chunk is then used to recover a single lost chunk in the group of chunks. The three chunks and the recovery chunk are sent to a receiver over a network. If one of the three chunks is lost, the recovery chunk is used to generate the lost chunk. The recovery chunk is used to recover a single lost chunk, however, the recovery chunk is not used to recover more than one lost chunk in the group. The number of chunks is also dynamically adjusted based on chunk loss at a receiver. This adjustment is based on the bandwidth used and reliability desired.