File Transfer Chunking with Priority Metadata for Unstable Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Transferring large files between computers in unstable network environments often results in interrupted communications, leading to loss of progress and inefficient resource usage, as existing methods require restarting and resending of files.
Innovation Solution
A method involving HTTP connections where a server computer receives metadata from a client computer, determines missing file parts, and requests specific chunks to be resent, allowing for dynamic chunk sizes and efficient reassembly of files without requiring a full file resend after connection loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire file is resent after connection loss, then the file transfer can be completed, but resource waste and transfer time increase significantly
Solution Approach 1:
The patent divides the file into multiple chunks with different priorities (first chunk, second chunk, third chunk). When connection is lost, only the necessary high-priority chunks need to be resent rather than the entire file, reducing resource waste while ensuring transfer completion.
Solution Approach 2:
The patent performs preliminary actions by sending high-priority chunks first and establishing priority markers before connection loss occurs. This preliminary segmentation and prioritization enables efficient recovery by only resending critical portions rather than the complete file.
2Reliability
If the entire file is resent after connection loss, then the file transfer can be completed, but transfer time increases significantly
Solution Approach 1:
The file is segmented into priority-based chunks allowing selective retransmission. Only high-priority chunks need to be resent after connection loss, significantly reducing the time required compared to resending the entire file while still ensuring complete transfer completion.
Solution Approach 2:
High-priority chunks are transmitted first as a preliminary action before connection loss. This enables faster recovery by focusing retransmission efforts only on the remaining necessary portions rather than restarting the complete file transfer.
3Reliability
If high-priority chunks are sent first with priority markers, then essential file parts are ensured to be received, but communication protocol complexity increases
Solution Approach 1:
The communication protocol is simplified by segmenting the file into priority-based chunks rather than implementing complex error correction or full retransmission protocols. Each chunk carries its priority information, enabling straightforward selective retransmission without elaborate protocol mechanisms.
Solution Approach 2:
The patent uses simple priority markers and chunk identifiers as metadata copies attached to each data chunk. This lightweight copying approach enables protocol simplicity by using basic identification tags rather than complex protocol handshakes or acknowledgment mechanisms.
4Productivity
If the file is divided into multiple chunks, then selective retransmission is enabled, but data structure complexity increases
Solution Approach 1:
The file is divided into chunks with simple priority levels (first, second, third) rather than complex segmentation schemes. This basic segmentation enables selective retransmission for improved transfer efficiency while maintaining relatively simple data structures through straightforward priority classification.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A file with a unique filename is transferred from a client computer to a server computer in chunks, which can be packaged into packets including metadata. The server computer can record indices of received chunks in a local metadata file associated with the file. Upon receiving the filename in a subsequent client request after a connection loss, the server computer can use the associated metadata file to determine whether portions of the file are missing. If so, the server computer can send indices corresponding to missing ranges instead of the client computer resending the whole file.