Data Stream Fragmentation for Efficient File Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data transfer methods over networks are slow, especially when handling large amounts of data, leading to inefficiencies in sharing and backing up data among networked users, and can result in data loss due to overwriting or lockouts when multiple users work on the same files simultaneously.
Innovation Solution
A method of storing data in streams of varying sizes, where data is fragmented into blocks and analyzed for content relationships, allowing for efficient transfer and storage by identifying and managing data fragments within a data block structure, using a data stream extension that enables efficient data access and management through a B+ tree indexing system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred over the network in traditional methods, then data can be shared between users, but transfer speed is slow and efficiency is reduced
Solution Approach 1:
The patent divides data into streams of varying sizes and further segments them into data fragments that are written to storage in data blocks. This segmentation allows the system to transfer only the specific data blocks that have changed rather than transferring entire files, significantly improving transfer speed and efficiency when sharing data between networked users.
Solution Approach 2:
The patent extracts and identifies only the changed data fragments from the original data streams. By analyzing data fragments to determine content relationships and placing related fragments in the same block, the system can extract and transfer only the necessary portions of data that have changed, eliminating the need to transfer unchanged data and thereby improving transfer efficiency.
2Reliability
If entire files are transferred for backup, then data can be backed up, but time and bandwidth are wasted transferring unchanged data
Solution Approach 1:
The patent segments data into identifiable streams and fragments organized in blocks, allowing the backup system to process and transfer only the specific blocks that contain changed data. This maintains complete backup reliability while dramatically reducing backup time and network bandwidth consumption compared to traditional full file transfer methods.
Solution Approach 2:
The system analyzes data fragments to determine content relationships and identifies which data blocks have changed. This feedback mechanism allows the backup process to selectively transfer only changed blocks, ensuring data completeness while minimizing transfer time and resources used.
3Adaptability or versatility
If multiple users work on the same files simultaneously, then collaboration is enabled, but data loss occurs due to overwriting or lockouts
Solution Approach 1:
The patent divides files into multiple data blocks containing related data fragments. When multiple users work on the same file simultaneously, each user's changes are captured as specific data block modifications rather than complete file overwrites. This segmentation allows the system to manage concurrent edits more effectively, placing related fragments in the same block to maintain data integrity and prevent conflicts.
Solution Approach 2:
The system extracts and identifies only the specific data blocks that have been modified by each user. By analyzing content relationships between data fragments, the system can selectively manage and merge changes from multiple users, preventing data loss and avoiding lockout scenarios while maintaining collaborative workflow.
Data Source
AI summary
Data is stored in streams of varying size, a plurality of data streams are linked by references to form an application file such that each data stream is a portion of an application file, and an operating system transfers said application file between storage and a client application. Data streams that are below a predetermined size are identified as data fragments, wherein a plurality of data fragments are written to storage in a data block. Each data fragment is analysed to determine whether it is related in content to a first set of data fragments or to a second set of data fragments, the first set of data fragments is placed in a first block and the second set of data fragments is placed in a second block.