Portable Storage Data Streaming Without Over-Provisioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for storing data copies in different locations often result in performance degradation and negative user experiences due to the need for over-provisioned storage space, which can lead to data loss if the remote storage operation takes longer than the consumption of this limited space.
Innovation Solution
A method and system for streaming data from portable storage devices that enables iterative data streaming to a remote storage system without requiring zero over-provisioning storage space for buffering incoming write operations, allowing continuous write operations while transmitting data changes to the remote storage system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional data copying methods are used to store data in different locations, then data redundancy is achieved, but device performance deteriorates and user experience worsens due to over-provisioned storage space requirements
Solution Approach 1:
The patent segments data transmission into iterative blocks, where data is divided into manageable chunks that are transmitted sequentially. This allows the system to maintain data redundancy goals while avoiding the performance degradation caused by traditional bulk copying methods, as each block can be processed and transmitted independently without blocking subsequent write operations
Solution Approach 2:
The system performs preliminary actions by pre-allocating minimal buffer space and preparing transmission mechanisms in advance. This allows the portable storage device to continue accepting write operations while the data transmission infrastructure is already in place, preventing performance degradation that would occur if transmission setup blocked write operations
2Reliability
If over-provisioned storage space is allocated for buffering data during remote storage operations, then data loss is prevented, but storage capacity is wasted and device complexity increases
Solution Approach 1:
Instead of allocating full over-provisioned storage space, the patent applies partial action by using minimal buffer space sufficient for just the current data block being transmitted. This partial buffering approach prevents data loss during transmission while avoiding the waste of allocating excessive storage capacity that would sit idle during transmission operations
Solution Approach 2:
The system discards the need for permanent over-provisioned storage space by using temporary, transient buffer space that is allocated only for the duration of each data block transmission. After each block is successfully transmitted, the buffer space is recovered and reused for the next block, eliminating the need for permanently reserved storage capacity
3Productivity
If data transmission to remote storage is performed before write operations complete, then continuous write operations can be serviced, but data loss may occur if transmission takes longer than buffer consumption
Solution Approach 1:
By segmenting data into discrete blocks with associated transaction group identifiers, the system can transmit each block independently once its specific data is written. This segmentation allows continuous write operations to proceed while each segment is transmitted at its own pace, eliminating the risk that a long transmission blocks subsequent writes while maintaining data integrity through the mapping mechanism
Solution Approach 2:
The system implements feedback through the mapping of block sets to transaction group identifiers, where the completion status of write operations for each block is tracked. This feedback mechanism allows the system to know precisely when each data block is safely written, enabling transmission to proceed only when appropriate without risking data loss, while maintaining continuous write capability
Data Source
AI summary
A method and system for streaming data from portable storage devices. Specifically, the disclosed method and system implement iterative data streaming from a portable storage device for remote storage operations, while requiring zero over-provisioning storage space for buffering incoming write operations to the portable storage device.