Real-Time Data Stream Splitting Across Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time recording of streaming data in distributed storage systems is hindered by the uncertainty of data stream length, leading to potential storage device fullness and interruptions during playback, as existing systems cannot seamlessly split and reassemble data streams across multiple devices.
Innovation Solution
The method employs metadata tags and temporary buffers to split data streams in real-time across connected storage devices, allowing seamless concatenation during playback, with metadata descriptors providing navigation information for reconstructing the original stream.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If data streams are stored on a single storage device, then storage management is simple, but the storage device may become full before the data stream is completely stored
Solution Approach 1:
The patent divides a continuous data stream into multiple segments (first data stream portion and second data stream portion) that can be stored on different storage devices. When the first storage device becomes full, the system automatically splits the data stream and continues storing the remaining portion on a second storage device, ensuring complete storage without requiring a single large device.
2Reliability
If the system switches to another storage device when the first is full, then complete storage is possible, but real-time behavior is hampered due to switching time
Solution Approach 1:
The system performs preliminary actions by pre-selecting a second storage device before the first device becomes completely full. The second storage device is prepared and ready in advance, so when the split occurs, the transition is seamless and does not interrupt the real-time data flow. This eliminates switching delays by having the backup device ready beforehand.
3Device complexity
If data streams are handled as single files, then storage organization is simple, but distributed storage cannot be effectively utilized
Solution Approach 1:
The patent segments the data stream into multiple portions that can be distributed across different storage devices while maintaining the ability to reconstruct the original stream. This enables effective utilization of distributed storage resources while the system automatically manages the segmentation process, keeping organization simple for the user.
Solution Approach 2:
The patent introduces metadata tags as intermediaries that contain information about the segmented data portions. These tags enable the system to track and reconstruct the original data stream from distributed segments without requiring complex user management of the distribution process.
4Speed
If metadata tags and temporary buffers are used for seamless splitting, then real-time behavior is maintained, but system complexity increases
Solution Approach 1:
The patent uses metadata tags as intermediaries to manage the segmented data streams. These tags contain essential information for reconstructing the original stream and coordinating between storage devices. The temporary buffers act as intermediaries to hold data during transitions. While these additions increase system complexity, they enable seamless real-time operation without data loss.
Data Source
AI summary
When recording packetized real-time streaming data, e.g. multimedia data, it is in general not possible to know in advance the size of the data stream, and thus the required storage area. Therefore the storage device may be full before the data stream is completely stored. The disclosed method for storing and retrieving the remaining part of the data stream on another storage device uses metadata tags (10,11) and data buffers (IB1,IB2) to split a data stream seamlessly into chunks while recording it, and distribute the chunks in real-time to different connected storage devices (D1,D2), so that the chunks can be seamlessly concatenated again in real-time for replaying the stream. The metadata tags (10,11) contain identifiers for the successive storage node (N2) and/or the preceding storage node (N1) and for the last stored application packet.