Data Writing Device Load Balancing for High-Speed Network Recording
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data writing methods struggle to record packets flowing through high-speed communication networks like Ethernet at 100 Gbps in secondary storages over a long time without data loss, requiring unique hardware and experiencing unstable writing performance due to varying storage capabilities.
Innovation Solution
A data writing device and method utilizing a general-purpose computer apparatus with a multiprocessing operating system, writing processes, a monitoring process, and a data distribution process to manage data distribution across multiple secondary storages, selecting the storage with the highest writing performance based on requested and finished data quantities, ensuring balanced load and continuous high-speed data recording.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional data writing methods using parallel secondary storages are used, then data writing rate is improved, but device complexity increases due to need for unique hardware
Solution Approach 1:
The patent replaces complex dedicated hardware control mechanisms with a software-based load management system. The access load managing device, data position determining device, and data position managing device are implemented as software processes that monitor storage load and distribute data accordingly, eliminating the need for specialized hardware while maintaining high parallel writing performance.
Solution Approach 2:
The patent employs general-purpose storage devices that can function both as individual storage units and as part of a parallel storage system. Each storage device maintains its standard functionality while also participating in load-balanced parallel operations through software coordination, making the system universally applicable to existing storage hardware without requiring specialized components.
2Productivity
If data is written in one of the secondary storages which is under the lowest load, then data input/output is improved, but writing performance stability deteriorates over long time
Solution Approach 1:
The patent implements a feedback mechanism where the access load managing device continuously monitors the load status of each storage device and dynamically adjusts data distribution decisions. This real-time feedback ensures that data is always directed to the currently least-loaded storage, adapting to changing performance conditions and maintaining stable high-speed writing over extended periods despite individual storage performance variations.
Solution Approach 2:
The patent transforms the static load distribution approach into a dynamic system that continuously adapts to changing storage conditions. The load management software actively monitors writing performance metrics and reconfigures data allocation in real-time, allowing the system to respond to performance degradation or improvement in individual storage devices and maintain optimal overall performance.
3Loss of information
If all packets flowing through communication network are recorded in secondary storages, then data capture completeness is improved, but data loss increases when writing rate cannot keep up with network speed
Solution Approach 1:
The patent segments the data writing operation into multiple parallel streams, each directed to a different storage device. By dividing the incoming data stream and distributing it across multiple storage units simultaneously, the system achieves an aggregate writing rate that exceeds the capability of any single storage device, thereby preventing packet loss even at high network speeds of 100 Gbps or more.
Solution Approach 2:
The patent combines the writing capabilities of multiple storage devices into a unified parallel writing system. The data position determining device coordinates multiple storage units to write different portions of the data stream simultaneously, merging their individual writing rates to achieve a collective throughput that matches or exceeds high-speed network input rates, thus eliminating data loss.
Data Source
AI summary
A data writing device includes secondary storages, an interface circuit which obtains data, and a computer apparatus which writes the data obtained in one of the secondary storages. The computer apparatus includes writing processes corresponding to the secondary storages as an application, respectively; a monitoring process which detects quantities of data whose writing is finished in the secondary storages; and a data distribution process which selects a secondary storage as a target for next writing based on requested data quantities of the secondary storages and the finished data quantities thereof, and instructs a writing process corresponding to the secondary storage as a target for next writing to write.