Buffer Accumulation Control for Loss-Free Real-Time Data Output
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face challenges in efficiently configuring the accumulation time for data registration, leading to potential data loss and impaired real-time performance due to prolonged storage times, especially when data is aggregated over extended periods.
Innovation Solution
A data processing system that determines an accumulation time equal to or shorter than a reference cycle by monitoring data loss and throughput, using a processor to configure the system to prioritize either data loss avoidance or application performance, adjusting parameters to optimize data storage without loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data aggregation uses a long accumulation time, then data loss is prevented, but real-time performance of data utilization is impaired
Solution Approach 1:
The patent applies dynamics by making the accumulation time configurable and adjustable based on system conditions. The accumulation time determination unit dynamically sets the accumulation time within a range from a first value (shorter) to a second value (longer), allowing the system to adapt between real-time performance and data loss prevention depending on the operational context.
Solution Approach 2:
The patent changes the parameter of accumulation time to resolve the contradiction. By configuring the accumulation time within a specific range and adjusting it based on data loss detection and reference cycle comparisons, the system optimizes the balance between preventing data loss and maintaining real-time performance.
2Speed
If data processing is performed sequentially each time data arrives, then real-time data registration is achieved, but processing overhead increases and data loss occurs
Solution Approach 1:
The patent applies periodic action by using accumulation time to group data registration operations. Instead of processing each data arrival immediately, the system accumulates data over a predetermined time period and processes them together, reducing processing overhead while preventing data loss through the periodic registration mechanism.
3Reliability
If accumulation time is extended to prevent data loss, then data completeness improves, but storage time to data registration destination increases
Solution Approach 1:
The system dynamically adjusts the accumulation time within a configured range based on actual system conditions. The accumulation time determination unit monitors data loss and adjusts the accumulation time accordingly, ensuring data completeness while minimizing unnecessary storage time delays.
Data Source
AI summary
A data processing system is provided, which executes processing for receiving data, storing the received data in a buffer, and outputting the data stored in the buffer to a predetermined data storage destination, the data processing system including one or more processors configured to: receive a reference cycle for referring to data in the data storage destination; determine an accumulation time, for which data stored in the buffer is held before being output to the data storage destination, to be a time equal to or shorter than the reference cycle, based on whether there is data loss by which data to be stored in the buffer cannot be stored in the buffer; and output data in the buffer to the data storage destination, in accordance with the determined accumulation time.