Intermediary System for Data Streams Using Cycle Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale computer systems, data streams from multiple sources can overwhelm intermediary systems, leading to processing interruptions and potential data loss, especially when external processors fail to confirm the completion of data processing in a timely manner.
Innovation Solution
Implementing a method where event data is assigned to cycles with scheduled transmission times, and only sent to external processors once confirmation of previous cycle processing is received, preventing data accumulation and allowing continuous data stream acceptance without loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the intermediary system continuously accepts and sends data streams to external processors, then data processing throughput is improved, but the system becomes overwhelmed causing processing interruptions and data loss
Solution Approach 1:
The patent segments the data stream into discrete cycles with scheduled transmission times. Each cycle is independent and can be managed separately, allowing the intermediary system to control when data is sent to external processors. This segmentation prevents overwhelming the external systems while maintaining continuous data acceptance capability.
Solution Approach 2:
The patent implements preliminary action by scheduling transmission times for each cycle in advance and requiring confirmation from external processors before sending the next cycle. This preliminary planning ensures that data is sent at appropriate intervals rather than continuously, preventing system overload while maintaining throughput.
2Reliability
If the intermediary system waits for confirmation of previous cycle processing before sending new data, then data loss is prevented, but processing time is increased
Solution Approach 1:
The patent implements periodic action by establishing fixed transmission intervals for each cycle. Data is sent at scheduled times rather than continuously, allowing the system to wait for confirmation only between periodic intervals. This maintains data loss prevention while limiting the time impact to specific cycle boundaries rather than continuous processing.
3Device complexity
If the intermediary system sends data cycles to external processors without cycle management, then system complexity is reduced, but data accumulation and overwhelming of external systems occurs
Solution Approach 1:
The patent divides the data stream into discrete, manageable cycles with scheduled transmission times. Each cycle is a separate unit that can be independently tracked and sent to external processors. This segmentation naturally limits data accumulation by controlling the size and timing of each cycle, preventing external system overload without requiring complex management mechanisms.
Data Source
AI summary
A computer-implemented method include sending a first cycle of received event data to an external system for processing and while waiting for the external system to confirm that processing of the first cycle of received event data has been completed, receiving a stream of additional event data for processing and assigning some of the additional event data to a second cycle of received event data, wherein the second cycle has a scheduled transmission time to be sent to the external system for processing. The second cycle of received event data is prevented from being sent to the external system when the scheduled transmission time arrives because the external system has not returned confirmation that processing has been completed for the first cycle.