Rolling Time Period Data Processing with Callback Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating statistical and graphical representations of time-varying data samples face challenges in maintaining real-time updates, particularly when dealing with large datasets or complex processing, leading to delays or outdated results when using rolling time periods.
Innovation Solution
A method that involves receiving new data samples, associating them with timestamps, and programming callbacks to update output data for rolling time periods, ensuring that data is always current by adjusting calculations based on new samples and removing their influence when callbacks occur.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time processing is performed to assemble data samples and generate statistical/graphical representations, then the results are up to date, but unacceptable delay occurs between user request and display of results when dealing with large numbers of data samples and complex processing
Solution Approach 1:
The system performs preliminary processing by maintaining pre-computed statistical representations for multiple rolling time periods. When new data samples arrive, the system updates these pre-computed representations incrementally rather than reprocessing all data from scratch, allowing immediate display of results without unacceptable delays.
Solution Approach 2:
The system dynamically adjusts the set of rolling time periods based on current time and newly received data samples. Callback mechanisms are programmed to trigger at specific future times corresponding to rolling time period boundaries, enabling the system to automatically update statistics when periods roll over without requiring continuous real-time processing.
2Productivity
If periodic recalculations are scheduled to display statistical/graphical representations immediately, then results are displayed without delay, but the results are not always up to date as they do not include new data samples received since the last recalculation
Solution Approach 1:
The system uses callback mechanisms that provide feedback when rolling time period boundaries are reached. These callbacks trigger incremental updates to the pre-computed statistical representations, ensuring that new data samples are incorporated at the appropriate moments without requiring continuous reprocessing. This feedback loop maintains both immediate display capability and data freshness.
Solution Approach 2:
The system maintains continuous availability of statistical representations by keeping pre-computed data for multiple rolling time periods ready for immediate display. When new data samples arrive, the system continuously updates these representations incrementally, ensuring that the useful action of providing up-to-date statistics continues without interruption or unacceptable delays.
3Adaptability or versatility
If all data samples are processed to generate statistics for multiple rolling time periods, then comprehensive coverage is achieved, but extensive reprocessing is required when new data samples arrive
Solution Approach 1:
The system segments the processing task by maintaining separate statistical representations for multiple distinct rolling time periods (e.g., 24 hours, 7 days, 30 days). Each time period's statistics are updated independently when new data samples arrive, avoiding the need to reprocess all data for all periods simultaneously. This segmentation reduces reprocessing complexity while maintaining comprehensive coverage.
Solution Approach 2:
The system performs partial processing by updating only the specific statistical representations that need adjustment when new data samples arrive, rather than reprocessing all data for all time periods. The callback mechanism enables the system to perform excessive action by pre-computing and maintaining statistics for future rolling time periods in advance, reducing the need for extensive reprocessing when those periods are reached.
Data Source
AI summary
The invention concerns a method of processing data to provide output data based on a group of data samples having a time stamp falling within at least one rolling time period, the method comprising: receiving a new data sample and associating said new data sample with a first time stamp; updating said output data corresponding to a first rolling time period to be based on said group of data samples including said new data sample; programming a first callback based on said first time stamp and on the duration of said first rolling time period; and in response to the occurrence of said first callback, updating said output data corresponding to said first rolling time period to no longer be based on said new data sample.