Parallel Recursive Filtering for Large Time Series Partitions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional serial filtering methods for large time series of data with recursive dependencies are computationally burdensome and time-intensive, especially when dealing with datasets that exceed memory capacity, leading to significant processing delays and memory constraints.
Innovation Solution
The filtering process is organized into parallel tasks, allowing for the computation of zero-input and zero-conditions partial solutions, which can be performed in parallel across partitions of the data, reducing the need for sequential computation and enabling faster processing by distributing the workload across multiple data processing devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If serial filtering methods are used for large time series data with recursive dependencies, then filtering accuracy is maintained, but computation time and processing burden increase significantly
Solution Approach 1:
The patent divides the time series data into multiple partitions and processes each partition independently using parallel computing. Each partition solves a portion of the recursive filter equation, obtaining K final filter values before obtaining all N filter values. This segmentation allows simultaneous processing of multiple data segments, dramatically reducing computation time while maintaining filtering accuracy through proper handling of recursive dependencies across partition boundaries.
2Reliability
If serial filtering methods are used for large time series data, then complete filtering is achieved, but memory requirements and processing burden on individual devices increase
Solution Approach 1:
The patent segments both the data and the computation across multiple processing devices. Each device handles a specific partition of the time series data, requiring only the memory capacity to store that partition and intermediate results. This distribution of data and computation reduces the memory burden on each individual device while ensuring complete filtering through coordinated processing of all partitions.
Solution Approach 2:
The patent introduces a parallel processing dimension by distributing computations across multiple devices simultaneously. Instead of processing data sequentially on a single device, the system operates in parallel across multiple processing units, each handling different partitions. This dimensional shift from serial to parallel architecture reduces per-device memory requirements and processing burden while maintaining overall filtering completeness.
Data Source
AI summary
Filtering apparatus and methods associated with filtering large time series of data are described. A filtering process containing recursive dependencies can be organized as a series of computational tasks, at least some of which can be performed in parallel. Because of parallel execution of some tasks, an amount of time for filtering large time series of data with a filter exhibiting recursive data dependencies can be reduced significantly.