Iterative Autocorrelation Calculation for Streamed Data

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current methods for processing and analyzing Big Data and streamed data are inefficient, requiring significant computational resources and failing to provide real-time decision-making capabilities, especially in applications like financial analysis, due to the need for large buffer sizes and redundant calculations during autocorrelation calculations.

Innovation Solution

The method iteratively calculates autocorrelation by updating only the components affected by new data elements in the computation window, reducing the need to revisit all data elements and minimizing redundant computations, thereby enhancing calculation efficiency and resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional autocorrelation calculation methods are used on streamed data, then calculation accuracy is maintained, but computational load and processing time increase significantly

Engineering Contradiction:
Improvecalculation efficiencyVSAvoidcomputational resources
Core Design Contradiction:
ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent segments the autocorrelation calculation into distinct components: mean calculation, variance calculation, and covariance calculation. Each component is computed separately and then combined to produce the final autocorrelation result. This segmentation allows for optimized computation of each part and enables parallel processing, thereby improving calculation efficiency while reducing overall computational load.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary calculations of the mean and variance before computing the covariance. By pre-computing these statistical parameters and storing them, the method avoids redundant calculations during the autocorrelation computation phase. This preliminary action significantly reduces the computational burden, especially when dealing with large buffered datasets.

Inventive Principle:
Principle #10Preliminary action

2Measurement precision

If large computation windows are used for autocorrelation analysis, then statistical accuracy is improved, but buffer requirements and memory usage increase

Engineering Contradiction:
Improvestatistical accuracyVSAvoidbuffer size
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent divides the large computation window into smaller manageable segments or blocks. Instead of loading the entire large window into memory at once, the method processes data in smaller chunks, computing statistical parameters for each segment and then aggregating the results. This approach maintains statistical accuracy over the full computation window while significantly reducing the buffer size required at any given time.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements a nested processing structure where smaller computation windows or blocks are nested within the larger computation window. Each nested block is processed independently, and their results are combined to produce the final autocorrelation for the entire large window. This nesting strategy enables accurate analysis of large datasets with limited memory resources.

Inventive Principle:
Principle #7Nested doll (Nesting)

3Speed

If autocorrelation is recalculated frequently for real-time decisions, then responsiveness is improved, but computational overhead and power consumption increase

Engineering Contradiction:
Improveresponse timeVSAvoidpower consumption
Core Design Contradiction:
SpeedVSLoss of energy

Solution Approach 1:

The patent establishes continuous computation streams where statistical parameters (mean, variance, covariance) are updated incrementally as new data arrives, rather than performing complete recalculations. This continuous update mechanism maintains real-time responsiveness by always having current statistical values ready, while minimizing computational overhead through incremental updates instead of frequent full recalculations.

Inventive Principle:
Principle #20Continuity of useful action

Solution Approach 2:

The patent pre-computes and maintains running statistics (mean, variance) that are updated continuously as data flows through the system. These pre-computed parameters are readily available when autocorrelation calculations are needed, eliminating the need for expensive full recalculations and enabling rapid response to new data while keeping power consumption low.

Inventive Principle:
Principle #10Preliminary action

4Reliability

If all data elements in the computation window are accessed for each autocorrelation calculation, then calculation completeness is ensured, but processing time and computational complexity increase

Engineering Contradiction:
Improvecalculation completenessVSAvoidprocessing time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the data access pattern by organizing computations around pre-calculated statistical parameters rather than accessing individual data elements. Instead of visiting all n data elements for each autocorrelation calculation, the method accesses the already-computed mean, variance, and covariance parameters, which encapsulate the information from all data elements. This ensures calculation completeness while dramatically reducing processing time.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent creates and uses copies of statistical parameters (mean, variance, covariance) that represent the entire data window. Rather than accessing the original data elements repeatedly, the system works with these computed parameter copies, which contain the essential statistical information. This copying approach maintains calculation reliability while minimizing data access time and computational complexity.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS10320685B1Iterative autocorrelation calculation for streamed data using components
Publication Date: 2019.06.11 CLOUD & STREAM GEARS LLC
  • US10320685B1 patent drawing
  • US10320685B1 patent drawing
  • US10320685B1 patent drawing

AI summary

The present invention extends to methods, systems, and computing system program products for iteratively calculating autocorrelation at a specified lag for streamed data in real time by iteratively calculating one or more components of autocorrelation at the specified lag l for a computation window of size n. Embodiments of the invention include iteratively calculating one or more components of autocorrelation at the specified lag l for an adjusted computation window based on the one or more components of the autocorrelation at the specified lag l calculated for a previous computation window and then calculating the autocorrelation at the specified lag l using the components. Iteratively calculating autocorrelation avoids visiting all data elements in the adjusted computation window and performing redundant computations thereby increasing calculation efficiency, saving computing resources and reducing computing system's power consumption.