Low-Rank Matrix Approximation via Incremental SVD Updates

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Analyzing large and streaming data sets with conventional methods requires immense computational and storage resources due to their massive size, making it inefficient to identify trends or features.

Innovation Solution

The use of low-rank matrix approximations generated through incremental singular value decomposition (SVD) updates, which reduce computational complexity while maintaining minimal accuracy, allowing for efficient processing and storage of large data sets.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional data analysis techniques are applied to extremely large data sets, then accurate identification of trends and relationships is achieved, but immense computational resources and high storage resources are required

Engineering Contradiction:
Improveaccuracy of trend identificationVSAvoidcomputational resources required
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts only the most important features and patterns from the complete data set by computing low-rank approximations. Instead of analyzing all data entries, the system identifies and retains the dominant singular vectors and values that capture the essential structure, discarding redundant information. This extraction approach maintains analytical accuracy while dramatically reducing computational and storage requirements.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent transforms the data representation by changing from the original high-dimensional matrix to a low-rank approximation with reduced dimensions. By parameterizing the data through singular value decomposition and selecting only the top k singular values and vectors, the system converts an intractable large-scale problem into a manageable smaller problem that preserves the essential characteristics needed for trend identification.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If conventional data analysis techniques are applied to extremely large data sets, then accurate identification of trends and relationships is achieved, but immense storage resources are required

Engineering Contradiction:
Improveaccuracy of trend identificationVSAvoidstorage resources required
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential structural information from the complete data set by computing low-rank approximations. Instead of storing all data entries, the system stores only the dominant singular vectors and values that capture the essential patterns, reducing storage requirements from O(nm) to O(k(n+m)) where k is much smaller than the original dimensions.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent creates a simplified copy or approximation of the original data set that preserves the essential information needed for analysis. The low-rank approximation serves as a compact representation that can be stored and manipulated much more efficiently than the full data set, while still enabling accurate trend identification through the preserved dominant patterns.

Inventive Principle:
Principle #26Copying

3Measurement precision

If singular value decomposition (SVD) is performed on large matrices, then accurate low-rank approximation is achieved, but computationally expensive operations are required

Engineering Contradiction:
Improveaccuracy of low-rank approximationVSAvoidcomputational efficiency
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent segments the computation of SVD into iterative steps that process data in batches or streams rather than requiring complete data loading. The system performs incremental SVD updates where each iteration works with a subset of data and updates the approximation progressively, avoiding the need to compute SVD on the entire large matrix at once. This segmentation enables parallel processing and reduces memory requirements.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary processing of data by computing randomized projections or sketching the data before performing the actual SVD computation. This preliminary action transforms the original large matrix into a smaller sketched matrix that captures the essential structure, allowing the subsequent SVD to be performed efficiently on reduced data while maintaining approximation accuracy.

Inventive Principle:
Principle #10Preliminary action

4Loss of time

If streaming data is continuously processed, then up-to-date analysis is achieved, but computational resources increase with growing data size

Engineering Contradiction:
Improvetimeliness of analysisVSAvoidcomputational resources required
Core Design Contradiction:
Loss of timeVSDevice complexity

Solution Approach 1:

The patent implements dynamic low-rank approximation that adapts to streaming data by continuously updating the singular value decomposition as new data arrives. The system maintains an evolving approximation that adjusts to changing patterns in the data stream, allowing timely analysis of current trends without reprocessing the entire historical data set. This dynamic approach keeps computational resources bounded rather than growing with data size.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent enables continuous processing of streaming data by maintaining an ongoing low-rank approximation that is updated incrementally as new data points arrive. Instead of performing batch processing at intervals, the system continuously updates the approximation in real-time, ensuring that analysis remains current and timely while using efficient incremental algorithms that avoid redundant computations.

Inventive Principle:
Principle #20Continuity of useful action

Data Source

PatentUS10318608B2Systems and methods for low-rank matrix approximation
Publication Date: 2019.06.11 THE CHARLES STARK DRAPER LABORATORY INC
  • US10318608B2 patent drawing
  • US10318608B2 patent drawing
  • US10318608B2 patent drawing

AI summary

Systems and methods may identify at least a first set of right singular vectors and a first set of singular values of a subset of the matrix, reduce the subset by an amount of energy of a selected data entry of the subset based on the first set of right singular vectors and the first set of singular values, incorporate a new data entry from the matrix into the subset, update the first set of right singular vectors and the first set of singular values of the subset based on the new data entry by a singular value decomposition (SVD) update, and generate the low-rank approximation of the matrix based on the updated first set of right singular vectors and the updated first set of singular values.