Streaming Tensor Analysis via Blind Source Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current tensor decomposition methods fail to scale to higher dimensional tensors and do not handle sparse data effectively, particularly in real-time analysis of streaming data, which is crucial for applications like malicious activity detection on data networks.
Innovation Solution
The system employs Independent Component Analysis of Tensors (ICAT) with parallel processing pipelines that update and demix tensor slices in different orientations, performing cycles of demixing, transitive matching, and tensor factor weight calculations to reconstruct sensor data and extract hidden patterns, using a blind source separation algorithm to handle streaming data efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional tensor decomposition methods (PARAFAC, ALS) are used, then the decomposition can be performed on complete data, but the methods do not scale to higher dimensional tensors and do not handle sparse data well
Solution Approach 1:
The patent segments the tensor decomposition problem into multiple independent pipeline operations that can be executed in parallel. Each pipeline processes different aspects of the decomposition (demixing, transitive matching, factor weight calculation) independently, allowing the system to handle higher dimensional tensors efficiently without being constrained by the computational complexity of traditional methods.
Solution Approach 2:
The patent implements dynamic updating of tensor slices in parallel pipelines, where the system adapts to streaming data by continuously updating demixing matrices and factor weights. This dynamic approach allows the system to handle sparse data effectively by updating only the necessary components rather than processing the entire tensor at each step.
2Measurement precision
If complete tensor decomposition is performed on all data, then accurate results are obtained, but processing time increases significantly for streaming data
Solution Approach 1:
The patent performs preliminary demixing operations on tensor slices before full decomposition. By pre-processing the data through parallel pipelines that compute demixing matrices and initial factor estimates, the system reduces the computational burden of subsequent decomposition steps, enabling real-time processing while maintaining accuracy.
Solution Approach 2:
The patent implements continuous updating of tensor decomposition results through parallel pipelines that process streaming data in real-time. Rather than performing batch processing, the system continuously updates demixing matrices, performs transitive matching, and recalculates factor weights as new data arrives, maintaining both accuracy and real-time performance.
3Productivity
If traditional decomposition methods are used on streaming data, then the full data set can be processed, but memory requirements become prohibitive
Solution Approach 1:
The patent segments the large streaming data set into smaller tensor slices that can be processed independently in parallel pipelines. Each pipeline maintains only the necessary demixing matrices and factor weights for its specific slice, dramatically reducing overall memory requirements while maintaining the ability to process the complete streaming data set.
Solution Approach 2:
The patent extracts and processes only the essential components of the tensor decomposition (demixing matrices, key factor weights) in the parallel pipelines, rather than storing and processing the entire data set. This extraction approach retains the critical information needed for accurate decomposition while minimizing memory consumption.
Data Source
AI summary
Described is a system for controlling a device based on streaming data analysis using blind source separation. The system updates a set of parallel processing pipelines for two-dimensional (2D) tensor slices of streaming tensor data in different orientations, where the streaming tensor data includes incomplete sensor data. In updating the parallel processing pipelines, the system replaces a first tensor slice with a new tensor slice resulting in an updated set of tensor slices in different orientations. At each time step, a cycle of demixing, transitive matching, and tensor factor weight calculations is performed on the updated set of tensor slices. The tensor factor weight calculations are used for sensor data reconstruction, and based on the sensor data reconstruction, hidden sensor data is extracted. Upon recognition of an object in the extracted hidden sensor data, the device is caused to perform a maneuver to avoid a collision with the object.