Vector Embedding Delta Compression for Time-Series Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for storing and managing vector embeddings of time-series data sets are inefficient in terms of storage size and lack effective methods to identify changes between data files at non-adjacent time points.
Innovation Solution
A system and method for compressing vector embeddings using delta encodings, which store differences between adjacent time points, allowing for efficient storage and reconstruction of vector data at any time point, and identifying changes using delta encoding information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If vector embeddings are stored in their original form for time-series data, then complete data is available for analysis, but storage requirements become excessively large
Solution Approach 1:
The patent segments the time-series vector embeddings into adjacent time point pairs and stores only the differences (delta encodings) between consecutive vectors rather than all complete vectors. This segmentation approach reduces storage requirements while maintaining the ability to reconstruct any vector embedding by applying delta encodings sequentially from a baseline time point.
Solution Approach 2:
The patent performs preliminary computation of delta encodings between adjacent time points during data ingestion or processing. These pre-computed differences are stored compactly and can be applied later to reconstruct any historical or future vector embeddings, eliminating the need to store all complete vectors while preserving data accessibility.
2Productivity
If all vector embeddings are stored for every time point, then data accessibility is maintained, but computational efficiency for identifying changes deteriorates
Solution Approach 1:
The patent extracts and stores only the change information (delta encodings) between adjacent time points rather than all complete vector embeddings. When identifying changes between non-adjacent time points, the system only needs to process and apply the relevant delta encodings, significantly reducing computational time compared to comparing complete vectors.
Solution Approach 2:
The patent pre-computes and stores delta encodings during data ingestion, so that when change identification is needed, the system only needs to apply these pre-computed differences rather than performing full vector comparisons or reprocessing the original data, thereby improving computational efficiency.
3Quantity of substance
If delta encodings are used to compress vector embeddings, then storage requirements are reduced, but complexity of data reconstruction increases
Solution Approach 1:
The patent segments the reconstruction process into simple sequential applications of delta encodings to a baseline vector. Each delta encoding represents a simple difference between adjacent time points, so reconstruction is achieved through straightforward addition operations rather than complex transformations, maintaining ease of implementation.
Solution Approach 2:
The patent uses delta encodings as intermediary representations that bridge the baseline vector and any target time point vector. Instead of storing and manipulating complete vectors directly, the delta encodings serve as compact intermediaries that can be applied sequentially to reconstruct any vector, simplifying the overall system complexity while reducing storage needs.
Data Source
AI summary
A method of recreating vector database data from delta encoding data includes receiving first time-series vector data from a vector database and comprising a first plurality of vector embeddings representative of a first plurality of data files corresponding to a first time point, receiving a first plurality of delta encodings representative of differences between corresponding vector embeddings of the first plurality of vector embeddings and a second plurality of vector embeddings, and recreating the second plurality of vector embeddings based on the first plurality of vector embeddings and the first plurality of delta encodings. The second plurality of vector embeddings is representative of a second plurality of data files corresponding to a second time point each vector embedding of the second plurality of vector embeddings corresponds to one vector embedding of the first plurality of vector embeddings.