Vector ID Vectors for Database Storage Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems require significant storage space and inefficiencies in data retrieval and versioning due to their reliance on discrete record structures, which limits performance and scalability, especially in multi-version databases.
Innovation Solution
Implementing a vector-based data structure that stores data as sequences of elements, allowing for efficient storage, retrieval, and versioning through the use of vector ID vectors and header tables, enabling re-use of identical vectors and efficient memory management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If discrete record structures are used to store table data, then data can be stored and retrieved, but storage space requirements increase and retrieval efficiency decreases
Solution Approach 1:
The patent merges multiple discrete records into a single vector structure. Instead of storing each record separately with its own row ID and data, the system creates a vector that contains all records as a continuous sequence of elements, eliminating redundant storage of row ID fields and improving retrieval efficiency through sequential access patterns.
Solution Approach 2:
The vector structure serves multiple functions simultaneously: it stores data, maintains relationships between records, enables versioning, and provides efficient retrieval mechanisms. The single vector structure replaces the need for separate discrete record structures, header tables, and reference mechanisms, achieving multi-functionality that resolves the storage and performance contradictions.
2Reliability
If discrete records with explicit references are used, then data relationships can be maintained, but device complexity increases
Solution Approach 1:
The patent combines multiple data structures (discrete records, references, headers) into a single unified vector structure. The vector inherently maintains relationships through its sequential element structure, eliminating the need for separate reference mechanisms and reducing overall system complexity while preserving data relationship integrity.
Solution Approach 2:
The patent extracts the relationship-maintenance function from discrete record structures and embeds it directly into the vector's sequential element architecture. This extraction eliminates redundant reference fields and simplifies the overall structure by taking out the complexity of managing separate reference mechanisms.
3Adaptability or versatility
If multi-version feature is implemented with separate record versions, then data versioning is supported, but storage requirements and processing time increase
Solution Approach 1:
The patent merges multiple versioned records into a single vector structure where versions are represented as sequential elements. Instead of storing complete separate record versions with all their fields, the system stores versions as a sequence of elements within the vector, sharing common fields and reducing redundant storage space while maintaining full versioning capability.
Solution Approach 2:
The patent discards redundant version information by using a shared vector structure where versions are differentiated only by their position and elements in the sequence. Common data fields are recovered and shared across all versions through the unified vector structure, reducing storage requirements while preserving version history.
4Ease of operation
If discrete records are stored with all fields repeated, then data can be accessed, but data redundancy increases
Solution Approach 1:
The patent merges repeated field data into a single vector structure where each element represents a data field value. Instead of storing complete copies of all fields for each record, the vector stores a single sequence of elements that can be accessed efficiently, eliminating data redundancy while maintaining ease of access through sequential indexing.
Data Source
AI summary
The systems and methods described here can reduce the storage space required (memory and/or disk) to store certain types of data, provide efficient (fast) creation, modification and retrieval of such data, and support such data within the framework of a multi-version database. In some embodiments, the systems and methods can store each field of a set of records as a vector of values, e.g., a data vector. A set of records can be represented using a vector id vector, or “vid” vector, wherein each element of the vid vector contains a reference to the memory location of a data vector. A header table can store associations between labels and “vid” vectors that pertain to those labels. Identical data vectors can be re-used between different record sets or vid vectors needing that vector, thus saving space.