Graph Data Storage Controller with Vector Similarity Pre-processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage devices lack efficient pre-processing and metadata management for graph data, leading to suboptimal performance in data storage and retrieval operations.
Innovation Solution
A storage device comprising a controller, non-volatile memory, and a buffer memory, where the controller processes original graph data into pre-processing graph data based on vector similarity and generates metadata, which is then stored in data blocks within the non-volatile memory, with an FPGA determining block sizes for efficient data storage and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If graph data is stored without pre-processing and metadata management, then the storage device structure remains simple, but data storage efficiency and retrieval performance deteriorate
Solution Approach 1:
The patent performs pre-processing of graph data before storage, including constructing adjacency lists, calculating vector representations of graph nodes, and organizing data into structured formats. This preliminary organization of data and generation of metadata enables efficient retrieval operations later, resolving the contradiction by improving productivity through advance data preparation without requiring complex device architecture changes
Solution Approach 2:
The patent introduces metadata as an intermediary layer between the stored graph data and retrieval operations. This metadata includes vector representations, adjacency information, and structural indices that mediate between the raw graph data and query operations, enabling efficient search and retrieval without modifying the fundamental storage device structure
2Reliability
If graph data is classified by vector similarity with metadata generation, then data retrieval performance improves, but processing time and computational resources increase
Solution Approach 1:
The patent calculates vector representations and performs similarity classification of graph data during the pre-processing phase before storage. By organizing data into similarity-based groups and generating corresponding metadata in advance, the system enables fast retrieval operations without performing complex computations during actual query execution, thus improving reliability while minimizing time loss
Solution Approach 2:
The patent segments graph data into multiple partitions based on vector similarity metrics, creating organized clusters of related graph nodes. This segmentation allows retrieval operations to focus on relevant partitions rather than scanning entire datasets, improving retrieval performance while distributing processing workload efficiently
3Productivity
If data block sizes are dynamically determined based on buffer capacity and transfer rate, then storage optimization improves, but control complexity increases
Solution Approach 1:
The patent implements dynamic determination of data block sizes based on real-time buffer memory capacity and data transfer rate conditions. The controller adjusts block sizes adaptively during data storage operations, optimizing transfer efficiency by matching block dimensions to current system state without requiring complex external control mechanisms
Solution Approach 2:
The patent enables the storage device's controller to autonomously determine optimal data block sizes by monitoring its own buffer capacity and transfer rate performance. This self-service approach allows the system to optimize its own storage operations without external intervention, improving productivity while keeping control complexity manageable through decentralized decision-making
Data Source
AI summary
A storage device includes a controller configured to receive a pre-processing instruction command from an external device, a non-volatile memory configured to store an original graph data, and a buffer memory connected to the controller and the non-volatile memory, wherein the controller is configured to load the original graph data from the non-volatile memory, generate pre-processing graph data by classifying the original graph data depending on vector similarity in response to the pre-processing instruction command, generate metadata on the basis of the pre-processing graph data, and provide the pre-processing graph data and the metadata to the non-volatile memory, the non-volatile memory is configured to store the pre-processing graph data and the metadata in a data block, and the buffer memory is configured to buffer the original graph data, the pre-processing graph data, and the metadata.