Graph Database Partition Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed graph databases face challenges in optimizing storage distribution, leading to high resource consumption and low efficiency in query statistics processing due to random data localization and high network communication overheads.
Innovation Solution
A method that periodically scans edges in partitions, calculates outgoing and incoming-edge correlations, and stores partitions with high relevancies on the same storage server using a preset correlation matrix weight, optimizing data storage and query statistics processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If fully symmetric distributed architecture is used for data storage, then load balancing is achieved, but data localization is random and network communication overhead is high
Solution Approach 1:
The patent applies local quality by calculating correlation values between different partitions based on edge relationships and storing highly correlated partitions together on the same storage server. This creates localized data groups where related data is physically close, reducing network communication overhead while maintaining load balancing across the distributed system.
Solution Approach 2:
The patent introduces asymmetry by moving from a fully symmetric distributed architecture to an asymmetric architecture where partitions are distributed based on calculated correlation values. Storage servers hold non-uniform sets of partitions according to their correlation relationships, creating an optimized asymmetric distribution that reduces network overhead.
2Quantity of substance
If traditional distributed storage is used, then data can be stored, but query statistics processing consumes high computational resources and has low efficiency
Solution Approach 1:
The patent applies preliminary action by pre-calculating correlation values between partitions based on edge relationships and organizing storage accordingly before query operations occur. This advance organization of data based on correlation metrics enables more efficient query statistics processing without requiring additional computational resources during actual queries.
3Ease of manufacture
If partitions are distributed randomly across storage servers, then storage is simple, but data localization is poor and affects query performance
Solution Approach 1:
The patent applies feedback by calculating correlation values between partitions based on edge relationships and using this calculated information to guide the storage distribution decision. This feedback mechanism ensures that partitions with high correlation are stored together, improving data localization accuracy while maintaining systematic storage organization.
Data Source
AI summary
Disclosed are a method and a system for optimizing data storage of query statistics of a graph database. The method includes: periodically scanning, on storage servers in which partitions are located, all edges in the partitions; determining, according to all the edges in the partitions, partitions to which start points and end points belong, and calculating outgoing-edge correlation and incoming-edge correlation between partitions; calculating relevancies between partitions through a preset correlation matrix weight according to the outgoing-edge correlation and the incoming-edge correlation between partitions; and storing partitions with high relevancies on a same storage server. The present disclosure solves the problems of difficult optimization of storage distribution, as well as high resource consumption and low efficiency of query statistics of a graph database, and implements partition distribution of a graph database with an optimized distribution architecture, thereby reducing resource consumption of data calling and improving performance of the graph database.