Storage Cluster Data Shifting via Graph-Based Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage clusters with both relational and non-relational database nodes face inefficiencies in managing data access and reliability, as existing systems lack effective methods to optimize data distribution based on usage patterns and operation types.
Innovation Solution
A method involving the creation of a graph where nodes represent data objects and edges represent joins, with node values and weights based on access statistics, to assign data objects to appropriate storage nodes, shifting frequently joined data to relational nodes and standalone data to non-relational nodes, thereby optimizing data distribution and access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data objects are stored on a single type of storage node, then the storage system is simple to manage, but data access efficiency and reliability deteriorate due to inability to optimize based on usage patterns
Solution Approach 1:
The patent segments data objects into different subsets based on their access patterns and join frequencies. Frequently joined data objects are assigned to relational database storage nodes, while standalone data objects are assigned to non-relational database storage nodes. This segmentation allows each storage node type to handle specific workloads optimally, improving overall data access efficiency while maintaining manageable complexity through automated classification.
Solution Approach 2:
The patent applies local quality by assigning different storage node types to different data objects based on their specific characteristics and usage patterns. Each data object or subset receives the appropriate storage node type (relational or non-relational) tailored to its access patterns, rather than using a uniform storage approach. This localized optimization improves performance for each data type while the system manages complexity through graph-based classification.
2Loss of time
If data distribution is not optimized based on usage patterns, then the storage system is easier to implement, but response times and resource utilization deteriorate
Solution Approach 1:
The patent performs preliminary action by analyzing access statistics and join operations in advance to classify data objects before they are accessed. The system creates a graph representation of data objects and their relationships, pre-determines optimal storage node assignments based on usage patterns, and implements this distribution strategy proactively. This preliminary classification and assignment reduces access response times by ensuring data is already optimally positioned when needed.
Solution Approach 2:
The patent employs feedback mechanisms by continuously monitoring access statistics and join operation patterns to refine data object classification and storage node assignment. The system uses this feedback to optimize the graph representation and reassign data objects to storage nodes that best match their usage patterns, thereby reducing response times and improving resource utilization over time while managing complexity through automated iterative optimization.
3Productivity
If all data objects are stored on the same storage node type, then the system is easier to operate, but join operations and data access become less efficient
Solution Approach 1:
The patent implements self-service by enabling the system to automatically classify and assign data objects to appropriate storage nodes based on their access patterns and join frequencies. The graph-based classification system autonomously determines which data objects should be stored on relational database storage nodes versus non-relational database storage nodes, eliminating the need for manual intervention. This automated self-assignment improves join operation efficiency while maintaining ease of operation through hands-free optimization.
Data Source
AI summary
A method performed by a computing system includes creating a graph, wherein nodes of the graph represent data objects of a data storage cluster, wherein edges of the graph represent joins between data objects represented by both nodes of respective edges, wherein node values of the nodes and weights of the edges are based on statistics related to use of the data objects. The method further includes assigning a first subset of the data objects to a relational database storage node within the data storage cluster, the first subset of data objects being represented by nodes within a cluster of the graph, and assigning a second subset of the data objects to a non-relational database storage node within the data storage cluster, the second subset of data objects being represented by nodes within the graph that are not part of a cluster.