Data Shard Storage Plan Execution via Cache-Aware Node Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed database systems, the primary computing node may not be optimally suited to execute data storage plans due to resource constraints or insufficient local caching, leading to inefficient plan execution.
Innovation Solution
Selecting a subscribing computing node with sufficient computing resources and optimal local caching capabilities to execute the data storage plan, identified by evaluating candidate nodes based on their current utilization and caching configuration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the primary computing node executes the data storage plan, then the data storage plan can be executed by the node responsible for the data shard, but the execution efficiency is reduced due to resource constraints and insufficient local caching
Solution Approach 1:
The patent introduces an intermediary selection mechanism that identifies and assigns a secondary computing node with better local caching capabilities to execute the data storage plan, rather than forcing the primary node to execute it directly. This mediator role resolves the conflict between responsibility fulfillment and execution efficiency.
Solution Approach 2:
The patent evaluates and selects computing nodes based on their local caching quality for specific data files. By choosing a node with superior local caching capabilities for the particular data being processed, the system optimizes execution efficiency while maintaining proper responsibility distribution.
2Ease of operation
If the primary computing node executes the data storage plan, then the node can maintain control over its own data shard, but the system overall performance is reduced due to resource constraints
Solution Approach 1:
The patent extracts the data storage plan execution task from the primary computing node and assigns it to a more suitable secondary node. This separation allows the primary node to maintain control and oversight while delegating the actual execution to a node with better resources, thereby improving system overall performance.
Solution Approach 2:
The patent implements a dynamic node selection mechanism that evaluates computing node capabilities and assigns execution tasks accordingly. This dynamic approach allows the system to adaptively allocate execution responsibilities based on current resource availability and caching capabilities, optimizing system performance while maintaining control.
3Speed
If a subscribing computing node with optimal local caching is selected to execute the data storage plan, then execution speed is improved, but the system complexity increases due to node selection and coordination overhead
Solution Approach 1:
The patent enables computing nodes to self-evaluate their local caching capabilities and voluntarily indicate their suitability for executing data storage plans. This self-service approach reduces the coordination overhead required for node selection, as nodes provide their own capability information rather than requiring centralized querying and evaluation of all nodes.
Solution Approach 2:
The patent implements preliminary registration of computing nodes and their local caching capabilities in a distributed registry. This preliminary action allows the system to quickly identify suitable execution nodes without complex real-time evaluation, reducing the selection overhead while maintaining the ability to choose optimal nodes for execution.
Data Source
AI summary
A distributed database system maintains a database including a data shard for which a primary computing node is responsible. The primary computing node identifies a data storage plan for the data shard. The plan identifies a file subset of data storage files of the shard to be merged into a larger data storage file, and a node subset of computing nodes of the system that subscribe to the data shard. The primary node identifies which computing nodes of the node subset each have sufficient computing resources to execute the plan, as candidate computing nodes. The primary node identifies which files of the file subset each candidate computing node locally caches. The primary node selects one candidate computing node to execute the plan, based on the files of the file subset that each candidate computing node locally caches. The primary node causes the selected candidate computing node to execute the plan.