Data Shard Storage Plan Execution via Cached-File Node Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed database systems, the primary computing node may not have sufficient computing resources or optimal caching of data storage files to efficiently execute data storage plans, leading to inefficient plan execution.
Innovation Solution
Selecting a subscribing computing node with sufficient resources and optimal caching of data storage files to execute the data storage plan, rather than relying solely on the primary computing node.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the primary computing node executes the data storage plan, then the data storage plan can be executed by the node responsible for the data shard, but the execution efficiency is reduced due to insufficient computing resources or suboptimal caching
Solution Approach 1:
The patent introduces a selection mechanism that acts as an intermediary between the data storage plan and the executing computing node. Instead of directly assigning execution to the primary computing node, the system selects an optimal subscribing computing node based on current resource availability and caching status, thereby resolving the contradiction between responsibility and execution efficiency
Solution Approach 2:
The patent implements dynamic node selection where the computing node responsible for executing the data storage plan is determined at runtime based on current system state. The primary computing node may delegate execution to another subscribing node if resources are insufficient or caching is suboptimal, making the execution role dynamic rather than static
2Productivity
If the primary computing node is used for execution, then the architecture remains simple with a single responsible node, but background file maintenance tasks are performed slowly
Solution Approach 1:
The patent applies partial action by having only the optimal subscribing computing node execute the data storage plan while other nodes continue their normal operations. This selective execution approach improves maintenance speed without requiring all nodes to participate in the selection process, balancing complexity and performance
Data Source
AI summary
A distributed database system maintains a database including a data shard for which a primary computing node is responsible. The primary computing node identifies a data storage plan for the data shard. The plan identifies a file subset of data storage files of the shard to be merged into a larger data storage file, and a node subset of computing nodes of the system that subscribe to the data shard. The primary node identifies which computing nodes of the node subset each have sufficient computing resources to execute the plan, as candidate computing nodes. The primary node identifies which files of the file subset each candidate computing node locally caches. The primary node selects one candidate computing node to execute the plan, based on the files of the file subset that each candidate computing node locally caches. The primary node causes the selected candidate computing node to execute the plan.