Coordinator Node Caching for Database Cluster Preheating
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In a cluster shared disk architecture, resources cannot be rapidly obtained from a high-speed coordinator node during node failures or additions, leading to inefficient processing node preheating due to underutilization of storage space and communication features like InfiniBand.
Innovation Solution
A resource coordination method for a database cluster that determines idle resources on the coordinator node and instructs processing nodes to upload clean or dirty page data accordingly, optimizing data caching and storage to fully utilize idle resources and improve efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the coordinator node is used only for coordination without sharing storage space, then data integrity and coordination functionality are ensured, but storage space on the coordinator node and high-speed communication features cannot be fully used, resulting in waste
Solution Approach 1:
The coordinator node is given multiple functions: it continues to perform coordination duties while also serving as a storage node for clean page data. This multi-functionality allows the coordinator node to fully utilize its storage space and high-speed communication capabilities, transforming previously idle resources into useful assets for the database cluster.
Solution Approach 2:
The patent merges the coordination function and storage function into a single coordinator node. By combining these previously separate functions, the system eliminates resource waste while maintaining data integrity through the coordinator's continued coordination role and enhancing storage efficiency through its participation in clean page storage.
2Reliability
If all data is obtained from disk array during node failure or addition, then data reliability is maintained, but processing node preheating efficiency is low due to slow data retrieval speed
Solution Approach 1:
The coordinator node pre-stores clean page data using its high-speed storage capabilities before node failures or additions occur. When a processing node needs to be restarted or added, the pre-stored clean data can be rapidly retrieved from the coordinator node through high-speed communication, avoiding the need to read from the slower disk array and thus accelerating the preheating process.
Solution Approach 2:
The coordinator node acts as an intermediary between the disk array and processing nodes during data retrieval operations. Instead of processing nodes directly accessing the slow disk array, they obtain clean page data from the coordinator node which has already retrieved and cached the data, thereby improving data retrieval speed while maintaining data reliability.
3Productivity
If the coordinator node stores clean page data from processing nodes, then idle resources are fully utilized and preheating efficiency is improved, but the coordinator node's workload and resource management complexity increase
Solution Approach 1:
The system implements feedback mechanisms where processing nodes report their dirty page data status to the coordinator node. Based on this feedback, the coordinator node intelligently determines which clean page data to request from processing nodes and how to manage its storage resources, thereby reducing manual configuration and simplifying resource management despite the increased functionality.
Solution Approach 2:
Processing nodes automatically identify and upload their clean page data to the coordinator node without requiring manual intervention. The coordinator node autonomously manages the received clean data, determining storage allocation and retrieval strategies, thereby reducing operational complexity while enhancing preheating efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A resource coordination method, an apparatus, and a system for a database cluster are provided. The method includes: An active coordinator node obtains status information corresponding to each processing node in multiple processing nodes (201), where the status information is used to indicate an operating load status of the processing node; determines, according to the status information corresponding to each processing node in multiple processing nodes, whether the active coordinator node has an idle resource whose capacity is a preset threshold X (203); and if the active coordinator node has the idle resource whose capacity is the preset threshold X, instructs each processing node to upload subsequently generated clean page data to the active coordinator node (205). More data generated by a processing node can be stored in a coordinator node by using this method. An idle resource on the coordinator node and a high-speed communication feature of a communication interface between the coordinator node and the processing node are fully used, so that starting of the processing node is more rapid and efficient.