Cloud Storage Access Node Fault Tolerance via Pre-Saved Index
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud storage systems face service disruptions and data loss when all management nodes are at fault or the management node cluster network is abnormal, as existing solutions like increasing management nodes or using HA/cluster technology are costly and do not guarantee system operation in such scenarios.
Innovation Solution
A data writing method and apparatus that utilize a pre-saved index to identify a target resource pool and storage server for data write requests, even when the management node is unavailable, by querying domain identification information and determining the storage server with the lowest current load for optimal load balancing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of management nodes is increased or standby management nodes are added to ensure system stability, then the reliability of the cloud storage system is improved, but the project cost increases
Solution Approach 1:
The access node pre-obtains and stores the mapping relationship between resource pools and domain identification information before management nodes fail. This preliminary action enables the access node to independently determine target storage servers without requiring management nodes during normal operation, thereby ensuring system reliability without adding redundant management nodes or increasing project cost.
2Reliability
If management nodes are added or standby management nodes are configured to take over tasks, then the cloud storage system can continue to provide services when some management nodes are at fault, but the cloud storage system cannot guarantee normal operation when all management nodes are at fault or the management node cluster network is abnormal
Solution Approach 1:
The access node is equipped with the capability to independently determine target storage servers by using pre-obtained domain identification information and resource pool mapping relationships. This self-service mechanism allows the access node to bypass faulty management nodes entirely and directly access storage servers, ensuring service continuity even when all management nodes are at fault or the management node cluster network is abnormal.
3Productivity
If the access node queries the management node for storage server allocation, then the data processing can be performed, but when the management node is at fault, the data processing will fail
Solution Approach 1:
Domain identification information serves as an intermediary that bridges the access node and storage servers. The access node uses this intermediary to directly locate and access target storage servers without requiring management node intervention, thereby maintaining high data processing efficiency while eliminating the single point of failure represented by the management node.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
A data writing and reading method and apparatus, and a cloud storage system are provided in embodiments of the present application. The data writing method includes: the access node receiving a data write request (301); identifying domain identification information contained in the data write request when failing to obtain an available management node, and querying a pre-saved index according to the domain identification information, wherein the index includes a correspondence between a resource pool and domain identification information (302); identifying a target resource pool that matches the domain identification information, wherein each resource pool includes at least one storage server (303); determining, from at least one storage server included in the target resource pool, one of the storage servers as a target storage server, and writing data to be written into the target storage server (304). The embodiments of the present application can ensure that the cloud storage system can still operate normally without additional cost when all management nodes are at fault or the management node cluster network is abnormal.