Self-healing distributed data storage method and system

Abstract

The invention discloses a self-healing distributed data storage method and system. The system comprises a message bus and a distributed cluster, the message bus is used for realizing cluster, node and data management; the distributed cluster comprises a master node and a plurality of slave nodes, and each node comprises a management module, a monitoring module and a storage module; the master node is used for processing all write operations, maintaining the latest state of data, and synchronizing data change to the slave nodes to ensure the consistency of the data; the slave nodes are used for data replication and fault recovery, and data consistency is maintained; when the master node breaks down, the slave nodes are promoted to be the master node according to a self-healing mechanism; the management module is used for communicating with a message bus to realize cluster management and system self-healing; the storage module is responsible for data storage, and data storage is optimized by adopting a data fragmentation strategy. According to the method, system self-healing can be automatically completed when a physical node or a network fails, and the characteristics of efficient access and high availability of the distributed data storage system are achieved.

Description

Technical Field

[0001] The present invention relates to the field of distributed data storage, and in particular to a self-healing distributed data storage method and system. Background Art

[0002] As a key component in modern computing architecture, distributed cache technology aims to solve the performance bottleneck caused by large-scale data processing and high concurrent access. Distributed cache technology can improve data access speed and enhance the scalability and fault tolerance of the system by storing data on multiple independent cache nodes.

[0003] Once the physical nodes or network of the distributed cache fails, the consistency of the distributed data storage will be destroyed, or the accessibility of the distributed nodes will be directly affected, resulting in the complete unavailability of the distributed cache.

[0004] Therefore, we propose a self-healing distributed data storage method and system, which is of great significance to improving the reliability of data storage. Summary of the invention

[0005] In view of the needs and shortcomings of current technological development, the present invention provides a self-healing distributed data storage method and system, which can automatically complete system self-healing when a physical node or network fails, thereby achieving efficient access and high availability characteristics of the distributed data storage system.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A self-healing distributed data storage system, comprising: a message bus, a distributed cluster;

[0008] The message bus is used to implement cluster, node, and data management, and provides a channel for upper-layer applications to interact with distributed data storage;

[0009] The distributed cluster includes a master node and several slave nodes. All nodes in the same cluster are in the same network segment. Each node includes a management module, a monitoring module and a storage module.

[0010] The master node is used to process all write operations and maintain the latest status of data, synchronizing data changes to slave nodes to ensure data consistency;

[0011] The slave nodes are used for data replication and fault recovery to maintain data consistency; when a master node fails, the slave node is promoted to the master node according to the self-healing mechanism;

[0012] The management module is used to communicate with the message bus to achieve cluster management and system self-healing;

[0013] The storage module is responsible for data storage and uses a data sharding strategy to optimize data storage.

[0014] Furthermore, the working process of the management module, the monitoring module and the message bus communication is as follows:

[0015] After the management module establishes a connection with the message bus, the management module periodically sends heartbeat messages to inform the message bus of the activity status of the node where the management module is located; the activity status of the node includes normal and abnormal; if the message bus fails to receive the heartbeat message sent by the management module within the threshold time, the message bus will consider the node status of the management module to be abnormal and delete the node information from the cluster status information table; when the message bus receives the heartbeat message sent by the management module, the message bus will consider the node status of the management module to be normal; if the node information of the management module does not exist in the cluster status information table at the current moment, it will be added to the cluster status information table; if there is a master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with a message containing the master node information; after the monitoring module receives the reply message from the message bus, it will set the node where the monitoring module is located as a slave node; if there is no master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with an empty message, and the monitoring module will automatically trigger the self-healing mechanism after receiving the reply.

[0016] Furthermore, the management module communicates with the message bus to implement cluster management. The specific steps are as follows:

[0017] Step (1.1), the message bus initializes the cluster status information table, which is used to store the status information of the cluster and nodes;

[0018] Step (1.2), the node adopts a heartbeat mechanism, and each management module in the cluster periodically sends a heartbeat message to the message bus, and the message includes the current state of the node; the message bus passively receives the heartbeat message and updates the node status information in the cluster status information table;

[0019] Step (1.3), the message bus provides cluster information, node information, and data to upper-layer applications.

[0020] Furthermore, in step (1.1), the cluster status information table includes: cluster name, master node name, master node address and a list of all nodes.

[0021] Furthermore, the management module communicates with the message bus to achieve system self-healing, and the specific steps are as follows:

[0022] When a message bus fails, the message bus self-healing mechanism is triggered: the management module is disconnected from the message bus, triggering a new message bus election process, and eventually generating a new message bus to achieve message bus self-healing;

[0023] When a cluster node crashes or actively goes offline, if the node is a slave node, no processing will be performed; if the node is the only slave node in the cluster, the message bus will trigger the cluster data migration mechanism, and the data of this cluster will be migrated to other clusters to ensure the integrity of distributed data storage; if the node is the master node, the master node self-healing mechanism will be triggered: based on the first-come-first-served principle, the first management module that detects that the master node is empty or in an abnormal state will promote its node to the master node. When the remaining nodes communicate with the message bus, they will be informed of the change in the master node and automatically synchronize data with the new master node to achieve master node self-healing, ensuring the consistency of cluster data and the stable operation of the system.

[0024] Furthermore, when a message bus fails, the message bus self-healing mechanism works as follows:

[0025] (2.1) When the management module detects a message bus failure, the node where the management module is located will mark itself as the initiating node;

[0026] (2.2) The management module of the initiating node broadcasts a message to generate a new message bus within the cluster;

[0027] (2.3) After receiving the broadcast message, the management modules of other nodes first check whether the old message bus is still alive;

[0028] (2.3a) If the management modules of other nodes believe that the old message bus is still alive, they will not reply to the broadcast notification;

[0029] (2.3b) If the management module of other nodes believes that the old message bus no longer exists, it will reply a confirmation message to the initiating node and switch the message bus address to the new message bus address;

[0030] (2.4) During the new message bus initiation cycle, the initiating node that receives confirmation replies from more than half of the nodes starts the new message bus. Otherwise, the initiating node will cancel the initiating node mark;

[0031] (2.5) After the new message bus is started, a broadcast notification will be sent to all clusters, and the notification content is the new message bus address;

[0032] (2.6) After receiving the broadcast notification, the management modules of all nodes in the cluster will connect to the new message bus.

[0033] Furthermore, the working process of the cluster data migration mechanism is as follows:

[0034] (3.1) The message bus identifies the cluster that needs to be taken offline, that is, the cluster to be migrated;

[0035] (3.2) The message bus uses the new calculation rules to perform hash consistency calculation on the data of the cluster to be migrated, which is used to redistribute the data of the cluster to be migrated to the new cluster; if the calculated hash value is the same as that of the cluster to be migrated, the new calculation rules are continued to be used for recalculation until the data is allocated to another cluster;

[0036] (3.3) The message bus sends a regrouping command and target cluster information to the cluster to be migrated;

[0037] (3.4) After receiving the migration command, the management module of the slave node in the cluster to be migrated goes offline directly;

[0038] (3.5) After receiving the migration command, the management module of the master node in the cluster to be migrated stores the data to the master node of the target cluster;

[0039] (3.6) After data transmission is completed, the master node of the cluster to be migrated goes offline;

[0040] (3.7) The slave nodes in the target cluster automatically synchronize data from the master node to ensure that the data on all nodes remains consistent.

[0041] Furthermore, when a master node fails, the master node self-healing mechanism works as follows:

[0042] (4.1) The management module of the slave node obtains from the message bus that the master node is empty and starts competing to become the master node;

[0043] (4.2) The management module of the slave node sends a lock request to the message bus to request an exclusive lock;

[0044] (4.3) After the message bus receives the lock request, the message bus checks whether the exclusive lock is free;

[0045] (4.3a.1) If yes, the message bus approves the first lock request that arrives, records the node information, and returns a "success" message to notify it that it has successfully obtained the exclusive lock;

[0046] (4.3a.2) The management module of the node that receives the notification of obtaining the exclusive lock sets this node as the master node;

[0047] (4.3b.1) If not, it means that another node has obtained the exclusive lock and has become the master node. The message bus will return the new master node information, indicating that the exclusive lock failed;

[0048] (4.3b.2) After receiving the failure message, the node sets itself as a slave node, updates the master node information, and starts synchronizing data;

[0049] (4.4) After obtaining the exclusive lock, the master node periodically sends a lock message to renew the lock. If the master node fails to successfully renew the lock within the threshold time, its exclusive lock will expire;

[0050] (4.5) When the master node fails to renew the lease or actively releases the lock, the exclusive lock will enter the idle state again, and the master node information of the cluster in the message bus will be cleared. After other nodes obtain the master node from the message bus and find that it is empty, they repeat the above steps to generate a new master node.

[0051] Furthermore, the data sharding strategy works as follows:

[0052] (5.1) The message bus divides the data into a specific number of data slices, which is equal to the number of clusters;

[0053] (5.2) The message bus performs hash calculation on each data item according to its key value, and the hash value range is [0, N-1], where N is the number of clusters;

[0054] (5.3) The message bus maps the data item to the corresponding cluster based on the calculated hash value.

[0055] A self-healing distributed data storage method is implemented using the above system, and the method comprises the following steps:

[0056] The management module communicates with the message bus to achieve cluster management and system self-healing;

[0057] The storage module is responsible for data storage and uses a data sharding strategy to optimize data storage;

[0058] The specific working process of the communication between the management module, the monitoring module and the message bus is as follows:

[0059] After the management module establishes a connection with the message bus, the management module periodically sends heartbeat messages to inform the message bus of the activity status of the node where the management module is located; the activity status of the node includes normal and abnormal; if the message bus fails to receive the heartbeat message sent by the management module within the threshold time, the message bus will consider the node status of the management module to be abnormal and delete the node information from the cluster status information table; when the message bus receives the heartbeat message sent by the management module, the message bus will consider the node status of the management module to be normal; if the node information of the management module does not exist in the cluster status information table at the current moment, it will be added to the cluster status information table; if there is a master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with a message containing the master node information; after the monitoring module receives the reply message from the message bus, it will set the node where the monitoring module is located as a slave node; if there is no master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with an empty message, and the monitoring module will automatically trigger the self-healing mechanism after receiving the reply.

[0060] The beneficial effects produced by the present invention are:

[0061] The present invention realizes efficient access and high availability characteristics of distributed data storage system based on message bus self-healing mechanism, cluster data migration mechanism and master node self-healing mechanism. Specifically, the message bus and cluster master node have self-healing characteristics. When the message bus or master node fails, the entire distributed storage structure can be maintained intact; when there are two or more nodes alive in any cluster, the distributed data storage system can work normally; at the same time, the distributed data storage system has read-write separation characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the structure of a self-healing distributed data storage system according to an embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram of the communication between the management module and the message bus according to an embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of the message bus self-healing mechanism flow in an embodiment of the present invention;

[0065] Figure 4 is a schematic diagram of a cluster data migration mechanism according to an embodiment of the present invention;

[0066] Figure 5 It is a timing diagram of the master node self-healing mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] See also Figure 1 As shown, a self-healing distributed data storage system according to an embodiment of the present invention includes: a message bus and a distributed cluster;

[0069] The message bus is used to implement cluster, node, and data management, and provides a channel for upper-layer applications to interact with distributed data storage;

[0070] The distributed cluster includes a master node and several slave nodes. All nodes in the same cluster are in the same network segment. Each node includes a management module, a monitoring module and a storage module.

[0071] The master node is used to process all write operations and maintain the latest status of data, synchronizing data changes to slave nodes to ensure data consistency;

[0072] The slave node is used for data replication and fault recovery to maintain data consistency; when a master node fails, the slave node can be promoted to the master node according to the self-healing mechanism;

[0073] The management module is used to communicate with the message bus to achieve cluster management and system self-healing;

[0074] The storage module is responsible for data storage and uses a data sharding strategy to optimize data storage;

[0075] Combination Figure 2 ,The working process of communication between the management module and the message bus is as follows:

[0076] After the management module establishes a connection with the message bus, the management module periodically sends heartbeat messages to inform the message bus of the activity status of the node where the management module is located; the activity status of the node only includes normal and abnormal. α If the heartbeat message sent by the management module is not received within 10 seconds, the message bus will consider the node where the management module is located to be abnormal and delete the node information from the cluster status information table. αGenerally, 2T1 is used. When the message bus receives the heartbeat message sent by the management module, the message bus will consider that the status of the node where the management module is located is normal; if the node information where the management module is located at the current moment does not exist in the cluster status information table, it will be added to the cluster status information table; if there is a master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with a message containing the master node information. After the monitoring module receives the reply message from the message bus, it sets the node where the monitoring module is located as a slave node; if there is no master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus replies with an empty message, and the monitoring module automatically triggers the self-healing mechanism after receiving the reply.

[0077] Specifically, the management module communicates with the message bus to implement cluster management. The specific steps are as follows:

[0078] (1.1) The message bus initializes the cluster status information table, which is used to store the status information of the cluster and nodes.

[0079] (1.2) The nodes adopt a heartbeat mechanism. Each management module in the cluster periodically sends a heartbeat message to the message bus. The message includes the current status of the node. The message bus passively receives these status reports (heartbeat messages) and updates the node status information in the cluster status information table.

[0080] (1.3) Provide cluster information, node information, and data to upper-level applications.

[0081] Specifically, the management module communicates with the message bus to achieve system self-healing, and the specific steps are as follows:

[0082] On the one hand, when the message bus fails, the message bus self-healing mechanism is triggered: the management module and the message bus will be disconnected, triggering a new message bus election process, and finally generating a new message bus to achieve message bus self-healing. On the other hand, when a cluster node crashes or is actively offline, if the node is a slave node, no processing will be done; if the node is the only slave node in the cluster, the message bus will trigger the cluster data migration mechanism, and the data of this cluster will be migrated to other clusters to ensure the integrity of distributed data storage; if the node is the master node, the master node self-healing mechanism is triggered: according to the first-come-first-served principle, the first management module that detects that the master node is empty or abnormal will promote its node to the master node. When the remaining nodes communicate with the message bus, they will be informed of the change in the master node, and automatically synchronize data with the new master node to achieve master node self-healing, ensuring the consistency of cluster data and the stable operation of the system.

[0083] Combination Figure 3 When a message bus fails, the message bus self-healing mechanism works as follows:

[0084] (2.1) When the management module detects a message bus failure, the node where the management module is located will mark itself as the initiating node.

[0085] (2.2) The management module of the initiating node broadcasts a message to generate a new message bus within the cluster.

[0086] (2.3) After receiving the broadcast message, the management modules of other nodes first check whether the old message bus is still alive.

[0087] (2.3a) If the management modules of other nodes believe that the old message bus is still alive, they will not reply to the broadcast notification.

[0088] (2.3b) If the management module of other nodes believes that the old message bus no longer exists, it will reply a confirmation message to the initiating node and switch the message bus address to the new message bus address.

[0089] (2.4) During the new message bus initiation cycle, the initiating node that receives confirmation replies from more than half of the nodes starts the new message bus. Otherwise, the initiating node will cancel the initiating node mark.

[0090] (2.5) After the new message bus is started, a broadcast notification will be sent to all clusters. The notification content is the new message bus address.

[0091] (2.6) After receiving the broadcast notification, the management modules of all nodes in the cluster will connect to the new message bus.

[0092] Combination Figure 4 The working process of the cluster data migration mechanism is as follows:

[0093] (3.1) The message bus identifies the cluster that needs to be taken offline, that is, the cluster to be migrated.

[0094] (3.2) The message bus uses the new calculation rules to perform hash consistency calculation on the data of the cluster to be migrated, which is used to redistribute the data of the cluster to be migrated to the new cluster; if the calculated hash value is the same as that of the cluster to be migrated, the new calculation rules are continued to be used for recalculation until the data is allocated to another cluster.

[0095] (3.3) The message bus sends the regrouping command and target cluster information to the cluster to be migrated.

[0096] (3.4) After receiving the migration command, the management module of the slave node in the cluster to be migrated goes offline directly.

[0097] (3.5) After receiving the migration command, the management module of the master node in the cluster to be migrated stores the data to the master node of the target cluster.

[0098] (3.6) After the data transfer is completed, the master node of the cluster to be migrated goes offline.

[0099] (3.7) The slave nodes in the target cluster automatically synchronize data from the master node to ensure that the data on all nodes remains consistent.

[0100] Combination Figure 5 When a master node fails, the working process of the master node self-healing mechanism is as follows:

[0101] (4.1) The management module of the slave node obtains from the message bus that the master node is empty and starts competing to become the master node.

[0102] (4.2) The management module of the slave node sends a lock request to the message bus. The lock request uses "lock" as the message to request an exclusive lock.

[0103] (4.3) After the message bus receives the lock request, it checks whether the exclusive lock is free.

[0104] (4.3a.1) If yes, the message bus approves the first lock request that arrives, records the node information, and returns a "success" message to notify it that it has successfully obtained the exclusive lock.

[0105] (4.3a.2) The management module of the node that receives the exclusive lock notification sets this node as the master node.

[0106] (4.3b.1) If not, it means that another node has obtained the exclusive lock and has become the master node. The message bus will return the new master node information, indicating that the exclusive lock failed.

[0107] (4.3b.2) After receiving the failure message, the node sets itself as a slave node, updates the master node information, and starts synchronizing data.

[0108] (4.4) After obtaining the exclusive lock, the master node needs to periodically send lock messages to renew the lock. If the master node fails to successfully renew the lock within the threshold time, its exclusive lock will expire.

[0109] (4.5) When the master node fails to renew the lease or actively releases the lock, the exclusive lock will enter the idle state again, and the master node information of the cluster in the message bus will be cleared. After other nodes obtain the master node from the message bus and find that it is empty, they repeat the above steps to generate a new master node.

[0110] The working process of the data sharding strategy is as follows:

[0111] (5.1) The message bus divides the data into a specific number of data slices, which is equal to the number of clusters.

[0112] (5.2) The message bus performs hash calculation on each data item based on its key value. The formula is: The hash value range is [0, N-1], where N is the number of clusters.

[0113] (5.3) The message bus maps the data item to the corresponding cluster based on the calculated hash value.

[0114] The embodiment of the present invention further provides a self-healing distributed data storage method, which is performed using the above system, and the method comprises the following steps:

[0115] The management module communicates with the message bus to achieve cluster management and system self-healing;

[0116] The storage module is responsible for data storage and uses a data sharding strategy to optimize data storage;

[0117] The specific working process of the communication between the management module, the monitoring module and the message bus is as follows:

[0118] After the management module establishes a connection with the message bus, the management module periodically sends heartbeat messages to inform the message bus of the activity status of the node where the management module is located; the activity status of the node includes normal and abnormal; if the message bus fails to receive the heartbeat message sent by the management module within the threshold time, the message bus will consider the node status of the management module to be abnormal and delete the node information from the cluster status information table; when the message bus receives the heartbeat message sent by the management module, the message bus will consider the node status of the management module to be normal; if the node information of the management module does not exist in the cluster status information table at the current moment, it will be added to the cluster status information table; if there is a master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with a message containing the master node information; after the monitoring module receives the reply message from the message bus, it will set the node where the monitoring module is located as a slave node; if there is no master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with an empty message, and the monitoring module will automatically trigger the self-healing mechanism after receiving the reply

[0119] The distributed data storage method and system of the present invention have a message bus self-healing mechanism, a master node self-healing mechanism and a cluster data migration mechanism, and when two or more nodes survive in any cluster, the system can work normally. When the message bus fails, the system can trigger the message bus self-healing mechanism, generate a new message bus, realize the self-healing of the message bus, and ensure the high availability of the system; when the master node fails, the system can trigger the master node self-healing mechanism, so that a specific slave node becomes the master node, ensuring the consistency of cluster data and the high availability of the system; when all slave nodes of the cluster fail, the system can trigger the cluster data migration mechanism to ensure the integrity of distributed data storage.

[0120] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A self-healing distributed data storage method, characterized in that: Including message bus and distributed cluster; The message bus is used to implement cluster, node, and data management, and provides a channel for upper-layer applications to interact with distributed data storage; The distributed cluster includes a master node and several slave nodes. All nodes in the same cluster are in the same network segment. Each node includes a management module, a monitoring module and a storage module. The master node is used to process all write operations and maintain the latest status of data, synchronizing data changes to slave nodes to ensure data consistency; The slave nodes are used for data replication and fault recovery to maintain data consistency; When the master node fails, the slave node is promoted to the master node according to the self-healing mechanism; The management module is used to communicate with the message bus to achieve cluster management and system self-healing; The storage module is responsible for data storage and uses a data sharding strategy to optimize data storage.

2. The self-healing distributed data storage system according to claim 1, characterized in that: The specific working process of the communication between the management module, the monitoring module and the message bus is as follows: After the management module establishes a connection with the message bus, the management module periodically sends heartbeat messages to inform the message bus of the activity status of the node where the management module is located; the activity status of the node includes normal and abnormal; if the message bus fails to receive the heartbeat message sent by the management module within the threshold time, the message bus will consider the node where the management module is located to be abnormal and delete the node information from the cluster status information table; when the message bus receives the heartbeat message sent by the management module, the message bus will consider the node where the management module is located to be normal; If the node information where the management module is located at the current moment does not exist in the cluster status information table, it will be added to the cluster status information table; if there is a master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with a message containing the master node information; after the monitoring module receives the reply message from the message bus, it will set the node where the monitoring module is located as a slave node; At the current moment, there is no master node in the cluster corresponding to the node where the management module is located. The message bus replies with an empty message. After receiving the reply, the monitoring module automatically triggers the self-healing mechanism.

3. The self-healing distributed data storage system according to claim 1, characterized in that: The management module communicates with the message bus to implement cluster management. The specific steps are as follows: Step (1.1), the message bus initializes the cluster status information table, which is used to store the status information of the cluster and nodes; Step (1.2), the node adopts a heartbeat mechanism, and each management module in the cluster periodically sends a heartbeat message to the message bus, and the message includes the current state of the node; The message bus passively receives heartbeat messages and updates the node status information in the cluster status information table; Step (1.3), the message bus provides cluster information, node information, and data to upper-layer applications.

4. The self-healing distributed data storage system according to claim 3, characterized in that: In step (1.1), the cluster status information table includes: cluster name, master node name, master node address and a list of all nodes.

5. The self-healing distributed data storage system according to claim 1, wherein: The management module communicates with the message bus to achieve system self-healing. The specific steps are as follows: When a message bus fails, the message bus self-healing mechanism is triggered: the management module is disconnected from the message bus, triggering a new message bus election process, and eventually generating a new message bus to achieve message bus self-healing; When a cluster node crashes or goes offline actively, if the node is a slave node, no processing is done; If the node is the only slave node in the cluster, the message bus will trigger the cluster data migration mechanism, and the data of this cluster will be migrated to other clusters to ensure the integrity of distributed data storage; If the node is the master node, the master node self-healing mechanism is triggered: according to the first-come-first-served principle, the first management module that detects that the master node is empty or in an abnormal state will promote its node to the master node. The remaining nodes will be informed of the change of the master node when communicating with the message bus, and automatically synchronize data with the new master node to achieve master node self-healing, ensuring the consistency of cluster data and the stable operation of the system.

6. The self-healing distributed data storage system according to claim 5, characterized in that: When a message bus fails, the message bus self-healing mechanism works as follows: (2.1) When the management module detects a message bus failure, the node where the management module is located will mark itself as the initiating node; (2.2) The management module of the initiating node broadcasts a message to generate a new message bus within the cluster; (2.3) After receiving the broadcast message, the management modules of other nodes first check whether the old message bus is still alive; (2.3a) If the management modules of other nodes believe that the old message bus is still alive, they will not reply to the broadcast notification; (2.3b) If the management module of other nodes believes that the old message bus no longer exists, it will reply a confirmation message to the initiating node and switch the message bus address to the new message bus address; (2.4) During the new message bus initiation cycle, the initiating node that receives confirmation replies from more than half of the nodes starts the new message bus. Otherwise, the initiating node will cancel the initiating node mark; (2.5) After the new message bus is started, a broadcast notification will be sent to all clusters, and the notification content is the new message bus address; (2.6) After receiving the broadcast notification, the management modules of all nodes in the cluster will connect to the new message bus.

7. The self-healing distributed data storage system according to claim 5, characterized in that: The working process of the cluster data migration mechanism is as follows: (3.1) The message bus identifies the cluster that needs to be taken offline, that is, the cluster to be migrated; (3.2) The message bus uses a new calculation rule to perform hash consistency calculation on the data of the cluster to be migrated, which is used to redistribute the data of the cluster to be migrated to the new cluster; If the calculated hash value is the same as that of the cluster to be migrated, the new calculation rule is used to recalculate the data until the data is allocated to another cluster. (3.3) The message bus sends a regrouping command and target cluster information to the cluster to be migrated; (3.4) After receiving the migration command, the management module of the slave node in the cluster to be migrated goes offline directly; (3.5) After receiving the migration command, the management module of the master node in the cluster to be migrated stores the data to the master node of the target cluster; (3.6) After data transmission is completed, the master node of the cluster to be migrated goes offline; (3.7) The slave nodes in the target cluster automatically synchronize data from the master node to ensure that the data on all nodes remains consistent.

8. The self-healing distributed data storage system according to claim 5, characterized in that: When a master node fails, the master node self-healing mechanism works as follows: (4.1) The management module of the slave node obtains from the message bus that the master node is empty and starts competing to become the master node; (4.2) The management module of the slave node sends a lock request to the message bus to request an exclusive lock; (4.3) After the message bus receives the lock request, the message bus checks whether the exclusive lock is free; (4.3a.1) If yes, the message bus approves the first lock request that arrives, records the node information, and returns a "success" message to notify it that the exclusive lock has been successfully obtained; (4.3a.2) The management module of the node that receives the notification of obtaining the exclusive lock sets this node as the master node; (4.3b.1) If not, it means that another node has obtained the exclusive lock and has become the master node. The message bus will return the new master node information, indicating that the exclusive lock failed; (4.3b.2) After receiving the failure message, the node sets itself as a slave node, updates the master node information, and starts synchronizing data; (4.4) After obtaining the exclusive lock, the master node periodically sends a lock message to renew the lock. If the master node fails to successfully renew the lock within the threshold time, its exclusive lock will expire; (4.5) When the master node fails to renew the lease or actively releases the lock, the exclusive lock will enter the idle state again, and the master node information of the cluster in the message bus will be cleared. After other nodes obtain the master node from the message bus and find that it is empty, they repeat the above steps to generate a new master node.

9. The self-healing distributed data storage system according to claim 1, characterized in that: The working process of the data sharding strategy is as follows: (5.1) The message bus divides the data into a specific number of data slices, which is equal to the number of clusters; (5.2) The message bus performs hash calculation on each data item according to its key value, and the hash value range is [0, N-1], where N is the number of clusters; (5.3) The message bus maps the data item to the corresponding cluster based on the calculated hash value.

10. A self-healing distributed data storage method, characterized in that: The method is performed using the system according to any one of claims 1 to 8, wherein the method comprises the following steps: The management module communicates with the message bus to achieve cluster management and system self-healing; The storage module is responsible for data storage and uses a data sharding strategy to optimize data storage; The specific working process of the communication between the management module, the monitoring module and the message bus is as follows: After the management module establishes a connection with the message bus, the management module periodically sends heartbeat messages to inform the message bus of the activity status of the node where the management module is located; the activity status of the node includes normal and abnormal; if the message bus fails to receive the heartbeat message sent by the management module within the threshold time, the message bus will consider the node where the management module is located to be abnormal and delete the node information from the cluster status information table; when the message bus receives the heartbeat message sent by the management module, the message bus will consider the node where the management module is located to be normal; If the node information where the management module is located at the current moment does not exist in the cluster status information table, it will be added to the cluster status information table; if there is a master node in the cluster corresponding to the node where the management module is located at the current moment, the message bus will reply with a message containing the master node information; after the monitoring module receives the reply message from the message bus, it will set the node where the monitoring module is located as a slave node; At the current moment, there is no master node in the cluster corresponding to the node where the management module is located. The message bus replies with an empty message. After receiving the reply, the monitoring module automatically triggers the self-healing mechanism.

Images