A method, device, and equipment for processing changes in cluster nodes based on cluster versions

Abstract

The present invention provides a method, apparatus, and device for processing changes in cluster nodes based on a cluster version. The method includes the following steps: Manage the nodes in the cluster and record the version information of the nodes in the current cluster; When data is sent to the cluster, perform a hash calculation according to the current version information to calculate the location of the data object and store the data normally; When it is sensed that there is a change in the nodes in the cluster, send a notification to each node in the cluster; After receiving the notification, each node updates the version information of the cluster on its own node; Retrieve the data corresponding to the location under the version information of the previous cluster version, and re-balance the data to each node according to the new version information. After the balancing is completed, delete the data corresponding to the location under the version information of the previous cluster version. In this way, the function of dynamically adding and deleting cluster nodes is realized. From a functional perspective, the product function and the competitiveness of the product are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster node management, and particularly relates to a method, device, and equipment for processing cluster node changes based on a cluster version. Background Art

[0002] In the current era of rapid development of the Internet, unstructured data such as pictures, videos, music, log files, backup data, containers, virtual machine images, etc. has exploded and continues to grow at a high speed. All sectors of society are actively looking for methods to effectively and economically store and access unstructured data, which is undoubtedly a challenge. Minio was born to meet such data storage requirements. It is compatible with the API of Amazon S3 cloud storage service and builds a high-performance infrastructure for machine learning, analysis, and application data workloads.

[0003] However, due to the simple design of Minio, there is no master node in the cluster and the hash positioning method is used to find the location of data objects when accessing data, resulting in that after the Minio cluster is built, the number of cluster nodes cannot be changed. Once the number of nodes changes, the data location calculated by the hash changes, and the location is disordered, resulting in the inability to obtain the correct data. And to expand the capacity, it can only be expanded in the form of federation or adding a fixed number of nodes at a time, which is obviously not flexible and convenient enough in use and management. Summary of the Invention

[0004] Aiming at the problem that once the number of nodes changes, the data location calculated by the hash changes, and the location is disordered, resulting in the inability to obtain the correct data, the present invention provides a method, device, and equipment for processing cluster node changes based on a cluster version.

[0005] The technical solution of the present invention is as follows:

[0006] In a first aspect, the technical solution of the present invention provides a method for processing cluster node changes based on a cluster version, including the following steps:

[0007] Manage the nodes in the cluster and record the version information of the nodes in the current cluster;

[0008] When data is sent to the cluster, perform hash calculation according to the current version information, calculate the location of the data object, and store the data normally;

[0009] When it is sensed that there is a change in the nodes in the cluster, send a notification to each node in the cluster;

[0010] After receiving the notification, each node updates the version information of the cluster on its own node;

[0011] Retrieve the data at the corresponding position under the version information of the previous cluster version, and rebalance the data to each node according to the new version information.

[0012] Record the version information of the nodes in the current cluster. When new data is sent down, the cluster performs a hash calculation according to the current version information, calculates the position of the data object, and stores the data normally; when a node leaves or joins the cluster, the event is sensed and notified to all the current nodes in the cluster. Then all the live nodes update the version information of the cluster on their own nodes according to the offline node information, calculate the hash based on the number of nodes in the previous cluster version and the version information of the current cluster version, retrieve the data from under the version information of the previous cluster version, and rebalance the data to each node according to the new version information. Based on the above method, handle the dynamic change events of the cluster nodes.

[0013] Preferably, the version information includes the id information, ip information, disk information of the cluster nodes and the cluster version number. Among them, each time the cluster nodes change, the cluster version number is incremented by 1. The cluster version number is set to facilitate the processing of the information on the changes of the cluster nodes.

[0014] Preferably, after the step of retrieving the data at the corresponding position under the version information of the previous cluster version and rebalancing the data to each node according to the new version information, it further includes:

[0015] After the balancing is completed, delete the data at the corresponding position under the version information of the previous cluster version. For the convenience of data access and storage.

[0016] Preferably, the step of each node updating the version information of the cluster on its own node after receiving the notification includes:

[0017] After receiving the notification, according to the change information, add the relevant information of the newly added nodes or delete the relevant information of the nodes that have left, and update the cluster version number.

[0018] Preferably, the step of retrieving the data at the corresponding position under the version information of the previous cluster version and rebalancing the data to each node according to the new version information includes:

[0019] Perform a hash calculation according to the version information of the previous cluster version to obtain the data in the cluster;

[0020] Perform a hash calculation according to the new version information, and redistribute the data in the cluster to each node.

[0021] Preferably, the step of managing the nodes in the cluster and recording the version information of the nodes in the current cluster includes:

[0022] When each node joins the cluster and starts up, apply for a globally unique id;

[0023] Generate a storage key value according to the ID and the node IP, store it in the database, and set the lease of the key value; and set the lease of the key value to be extended regularly when the cluster starts.

[0024] Record the version information of the nodes in the current cluster.

[0025] Preferably, when it is sensed that there is a change in the nodes in the cluster, the step of sending a notification to each node in the cluster includes:

[0026] When a node applies for an ID, it is determined that a node has joined the cluster.

[0027] When the lease of the storage key value of a node expires, the storage key value will be automatically deleted, and it is determined that the node has left the cluster.

[0028] Record the information of the MinIO cluster, including the current number of nodes in the cluster, the information of each node, such as the node IP, disk information, and the current epoch version number. When there is a node joining or leaving the cluster and this event is sensed, according to the event type (node going online or leaving) and the information of the node in the event and the information of the current epoch in the cluster, add the node information or delete the corresponding node information in the current epoch, increase the cluster version number, and then save it as the latest epoch of the cluster. Then each node calculates the hash based on the information of the previous cluster version epoch to obtain the data existing on the current cluster, and calculates the hash based on the information of the current new epoch, and redistributes the cluster data to each node in the cluster, so as to realize the dynamic processing of node changes in the MinIO cluster.

[0029] In a second aspect, the technical solution of the present invention provides a cluster node change processing device based on the cluster version, including a cluster management module, an output processing module, and a node information processing module;

[0030] The cluster management module is used to manage the nodes in the cluster and record the version information of the nodes in the current cluster; when it is sensed that there is a change in the nodes in the cluster, send a notification to each node in the cluster; wherein, the version information includes the ID information, IP information, disk information, and cluster version number of the cluster nodes. Each time there is a change in the cluster nodes, the cluster version number is incremented by 1.

[0031] The output processing module is used to perform hash calculation according to the current version information when data is sent to the cluster, calculate the position of the data object, and store the data normally;

[0032] The node information processing module is used to update the version information of the cluster on the local node after receiving the notification; take out the data at the corresponding position under the version information of the previous cluster version, and rebalance the data to each node according to the new version information.

[0033] Preferably, the node information processing module is further configured to, after the balancing is completed, delete the data at the corresponding position under the version information of the previous cluster version.

[0034] Preferably, the node information processing module is specifically configured to, after receiving the notification, according to the change information, add the relevant information of the newly added node or delete the relevant information of the node that has left, and update the cluster version number; specifically, it is further configured to perform a hash calculation according to the version information of the previous cluster version to obtain the data in the cluster; perform a hash calculation according to the new version information, and redistribute the data in the cluster to each node.

[0035] Preferably, the cluster management module is specifically configured to, when each node joins the cluster and starts up, allocate a globally unique id; generate a storage key value according to the id and the node ip, store it in the database, and set the lease of the key value; and set a timer to extend the lease of the key value when the cluster starts up; record the version information of the nodes in the current cluster; specifically, when a node applies for an id, it is determined that a node has joined the cluster; when the lease of the storage key value of the node expires, the storage key value will be automatically deleted, and it is determined that the node has left the cluster.

[0036] When a node leaves or joins the cluster, after perceiving this event, it notifies each node in the cluster. Each node updates the epoch information of the cluster on its own node (add or delete the corresponding node information in the epoch) according to the added or deleted node information, and recalculates the hash according to the epoch information of the previous cluster version and the epoch information of the current cluster version, retrieves the data according to the epoch information of the previous cluster version, and redistributes the data to each node according to the new epoch information. After the balancing is completed, just delete the data at the corresponding position under the epoch of the previous cluster version. In this way, the function of dynamically adding and deleting Minio nodes is realized. From a functional perspective, the product function and the competitiveness of the product are improved.

[0037] In a third aspect, the technical solution of the present invention provides a computer device, including a processor and a memory. The processor and the memory complete communication with each other through a bus; the memory stores program instructions executable by the processor, and the processor can execute the cluster node change processing method based on the cluster version as described in the first aspect by invoking the program instructions.

[0038] As can be seen from the above technical solutions, the present invention has the following advantages: when it is sensed that a node leaves or joins the cluster, the information of the leaving or joining node is notified to each node in the cluster. Each node updates the version information of the cluster on its own node according to the added or deleted node information, and recalculates the hash value according to the version information of the previous cluster version and the version information of the current cluster version. The data is retrieved according to the version information of the previous version, and the data is rebalanced to each node according to the new version information. After the balancing is completed, the data corresponding to the position under the version information of the previous cluster version can be deleted. In this way, the function of dynamically adding and deleting cluster nodes is realized. From a functional perspective, the product function and the competitiveness of the product are improved.

[0039] In addition, the design principle of the present invention is reliable, the structure is simple, and it has a very broad application prospect.

[0040] Thus, compared with the prior art, the present invention has prominent substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0042] Figure 1 It is a schematic flowchart of the method according to an embodiment of the present invention.

[0043] Figure 2 It is a schematic block diagram of the device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0045] The embodiment of the present invention provides a method for processing cluster node changes based on cluster versions, including the following steps:

[0046] Step 11: Manage the nodes in the cluster and record the version information of the nodes in the current cluster;

[0047] In this step, it should be noted that the version information includes the id information, ip information, disk information of the cluster nodes, and the cluster version number. Among them, each time a cluster node changes, the cluster version number is incremented by 1. The cluster version number is set to facilitate the processing of information on cluster node changes.

[0048] Step 12: When data is sent to the cluster, perform a hash calculation according to the current version information, calculate the location of the data object, and store the data normally;

[0049] Step 13: When it is sensed that there is a change in the nodes in the cluster, send a notification to each node in the cluster;

[0050] Step 14: After receiving the notification, each node updates the version information of the cluster on its own node;

[0051] Step 15: Retrieve the data at the corresponding location under the version information of the previous cluster version, and re - balance the data to each node according to the new version information.

[0052] Record the version information of the nodes in the current cluster. When new data is sent, the cluster performs a hash calculation according to the current version information, calculates the location of the data object, and stores the data normally; when a node leaves or joins the cluster, this event is sensed and notified to all current nodes in the cluster. Then all the surviving nodes update the version information of the cluster on their own nodes according to the offline node information, calculate the hash based on the number of nodes in the previous cluster version and the version information of the current cluster version, retrieve the data from under the version information of the previous cluster version, and re - balance the data to each node according to the new version information. Based on the above method, dynamic change events of cluster nodes are processed.

[0053] As Figure 1 shown, an embodiment of the present invention provides a method for processing cluster node changes based on cluster version, including the following steps:

[0054] Step 21: Manage the nodes in the cluster and record the version information of the nodes in the current cluster;

[0055] It should be noted that the version information includes the id information, ip information, disk information of the cluster nodes, and the cluster version number. Among them, each time a cluster node changes, the cluster version number is incremented by 1. The cluster version number is set to facilitate the processing of information on cluster node changes;

[0056] In this step, the steps for managing the nodes in the cluster include: when each node joins the cluster and starts up, apply for a globally unique id; generate a storage key - value according to this id and the node ip, store it in the database, and set the lease of this key - value; and set a timer to extend the lease of this key - value when the cluster starts up.

[0057] In the specific implementation process, this method manages the MinIO cluster by leveraging the etcd cluster management function. Each MinIO node uses the revision mechanism of etcd or a globally unique ID to form the key of the node with this ID and the node IP, such as ip / xxxxx / id, and stores it in the etcd database and sets a lease. The lease period of this lease is adjustable. When MinIO starts, it generates this key and periodically extends the lease period of this key, so that the lease does not expire, which means the node is alive; when the node fails, it cannot renew the lease for this key, and this key will be deleted by etcd.

[0058] Step 22: When data is sent to the cluster, perform a hash calculation according to the current version information, calculate the location of the data object, and store the data normally;

[0059] Step 23: When it is sensed that there are changes in the nodes in the cluster, send a notification to each node in the cluster;

[0060] In this step, when a node applies for an ID, it is judged that a node joins the cluster; when the lease of the storage key value of a node expires, this storage key value will be automatically deleted, and it is judged that the node leaves the cluster.

[0061] It should be noted that when the MinIO cluster starts, each node generates epoch information according to the current cluster information. The epoch information includes how many nodes there are in the current node and what the information (IP information, disk information, version number) of each node is, and uses this epoch as the cluster epoch and stores it in the etcd database.

[0062] Step 24: After receiving the notification, each node updates the version information of the cluster on its own node;

[0063] In this step, after receiving the notification, according to the change information, add the relevant information of the newly added node or delete the relevant information of the node that left, and update the cluster version number.

[0064] When a new node joins or a node leaves the MinIO cluster, MinIO senses the corresponding event through etcd. According to the information of the newly added or left node, add or delete the corresponding node information in the current cluster epoch, increase the epoch version number, and save the modified information as the new epoch information.

[0065] Step 25: Retrieve the data at the corresponding location under the version information of the previous cluster version, and rebalance the data to each node according to the new version information.

[0066] In this step, hash calculation is performed based on the version information of the previous cluster version to obtain the data in the cluster; hash calculation is performed based on the new version information, and the data in the cluster is redistributed to each node.

[0067] Step 26: After balancing is completed, delete the data at the corresponding position under the version information of the previous cluster version. For the convenience of data access and storage.

[0068] After the data is redistributed, delete the useless data at the corresponding position under the epoch information of the previous cluster version.

[0069] The embodiment of the present invention provides a method for processing cluster node changes based on cluster versions, and the specific implementation process is as follows:

[0070] (1) Use etcd as the cluster management module of the minio cluster. Utilize the revision mechanism of etcd to assign a globally unique id to each minio node. Use this id and the node ip as part of the key corresponding to this node in etcd, such as ip / xxxxx / id, and use the lease mechanism of etcd to set a lease period (a certain time, such as 15s, and this time is adjustable) for this key.

[0071] etcd is an open-source component that can provide functions such as cluster management, watch, and database.

[0072] (2) When the minio node starts up, it needs to apply for this id from the etcd cluster, and record this id as part of the node key in the etcd database. At the same time, minio needs to renew the lease for this key regularly to prevent this key from expiring, indicating the survival of this minio node; when this minio node fails, this key will expire.

[0073] (3) When the lease expires, this key will be automatically deleted; use the watch mechanism of etcd to monitor the deletion event of this key, and then the departure of the minio node can be sensed; similarly, when a new node joins the cluster, the new node will apply for a new id and set a new key, and renew the lease for the key. At this time, etcd can also sense the addition of a new node through the watch mechanism.

[0074] (4) When the minio node starts up, it will generate the epoch information (the same as the version information in this application) of the cluster on this node according to the information of the current cluster, and record this information in the etcd database.

[0075] (5) When MinIO perceives node events through etcd, it modifies the epoch information of the cluster according to the event type and uses the modified information as the latest epoch of the cluster. At the same time, each node in the cluster obtains each data block according to the epoch information of the previous version and redistributes the data blocks stored in the cluster according to the latest epoch. After the data redistribution is completed, the data at the old location under the epoch of the previous cluster version in the cluster is deleted. Thus, the dynamic change of nodes is realized. That is, MinIO perceives the changes of nodes in the cluster through etcd, modifies the current epoch information according to the change information, adds or deletes the information of newly added and left nodes, and saves it as the new cluster epoch information. Each node calculates the hash according to the epoch information of the previous cluster version, obtains the existing data in the cluster, calculates the hash according to the latest epoch information, and redistributes the data in the cluster to each node. After the data redistribution is completed, the data at the corresponding location under the epoch of the previous cluster version is deleted.

[0076] As Figure 2 shown, an embodiment of the present invention further provides a cluster node change processing device based on the cluster version, including a cluster management module, an output processing module, and a node information processing module;

[0077] The cluster management module is used to manage the nodes in the cluster and record the version information of the nodes in the current cluster; when it perceives that there are changes in the nodes in the cluster, it sends notifications to each node in the cluster; among them, the version information includes the id information, ip information, disk information, and cluster version number of the cluster nodes. Among them, each time the cluster nodes change, the cluster version number is incremented by 1.

[0078] The output processing module is used to perform hash calculation according to the current version information when data is sent to the cluster, calculate the position of the data object, and store the data normally;

[0079] The node information processing module is used to update the version information of the cluster on this node after receiving the notification; take out the data at the corresponding position under the version information of the previous cluster version, and rebalance the data to each node according to the new version information; after the balancing is completed, delete the data at the corresponding position under the version information of the previous cluster version.

[0080] It should be noted that in the embodiments of the present invention, etcd is used as the cluster management module of MinIO, which is responsible for perceiving the departure and joining of nodes and recording the epoch information in the current MinIO cluster. When data is sent to the MinIO cluster, the cluster performs hashing based on the number of cluster nodes in the current epoch information, calculates the location of the data object, and stores the data normally; when a node leaves or joins the cluster, etcd perceives this event and notifies the information of the leaving or joining node to each node in the cluster. Each node updates the epoch information of the cluster on its own node (adds or deletes the corresponding node information in the epoch) according to the added or deleted node information, recalculates the hash based on the epoch information of the previous cluster version and the epoch information of the current cluster version, retrieves the data according to the epoch information of the previous cluster version, and redistributes the data to each node according to the new epoch information. After the redistribution is completed, the data at the corresponding location under the epoch of the previous cluster version can be deleted. In this way, the function of dynamically adding and deleting MinIO nodes is realized. From the perspective of function, the product function and the competitiveness of the product are improved.

[0081] The embodiments of the present invention further provide a cluster node change processing device based on the cluster version, including a cluster management module, an output processing module, and a node information processing module;

[0082] The cluster management module is used to manage the nodes in the cluster and record the version information of the nodes in the current cluster; when it perceives a change in the nodes in the cluster, it sends a notification to each node in the cluster; among them, the version information includes the id information, ip information, disk information, and cluster version number of the cluster nodes. Each time a cluster node changes, the cluster version number is incremented by 1. Specifically, when each node joins the cluster and starts up, a globally unique id is assigned; a storage key value is generated according to this id and the node ip and stored in the database, and a lease of this key value is set; and the lease of this key value is extended regularly when the cluster starts up; record the version information of the nodes in the current cluster; specifically, when a node applies for an id, it is judged that a node joins the cluster; when the lease of the storage key value of the node expires, this storage key value will be automatically deleted, and it is judged that this node leaves the cluster.

[0083] In the embodiments of the present application, the etcd cluster management function is used as a cluster management module to manage the Minio cluster. Each Minio node will use the revision mechanism of etcd or a globally unique ID to form the key of the node with the ID and the node IP, such as ip / xxxxx / id, and store it in the etcd database and set a lease; the lease period of the lease is adjustable. When Minio starts, it will generate this key and periodically extend the lease period of this key, so that the lease does not expire, which means the node is alive; when the node fails, it cannot renew the lease for this key, and this key will be deleted by etcd.

[0084] An output processing module, configured to perform hash calculation according to the current version information when data is sent to the cluster, calculate the position of the data object, and store the data normally;

[0085] When the Minio cluster starts, each node will generate an epoch according to the current cluster information. The epoch includes how many nodes there are in the current node and what the information (IP information, disk information, version number) of each node is, and use this epoch as the cluster epoch and store it in the etcd database.

[0086] A node information processing module, configured to update the version information of the cluster on the local node after receiving a notification; take out the data at the corresponding position under the version information of the previous cluster version, and redistribute the data to each node according to the new version information; after the redistribution is completed, delete the data at the corresponding position under the version information of the previous cluster version; specifically configured to add the relevant information of the newly added node or delete the relevant information of the node that has left according to the change information after receiving the notification, and update the cluster version number; specifically also configured to perform hash calculation according to the version information of the previous cluster version to obtain the data in the cluster; perform hash calculation according to the new version information, and redistribute the data in the cluster to each node.

[0087] It should be noted that when a new node joins or a node leaves the Minio cluster, Minio senses the corresponding event through etcd, adds or deletes the corresponding node information to / from the epoch of the current cluster according to the information of the newly added or left node, increases the cluster version number, and saves the modified information as the new epoch information. Each node in the Minio cluster will calculate the hash according to the epoch of the previous cluster version to obtain the existing data in the cluster, and calculate the hash according to the latest epoch information, and redistribute the data to the new cluster. After the data is redistributed, delete the useless data at the corresponding position under the epoch information of the previous cluster version.

[0088] A computer device provided by an embodiment of the present invention may include: a processor, a communication interface, a memory, and a bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the bus. The bus can be used for information transmission between the electronic device and the sensor. The processor can call the logical instructions in the memory to execute the following method: Step 21: Manage the nodes in the cluster and record the version information of the nodes in the current cluster; Step 22: When data is sent to the cluster, perform hash calculation according to the current version information, calculate the position of the data object, and store the data normally; Step 23: When it is sensed that there is a change in the nodes in the cluster, send a notification to each node in the cluster; Step 24: After receiving the notification, each node updates the version information of the cluster on its own node; Step 25: Retrieve the data corresponding to the position under the version information of the previous cluster version, and rebalance the data to each node according to the new version information. Step 26: After the balancing is completed, delete the data corresponding to the position under the version information of the previous cluster version.

[0089] In some specific embodiments, the processor can call the logical instructions in the memory to execute the following method: When each node joins the cluster and starts up, it applies for a globally unique id; generates a storage key value according to the id and the node ip, stores it in the database, and sets the lease of the key value; and sets a timer to extend the lease of the key value when the cluster starts up.

[0090] In some specific embodiments, the processor can call the logical instructions in the memory to execute the following method: When a new node joins or a node leaves the MinIO cluster, MinIO senses the corresponding event through etcd, and according to the information of the newly added or left node, adds or deletes the corresponding node information to the epoch of the current cluster, increases the epoch version number, and saves the modified information as the new epoch information.

[0091] In addition, when the above-mentioned logical instructions in the memory are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0092] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, those of ordinary skill in the art can make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions should all be within the scope covered by the present invention. / Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, and they should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for processing changes in cluster nodes based on a cluster version, characterized in that, It includes the following steps: Manage the nodes in the cluster and record the version information of the nodes in the current cluster; When data is sent to the cluster, perform a hash calculation according to the current version information, calculate the location of the data object, and store the data normally; When it is sensed that there is a change in the nodes in the cluster, send a notification to each node in the cluster; After receiving the notification, each node updates the version information of the cluster on its own node; Take out the data corresponding to the location under the version information of the previous cluster version, and rebalance the data to each node according to the new version information; The version information includes the id information, ip information, disk information of the cluster nodes and the cluster version number. Among them, each time the cluster nodes change, the cluster version number is incremented by 1; After the step of taking out the data corresponding to the location under the version information of the previous cluster version and rebalancing the data to each node according to the new version information, it further includes: After the balancing is completed, delete the data corresponding to the location under the version information of the previous cluster version.

2. The method for processing changes in cluster nodes based on the cluster version according to claim 1, wherein The step of each node updating the version information of the cluster on its own node after receiving the notification includes: After receiving the notification, according to the change information, add the relevant information of the newly added nodes or delete the relevant information of the nodes that have left, and update the cluster version number.

3. The method for processing changes in cluster nodes based on the cluster version according to claim 1, characterized in that The step of taking out the data corresponding to the location under the version information of the previous cluster version and rebalancing the data to each node according to the new version information includes: Perform a hash calculation according to the version information of the previous cluster version to obtain the data in the cluster; Perform a hash calculation according to the new version information, and redistribute the data in the cluster to each node.

4. The method for processing changes of cluster nodes based on a cluster version according to claim 1, wherein The step of managing the nodes in the cluster and recording the version information of the nodes in the current cluster includes: When each node joins the cluster and starts up, apply for a globally unique id; Generate a storage key value according to the id and the node ip, store it in the database, and set the lease of the key value; and set a timer to extend the lease of the key value when the cluster starts up; Record the version information of the nodes in the current cluster.

5. The method for processing changes of cluster nodes based on the cluster version according to claim 4, characterized in that, In the step of sending a notification to each node in the cluster when it is sensed that there is a change in the nodes in the cluster, the step of sensing the change in the nodes in the cluster includes: When a node applies for an id, it is determined that a node has joined the cluster; When the lease of the storage key value of a node expires, the storage key value will be automatically deleted, and it is determined that the node has left the cluster.

6. A cluster node change processing device based on a cluster version, characterized in that, It includes a cluster management module, an output processing module, and a node information processing module; The cluster management module is used to manage the nodes in the cluster and record the version information of the nodes in the current cluster; when it is sensed that there is a change in the nodes in the cluster, send a notification to each node in the cluster; among them, the version information includes the id information, ip information, disk information of the cluster nodes and the cluster version number. Among them, each time the cluster nodes change, the cluster version number is incremented by 1; The output processing module is used to perform a hash calculation according to the current version information when data is sent to the cluster, calculate the location of the data object, and store the data normally; The node information processing module is used to update the version information of the cluster on this node after receiving a notification; retrieve the data at the corresponding position under the version information of the previous cluster version, and rebalance the data to each node according to the new version information. After the balancing is completed, delete the data at the corresponding position under the version information of the previous cluster version.

7. The cluster node change processing device based on the cluster version according to claim 6, wherein The cluster management module is specifically used to assign a globally unique id when each node joins the cluster and starts up; generate a storage key value based on the id and the node ip, store it in the database, and set the lease of the key value; and set a timer to extend the lease of the key value when the cluster starts up. Record the version information of the nodes in the current cluster.

8. A computer device, characterized in that, It includes a processor and a memory. The processor and the memory communicate with each other through a bus. The memory stores program instructions executable by the processor. The processor can execute the method for processing changes of cluster nodes based on cluster versions as described in any one of claims 1 to 5 by calling the program instructions.

Images