Distributed cluster management method and device, medium and electronic equipment

By storing tasks and execution status in the database of a distributed cluster, the master node regularly reads and distributes unfinished tasks, solving the problem of inefficient task processing caused by node exceptions, and achieving efficient management and task processing of distributed clusters.

CN118945178APending Publication Date: 2024-11-12TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310534400.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

After the task is allocated, if an exception occurs in a node, the task cannot be processed in time, resulting in inefficient task processing.

Method used

By storing the task and task execution status in the database of a distributed cluster, the master node regularly reads and obtains unfinished tasks and distributes them to other nodes for processing to ensure timely processing of tasks.

Benefits of technology

The task processing efficiency of distributed clusters is improved, and the effective management of distributed clusters is realized, ensuring that tasks can be transferred and processed in a timely manner when node abnormalities are not done.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945178A_ABST
    Figure CN118945178A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a distributed cluster management method and device, a medium and electronic equipment. When each distributed node in the distributed cluster receives a task sent by a task creation party, the task sent by the task creation party is stored in a database corresponding to the distributed cluster, and the execution state of each task is also stored in the database. The management method comprises the steps of reading execution states of all tasks stored in the database according to a set period, wherein the execution states stored in the database are updated according to task processing conditions of distributed nodes; obtaining unprocessed tasks in the database according to the execution state of each task; and distributing the obtained tasks which are not processed completely to distributed nodes in the distributed cluster for processing. According to the technical scheme, the task processing efficiency of the distributed cluster can be improved, and effective management of the distributed cluster is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer and communication technology, and in particular to a management method, device, medium and electronic device for a distributed cluster. Background Art

[0002] Distributed clusters combine multiple distributed nodes to jointly process tasks. Compared with a single node, distributed clusters can carry a higher volume of tasks. In related technologies, although it is proposed that tasks can be assigned to each node for processing through load balancing, after being assigned to a node, if a node fails to process the task due to an abnormality, the task will still not be processed in time, which will affect the efficiency of task processing. Summary of the invention

[0003] The embodiments of the present application provide a distributed cluster management method, device, medium and electronic device, which can improve the task processing efficiency of the distributed cluster and realize effective management of the distributed cluster.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] In the first aspect, an embodiment of the present application provides a method for managing a distributed cluster. When each distributed node in the distributed cluster receives a task sent by a task creator, the distributed node stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. The method for managing the distributed cluster includes: reading the execution status of each task stored in the database according to a set period, and the execution status stored in the database is updated according to the processing status of the task by the distributed node; obtaining the unfinished tasks in the database according to the execution status of each task; and distributing the obtained unfinished tasks to the distributed nodes in the distributed cluster for processing.

[0006] In the second aspect, an embodiment of the present application provides a method for managing a distributed cluster. When each distributed node in the distributed cluster receives a task sent by a task creator, the distributed node stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. The method for managing the distributed cluster includes: receiving task distribution information sent by a master node in the distributed cluster, the task distribution information including unfinished tasks obtained by the master node from the database according to a set period, and slave node information to which the unfinished tasks are to be distributed; and distributing the unfinished tasks to corresponding slave nodes for processing according to the slave node information to which the unfinished tasks are to be distributed.

[0007] In the third aspect, an embodiment of the present application provides a method for managing a distributed cluster. When each distributed node in the distributed cluster receives a task sent by a task creator, the node stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. The method for managing the distributed cluster includes: receiving a target task distributed by a master node in the distributed cluster, where the target task is an unfinished task obtained by the master node from the database; executing the target task and setting the execution status of the target task stored in the database to a processing status; and updating the execution status of the target task stored in the database according to the execution status of the target task.

[0008] In a fourth aspect, an embodiment of the present application provides a management device for a distributed cluster. When each distributed node in the distributed cluster receives a task sent by a task creator, the distributed node stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. The management device for the distributed cluster includes: a reading unit, configured to read the execution status of each task stored in the database according to a set period, and the execution status stored in the database is updated according to the processing status of the task by the distributed node; an acquisition unit, configured to acquire unfinished tasks in the database according to the execution status of each task; and a distribution unit, configured to distribute the acquired unfinished tasks to the distributed nodes in the distributed cluster for processing.

[0009] In the fifth aspect, an embodiment of the present application provides a management device for a distributed cluster. When each distributed node in the distributed cluster receives a task sent by a task creator, the distributed node stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. The management device of the distributed cluster includes: a receiving unit, configured to receive task distribution information sent by a master node in the distributed cluster, the task distribution information including unfinished tasks obtained by the master node from the database according to a set period, and slave node information to which the unfinished tasks are to be distributed; a sending unit, configured to distribute the unfinished tasks to corresponding slave nodes for processing according to the slave node information to which the unfinished tasks are to be distributed.

[0010] In the sixth aspect, an embodiment of the present application provides a management device for a distributed cluster. When each distributed node in the distributed cluster receives a task sent by a task creator, the distributed node stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. The management device for the distributed cluster includes: a receiving unit, configured to receive a target task distributed by a master node in the distributed cluster, wherein the target task is an unfinished task obtained by the master node from the database; an execution unit, configured to execute the target task and set the execution status of the target task stored in the database to a processing status; and an updating unit, configured to update the execution status of the target task stored in the database according to the execution status of the target task.

[0011] In a seventh aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processor, the distributed cluster management method as described in the above embodiment is implemented.

[0012] In an eighth aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device for storing one or more computer programs, wherein when the one or more computer programs are executed by the one or more processors, the electronic device implements the distributed cluster management method as described in the above embodiments.

[0013] In a ninth aspect, an embodiment of the present application provides a computer program product, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of an electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device executes the distributed cluster management method provided in the above-mentioned various optional embodiments.

[0014] In the technical solutions provided in some embodiments of the present application, when each distributed node in the distributed cluster receives a task sent by the task creator, it stores the task sent by the task creator in the database corresponding to the distributed cluster, and the database also stores the execution status of each task, so that the master node in the distributed cluster can regularly read the execution status of each task stored in the database, and then obtain the unfinished tasks in the database, and then distribute the unfinished tasks to the distributed nodes in the distributed cluster for processing. It can be seen that the technical solution of the embodiment of the present application enables the distributed nodes in the distributed cluster to record the execution status information of the task through the database, so that the master node can distribute the unfinished tasks to other nodes for processing in a timely manner, thereby improving the task processing efficiency of the distributed cluster and realizing the effective management of the distributed cluster.

[0015] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied;

[0017] Figure 2 A flow chart of a distributed cluster management method according to an embodiment of the present application is shown;

[0018] Figure 3 A flow chart of a distributed cluster management method according to an embodiment of the present application is shown;

[0019] Figure 4 A flow chart of a distributed cluster management method according to an embodiment of the present application is shown;

[0020] Figure 5 A schematic diagram of the deployment of a cluster management framework according to an embodiment of the present application is shown;

[0021] Figure 6 A flowchart of processing for bringing a new node online in a distributed cluster according to an embodiment of the present application is shown;

[0022] Figure 7 A processing flow chart of master node switching in a distributed cluster according to an embodiment of the present application is shown;

[0023] Figure 8 A schematic diagram showing a system architecture according to an embodiment of the present application is shown;

[0024] Fig. 9 A flow chart of a distributed cluster processing method according to an embodiment of the present application is shown;

[0025] Fig.10 A block diagram of a distributed cluster management device according to an embodiment of the present application is shown;

[0026] Fig.11 A block diagram of a distributed cluster management device according to an embodiment of the present application is shown;

[0027] Fig.12 A block diagram of a distributed cluster management device according to an embodiment of the present application is shown;

[0028] Fig.13 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] The exemplary embodiments are now described in a more comprehensive manner with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be understood as being limited to these examples; on the contrary, the purpose of providing these embodiments is to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0030] In addition, the features, structures or characteristics described in the present application may be combined in one or more embodiments in any suitable manner. In the following description, there are many specific details so that the embodiments of the present application can be fully understood. However, those skilled in the art will appreciate that when implementing the technical scheme of the present application, all the detailed features in the embodiments may not be needed, one or more specific details may be omitted, or other methods, elements, devices, steps, etc. may be adopted.

[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0032] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0033] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0034] It can be understood that in the specific implementation of the present application, related data such as the node status of distributed nodes and the execution status of tasks, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0035] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied is shown.

[0036] like Figure 1 As shown, the distributed cluster 102 includes multiple distributed nodes, and the multiple distributed nodes may include a master node, such as distributed node 1021 as the master node, and other distributed nodes as slave nodes. The distributed cluster 102 corresponds to a database 103, and the database 103 is used to store the tasks received by each distributed node, that is, when each distributed node receives a task created by a task creator (such as terminal device 101), it can store the received task in the database 103. At the same time, the database 103 also stores the execution status of each task, such as the executing status, the unprocessed status, the successful execution status, the failed execution status, etc.

[0037] In some optional embodiments, when a distributed node (which can be either a master node or a slave node) in the distributed cluster 102 receives a pending task sent by a task creator, the received pending task can be stored in the database 103, and then it can be determined whether the received pending task can be executed according to the current state of the distributed node. If it is determined that the received pending task can be executed according to the current state of the distributed node, the current distributed node can execute the received pending task and set the execution state of the pending task stored in the database 103 to the processing state, and then the execution state of the pending task stored in the database 103 can be updated according to the execution status of the pending task. For example, if the current distributed node completes the execution of the received pending task, the execution state of the pending task stored in the database 103 is updated to the state of successful execution or failed execution.

[0038] Optionally, when a distributed node receives a pending task sent by a task creator, if it determines that the received pending task cannot be executed based on the current state of the distributed node, for example, the task load of the current distributed node is large, then when the received pending task is stored in database 103, the execution state of the pending task stored in database 103 can be set to an unprocessed state.

[0039] In some optional embodiments, the master node in the distributed cluster 102 can read the execution status of each task stored in the database 103 according to a set period, and then obtain the unfinished tasks in the database 103 according to the execution status of each task, such as unprocessed tasks or tasks whose processing time exceeds a set time threshold. The master node then distributes the obtained unfinished tasks to the distributed nodes in the distributed cluster for processing, thereby improving the task processing efficiency of the distributed cluster and realizing effective management of the distributed cluster.

[0040] Understandably, Figure 1 The terminal device 101 shown in the figure is connected to the distributed nodes in the distributed cluster 102 through network communication, and the network may include various connection types, such as wireless communication links, wired and optical fiber cables, etc. The terminal device 101 may be a smart phone, a tablet, a laptop, a computer, an intelligent voice interaction device, a smart home appliance, an intelligent wearable device, an aircraft, etc. The distributed node may be a server that provides various services, which may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), big data and artificial intelligence platforms, etc. In addition, according to implementation needs, the distributed cluster 102 may include any number of distributed nodes.

[0041] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0042] Figure 2 A flow chart of a distributed cluster management method according to an embodiment of the present application is shown. When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task.

[0043] Optionally, Figure 2The management method of the distributed cluster shown in can be executed by a designated node in the distributed cluster, and the designated node can be a master node in the distributed cluster. Figure 2 As shown, the distributed cluster management method at least includes S210 to S230, which are described in detail as follows:

[0044] In S210, the execution status of each task stored in the database is read according to a set period. The execution status stored in the database is updated according to the processing status of the task by the distributed nodes.

[0045] In some optional embodiments, when a distributed node (including a master node and a slave node) receives a pending task sent by a task creator, the distributed node (including a master node and a slave node) can store the received pending task in a database, and then determine whether the received pending task can be executed based on the current state of the distributed node. If it is determined that the received pending task can be executed based on the current state of the distributed node (for example, the load of the current distributed node is low), the current distributed node can execute the received pending task and set the execution state of the pending task stored in the database to a processing state. After that, the execution state of the pending task stored in database 103 can be updated based on the execution status of the pending task.

[0046] In some optional embodiments, if it is determined that the pending task cannot be executed based on the current state of the distributed node, the execution state of the pending task stored in the database can be set to an unfinished processing state, such as an unprocessed state.

[0047] It should be noted that when a distributed node receives a pending task sent by a task creator, it can directly store it in the database, and then determine whether the received pending task can be executed. After that, the execution status of the pending task stored in the database is adjusted according to whether the pending task can be executed, that is, when it can be executed, it is adjusted to the executing state; when it cannot be executed, it is adjusted to the unfinished processing state. Alternatively, when a distributed node receives a pending task sent by a task creator, it can first determine whether the received pending task can be executed. If it can be executed, then while storing the pending task in the database, the execution status of the pending task stored in the database is set to the executing state; if it cannot be executed, then while storing the pending task in the database, the execution status of the pending task stored in the database is set to the unfinished processing state.

[0048] In S220 , unfinished tasks in the database are acquired according to the execution status of each task.

[0049] In some optional embodiments, tasks that have not been processed can be regarded as unfinished tasks according to the execution status of each task. Alternatively, tasks whose execution time exceeds a set time threshold can be regarded as unfinished tasks according to the execution status of each task. Alternatively, tasks that have not been processed and tasks whose execution time exceeds a set time threshold can be regarded as unfinished tasks. Of course, tasks that have failed to execute can also be regarded as unfinished tasks.

[0050] Optionally, when creating a task, a task start time field can be set in the task, so that for a task whose execution status is in progress, it can be determined whether the execution time of the task exceeds the set time threshold based on the current time and the start time indicated by the task start time field. Alternatively, the distributed node can also record the execution time of the task by itself, and then when the execution time of the task exceeds the set time threshold, the task is marked in the database as having an execution time exceeding the set time threshold.

[0051] In S230 , the acquired unfinished tasks are distributed to distributed nodes in the distributed cluster for processing.

[0052] In some optional embodiments, the process of distributing the acquired unfinished tasks to the distributed nodes in the distributed cluster for processing may be: obtaining the node status of the distributed nodes in the distributed cluster, such as the node status may include the node load status, and then distributing the unfinished tasks to a set number of distributed nodes for processing in the order of the node load from low to high. Optionally, the set number may be 1, for example, the unfinished tasks may be preferentially distributed to the distributed nodes with the lowest load; the set number may also be multiple, that is, multiple distributed nodes with lower loads are selected to execute the unfinished tasks.

[0053] The technical solution of this embodiment enables the master node to distribute unfinished tasks to other nodes for processing in a timely manner, thereby improving the task processing efficiency of the distributed cluster and achieving effective management of the distributed cluster.

[0054] In some optional embodiments, distributed nodes in a distributed cluster may communicate through a cluster management framework, and the distributed nodes may call different callback functions to perform data sending and receiving operations.

[0055] For example, the master node in the distributed cluster can call the first callback function to transmit the message that needs to be sent to the slave node in the distributed cluster to the slave node. The first callback function is used by the master node in the distributed cluster to send messages to the slave node. For example, the first callback function can be SendToFollower.

[0056] Optionally, the master node in the distributed cluster may receive a message from a slave node sent by the cluster management framework calling a second callback function, where the second callback function is used to receive a message sent from the slave node to the master node. For example, the second callback function may be ReceiveFromFollower.

[0057] Optionally, the master node in the distributed cluster may receive slave node status information in the distributed cluster sent by the cluster management framework calling a third callback function, where the third callback function is used to send slave node status information to the master node, such as FollowerStatusChange. The slave node status information may be, for example, the heartbeat status and load status of the slave node.

[0058] Figure 2 The technical solution of the embodiment of the present application is explained from the perspective of the master node in the distributed cluster. Figure 3 The technical solution of the embodiment of the present application is explained from the perspective of a slave node in a distributed cluster.

[0059] Figure 3 A flow chart of a distributed cluster management method according to an embodiment of the present application is shown. When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task.

[0060] Optionally, Figure 3 The distributed cluster management method shown in can be executed by a slave node in the distributed cluster. Figure 3 As shown, the distributed cluster management method at least includes S310 to S330, which are described in detail as follows:

[0061] In S310 , a target task distributed by a master node in a distributed cluster is received, where the target task is an unfinished task obtained by the master node from a database.

[0062] In some optional embodiments, the master node may obtain unfinished tasks from the database according to a set period, and then distribute the unfinished tasks to a set number of distributed nodes for processing in the order of low to high node load. Optionally, the unfinished tasks may include tasks that have not been processed and / or tasks whose execution time exceeds a set time threshold.

[0063] In S320 , the target task is executed, and the execution status of the target task stored in the database is set to a processing status.

[0064] In S330 , the execution status of the target task stored in the database is updated according to the execution status of the target task.

[0065] In some optional embodiments, if the execution of the target task is completed, the execution status of the target task stored in the database may be updated to a state of execution completion.

[0066] In some optional embodiments, distributed nodes in a distributed cluster may communicate through a cluster management framework, and the distributed nodes may call different callback functions to perform data sending and receiving operations.

[0067] For example, the slave node in the distributed cluster can call the seventh callback function to transmit to the master node the message that needs to be sent to the master node or the message that needs to be forwarded by the master node to other slave nodes. The seventh callback function is used by the slave node in the distributed cluster to send messages to the master node. For example, the seventh callback function can be SendToLeader.

[0068] Optionally, the slave node in the distributed cluster may receive a message from the master node sent by the cluster management framework calling an eighth callback function, where the eighth callback function is used to receive a message sent from the master node to the slave node. For example, the eighth callback function may be ReceiveFromLeader.

[0069] The above is an explanation of the technical solutions of the embodiments of the present application from the perspectives of the master node and the slave node in the distributed cluster. Figure 4 The technical solution of the embodiment of the present application is explained from the perspective of a cluster management framework.

[0070] Figure 4 A flow chart of a distributed cluster management method according to an embodiment of the present application is shown. When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task.

[0071] Optionally, Figure 4 The distributed cluster management method shown in can be executed by the cluster management framework. Figure 4 As shown, the distributed cluster management method at least includes S410 to S420, which are described in detail as follows:

[0072] In S410, task distribution information sent by a master node in a distributed cluster is received, the task distribution information including unfinished tasks obtained by the master node from a database according to a set period, and information of slave nodes to which the unfinished tasks are to be distributed.

[0073] In some optional embodiments, unfinished tasks may include unprocessed tasks and / or tasks whose execution time exceeds a set time threshold. Optionally, the slave node information to which the unfinished tasks are to be distributed may include a set number of distributed nodes selected by the master node in the order of low to high node load, and may also include task information (such as task identification, task quantity, etc.) distributed to each distributed node selected by the master node.

[0074] Optionally, the slave node information to which the unfinished tasks are to be distributed may be: tasks with task identifiers 01 and 02 are distributed to node 1, tasks with task identifier 03 is distributed to node 2, tasks with task identifier 04 is distributed to node 3, and so on.

[0075] In S420, the unfinished tasks are distributed to corresponding slave nodes for processing according to the information of the slave nodes to which the unfinished tasks are to be distributed.

[0076] It should be noted that since tasks are stored in the database, the cluster management framework only needs to distribute task identification information when distributing unfinished tasks to corresponding slave nodes for processing, so that the slave nodes can read the corresponding tasks from the database according to the task identification information for processing.

[0077] In some optional embodiments, each distributed node can register with the cluster management framework when it goes online. In this case, the cluster management framework can receive registration requests sent by distributed nodes in the distributed cluster, and then select a master node based on the information of the registered distributed nodes, and send a first notification message to the distributed nodes in the distributed cluster, where the first notification message is used to indicate the information of the selected master node.

[0078] For example, after startup, the distributed node can register a callback function through the registration request, which includes a fourth callback function (such as becomeLeader) for notifying the distributed node that it is selected as the master node, a fifth callback function (such as LostLeader) for notifying the master node to change to a slave node, and a sixth callback function (such as leaderChange) for notifying the master node in the distributed cluster of changes.

[0079] Then the process of the cluster management framework sending the first notification message to the distributed nodes in the distributed cluster may be: calling the fourth callback function to send the notification message to the selected master node, and calling the sixth callback function to send the notification message to other nodes except the master node.

[0080] In some optional embodiments, if the cluster management framework detects that an abnormality occurs in the master node, the cluster management framework may reselect the master node from the registered distributed nodes, and then send a second notification message to the distributed nodes in the distributed cluster according to the reselected master node, where the second notification message is used to indicate the reselected master node. For example, the cluster management framework may call the fifth callback function to send a notification message to the original master node, and call the fourth callback function to send a notification message to the reselected master node.

[0081] The above respectively describes the technical solutions of the embodiments of the present application from the perspectives of the master node, the slave node and the cluster management framework in the distributed cluster. The following describes the technical solutions of the embodiments of the present application from the perspective of the interaction between various entities.

[0082] In one embodiment of the present application, Figure 5 As shown, node 1 and node 2 can be two nodes of distributed deployment, and a cluster management framework can be deployed in the nodes. The cluster management framework can be implanted into the application program through SDK (Software Development Kit), and can actually run in the same process of the node with the application program. For the application program, it can communicate with other nodes in the cluster through the API (Application Programming Interface) provided by the cluster management framework, or receive event notifications in the cluster. And the cluster management framework itself can be responsible for the interaction between clusters, including selecting the master node, master-slave switching and message communication.

[0083] In some optional embodiments, the cluster management framework provides an API for cluster event subscription. When an application node goes online, it can actively register with the cluster management framework. When the node identity in the cluster changes, a notification will be sent to the application. The API type and function used for notification can be shown in Table 1:

[0084] API Function becomeLeader The current node is selected as the master node of the cluster LostLeader The current node was previously the master node of the cluster and now becomes a slave node leaderChange The master node in the cluster has switched (the slave node receives the master node change notification)

[0085] Table 1

[0086] When distributed nodes in a cluster are in different roles, applications can call different APIs to perform operations. For example Figure 5 As shown in Table 2, when the node is a master node, the application can call the Leader API and Leader Callback; when the node is a slave node, the application can call the Follower API and Follower Callback. The functions of these APIs can be shown in Table 2:

[0087]

[0088] Table 2

[0089] In one embodiment of the present application, the processing flow of a new node going online in a distributed cluster is as follows: Figure 6 As shown, including:

[0090] S601, node 1 comes online and registers a callback function with the framework.

[0091] For example, node 1 is the first node to go online in the distributed cluster. It starts the cluster management framework and registers callback functions with the framework, including the becomeLeader callback, lostLeader callback, and leaderChange callback in Table 1 above.

[0092] S602, node 1 is selected as the master node.

[0093] Since node 1 is the first node to go online in the distributed cluster, the cluster management framework can select node 1 as the master node.

[0094] S603, the cluster management framework notifies the application that it has become the master node through the callback function becomeLeader() registered by the application.

[0095] S604, node 1 records its own node status.

[0096] Optionally, after receiving the callback from the cluster management framework, node 1 can record its own node status and start to perform corresponding operations as the master node, such as obtaining the load status of the slave nodes, regularly reading unfinished tasks from the database for distribution, etc.

[0097] S605, node 2 comes online and registers a callback function with the framework.

[0098] After node 1 comes online, node 2 comes online and registers callback functions with the framework, including the becomeLeader callback, lostLeader callback, and leaderChange callback in Table 1 above.

[0099] S606, node 2 is selected as a slave node.

[0100] Since there is already a healthy master node (node ​​1) in the cluster, the newly launched node 2 is selected as the slave node.

[0101] S607, the cluster management framework notifies the application program that the current master node is node 1 through the callback function leaderChange() registered by the application program.

[0102] S608, node 2 records its own node status.

[0103] Optionally, after receiving the callback from the cluster management framework, node 2 may record its own node status and start to perform corresponding operations as a slave node, such as executing tasks distributed by the master node.

[0104] In one embodiment of the present application, the processing flow of master node switching in a distributed cluster is as follows: Figure 7 As shown, including:

[0105] S701, when node 1 has been selected as a master node and node 2 as a slave node in a distributed cluster, node 1 has an abnormal operation. For example, if it is detected that the network status of node 1 is poor or down, it is determined that node 1 has an abnormal operation.

[0106] S702: The cluster management framework detects that node 1 is operating abnormally, and then reselects node 2 as the master node.

[0107] S703, the cluster management framework notifies node 1 to become a slave node through the callback function lostLeader() registered by the application.

[0108] It should be noted that if node 1 is offline, then lostLeader() notification cannot be sent to it. In this case, S703 is an optional step.

[0109] S704, node 1 records its own node status.

[0110] Optionally, after receiving the lostLeader() notification sent by the cluster management framework, node 1 may record its own node status (ie, become a slave node) and start to perform corresponding operations as a slave node.

[0111] S705, the cluster management framework notifies node 2 to become the master node through the callback function becomeLeader() registered by the application.

[0112] S706, node 2 records its own node status.

[0113] Optionally, after receiving the becomeLeader() notification sent by the cluster management framework, node 2 may record its own node status (ie, become the master node) and start to perform corresponding operations as the master node.

[0114] Based on the master node selection and master-slave switching scheme described in the foregoing embodiments, in a specific application scenario of the present application, Figure 8 As shown, the master node and the slave node communicate through the cluster management framework, and the master node and the slave node share the same database. The master node has a timer, an executor for executing tasks, and a distributor for task distribution, and the slave node has an executor for executing tasks.

[0115] based on Figure 8 In the system framework shown, when any node in the distributed cluster receives a new task, it saves the task to the database, then starts executing the task at the node and updates the task status to "in execution". If the load of the node is too high and the new task can no longer be executed, the task is only saved without being executed, and the task status is set to "initialized" (i.e., not processed). The master node in the distributed cluster can start a timer, which is responsible for collecting the health status of the slave nodes and regularly reading the unfinished tasks in the database, and then distributes the tasks to other slave nodes for execution.

[0116] Specifically, Fig. 9 As shown, a distributed cluster processing method according to an embodiment of the present application includes:

[0117] S901, after the master node and the slave node have been selected in the distributed cluster, the master node starts a timer to periodically detect tasks in the database.

[0118] S902: The client creates a task in the slave node.

[0119] S903, the slave node inserts the task into the database, and if the current slave node can execute the task, the task status is set to being executed.

[0120] S904: If the current slave node is capable of executing the task, the current slave node starts executing the task.

[0121] S905, after the current slave node completes the task execution, it updates the task status in the database.

[0122] S906: The client creates a task in the slave node.

[0123] S907, the slave node inserts the task into the database. If the current slave node cannot execute the task, the task status is set to initialization, that is, a state in which no processing is performed.

[0124] S908: After the master node timer expires, it is found that there are unexecuted tasks in the database.

[0125] S909, the master node takes out the task and sends it to other slave nodes for execution through the cluster management framework. Optionally, if the master node is capable of processing the task, the master node can also allocate the task to itself for processing after taking out the task.

[0126] It can be seen that in an embodiment of the present application, when the nodes in the distributed cluster are normally online and the master node and the slave node are selected through the cluster management framework, the master node starts the timer and starts to detect the tasks in the database. If it is found that there are tasks with an initialization status (i.e., not processed), or tasks with an execution status but a long time away from the start time (i.e., the execution time has timed out), they will be taken out and distributed through the cluster management framework to ensure that the tasks can be processed in a timely manner.

[0127] Based on this, in an application scenario of the present application, after the client creates a task by calling the API of the slave node, the task is sent to the slave node, and the slave node inserts the task into the database. If the node is currently able to execute the task, the task status is inserted into the database as being executed; and the slave node starts to execute the task, and after the task is executed, the task status in the database is updated to be executed successfully or failed.

[0128] In an application scenario of the present application, the client creates a task by calling the API of the slave node, and the task is sent to the slave node. However, at this time, the running load of the slave node is too high (CPU or memory exceeds the configured threshold), and it is judged that the task cannot be executed. Then, when the slave node inserts the task into the database, the status of the task is initialized (i.e., not processed). In this case, the slave node does not execute the task and returns directly.

[0129] When the master node's timer period arrives, if it finds a task in the initialization state, it will take out the task and send it to other slave nodes for processing through the cluster management framework. When the master node assigns tasks, it will combine the load of all slave nodes in the current cluster and give priority to sending to the node with the lowest current load. Optionally, if all nodes in the cluster have reached the load threshold, a system alarm will be generated.

[0130] In an application scenario of the present application, when creating an executable task, the slave node can fill in the task start time field. If the node unexpectedly crashes during the task execution, the task execution will be interrupted. Therefore, if the maximum task execution time is configured when the system starts, such as 5 minutes, then if the master node timer cycle arrives and the database is queried to find that there is an executing task, and the execution time is determined to have exceeded 5 minutes based on the task start time field, then the task execution can be considered abnormal. Then, an alarm message can be sent, and then the task can be modified to the initialized state, and then it can be redistributed to a new node for execution.

[0131] The technical solution of the embodiment of the present application can realize the master-slave management, detection, inter-cluster communication and other capabilities of distributed nodes through the cluster management framework. The cluster management framework is very lightweight and can be embedded in the application through SDK for deployment and operation without consuming process resources separately. At the same time, based on the framework, a distributed task management service is provided, which can automatically allocate appropriate instances to perform tasks according to the load conditions of each node in the cluster. It also provides the capabilities of task redistribution when nodes are abnormal and redistribution when the node load is too high, thereby improving the task processing efficiency of the distributed cluster and realizing effective management of the distributed cluster.

[0132] The following describes an apparatus embodiment of the present application, which can be used to execute the distributed cluster management method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the distributed cluster management method of the present application.

[0133] Fig.10 A block diagram of a distributed cluster management device according to an embodiment of the present application is shown. When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. Optionally, Fig.10 The management device of a distributed cluster shown in can be applied to a designated node in a distributed cluster, and the designated node can be a master node in the distributed cluster.

[0134] Reference Fig.10 As shown, a distributed cluster management device 1000 according to an embodiment of the present application includes: a reading unit 1002 , an acquiring unit 1004 and a distributing unit 1006 .

[0135] Among them, the reading unit 1002 is configured to read the execution status of each task stored in the database according to a set period, and the execution status stored in the database is updated according to the processing status of the tasks by the distributed nodes; the acquisition unit 1004 is configured to acquire the unfinished tasks in the database according to the execution status of each task; the distribution unit 1006 is configured to distribute the acquired unfinished tasks to the distributed nodes in the distributed cluster for processing.

[0136] In some embodiments of the present application, based on the aforementioned scheme, the acquisition unit 1004 is configured to: acquire unprocessed tasks and target tasks whose execution duration exceeds a set duration threshold according to the execution status of the various tasks; and use the acquired unprocessed tasks and the target tasks as the unfinished tasks.

[0137] In some embodiments of the present application, based on the aforementioned scheme, the distribution unit 1006 is configured to: obtain the node status of the distributed nodes in the distributed cluster, the node status including the node load status; and distribute the unfinished processed tasks to a set number of distributed nodes for processing in order of node load from low to high.

[0138] In some embodiments of the present application, based on the above solution, the distributed cluster management device 1000 further includes: a processing unit, wherein the processing unit is configured to perform at least one of the following steps:

[0139] Calling a first callback function to transmit a message to be sent to a slave node in the distributed cluster to the slave node, wherein the first callback function is used for the master node in the distributed cluster to send a message to the slave node;

[0140] Receiving a message from a slave node sent by a cluster management framework calling a second callback function, wherein the second callback function is used to receive a message sent from the slave node to the master node;

[0141] The slave node status information in the distributed cluster is received and sent by the cluster management framework calling a third callback function, where the third callback function is used to send the slave node status information to the master node.

[0142] In some embodiments of the present application, based on the above solution, the distributed cluster management device 1000 further includes:

[0143] A storage unit, configured to store the to-be-processed task in the database upon receiving the to-be-processed task sent by the task creator;

[0144] an execution unit, configured to execute the task to be processed if it is determined that the task to be processed can be executed according to the current state of the distributed node, and set the execution state of the task to be processed stored in the database to a state of being processed;

[0145] An updating unit is configured to update the execution status of the task to be processed stored in the database according to the execution status of the task to be processed.

[0146] In some embodiments of the present application, based on the aforementioned scheme, the execution unit is further configured to: if it is determined that the task to be processed cannot be executed according to the state of the current distributed node, the execution state of the task to be processed stored in the database is set to an unfinished processing state.

[0147] Fig.11A block diagram of a distributed cluster management device according to an embodiment of the present application is shown. When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. Optionally, Fig.11 The management device for a distributed cluster shown in can be applied to slave nodes in a distributed cluster.

[0148] Reference Fig.11 As shown, a distributed cluster management device 1100 according to an embodiment of the present application includes: a receiving unit 1102 , an executing unit 1104 and an updating unit 1106 .

[0149] Among them, the receiving unit 1102 is configured to receive the target task distributed by the master node in the distributed cluster, and the target task is an unfinished task obtained by the master node from the database; the execution unit 1104 is configured to execute the target task and set the execution status of the target task stored in the database to a processing status; the updating unit 1106 is configured to update the execution status of the target task stored in the database according to the execution status of the target task.

[0150] In some embodiments of the present application, based on the above solution, the distributed cluster management device 1100 further includes: a processing unit, wherein the processing unit is configured to perform at least one of the following steps:

[0151] Calling a seventh callback function to transmit a message that needs to be sent to the master node or a message that needs to be forwarded by the master node to other slave nodes to the master node, wherein the seventh callback function is used for sending a message from a slave node in the distributed cluster to the master node;

[0152] The cluster management framework calls an eighth callback function to send a message from the master node, where the eighth callback function is used to receive a message sent by the master node to the slave node.

[0153] Fig.12 A block diagram of a distributed cluster management device according to an embodiment of the present application is shown. When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster. The database also stores the execution status of each task. Optionally, Fig.12 The distributed cluster management device shown in can be applied to a cluster management framework.

[0154] Reference Fig.12As shown, a distributed cluster management device 1200 according to an embodiment of the present application includes: a receiving unit 1202 and a sending unit 1204 .

[0155] Among them, the receiving unit 1202 is configured to receive task distribution information sent by the master node in the distributed cluster, and the task distribution information includes the unfinished processed tasks obtained by the master node from the database according to the set period, and the slave node information to which the unfinished processed tasks are to be distributed; the sending unit 1204 is configured to distribute the unfinished processed tasks to the corresponding slave nodes for processing according to the slave node information to which the unfinished processed tasks are to be distributed.

[0156] In some embodiments of the present application, based on the aforementioned scheme, the receiving unit 1202 is also configured to: before receiving the task distribution information sent by the master node in the distributed cluster, receive a registration request sent by the distributed node in the distributed cluster; the sending unit 1204 is also configured to: select a master node based on the information of the registered distributed nodes, and send a first notification message to the distributed nodes in the distributed cluster, wherein the first notification message is used to indicate the information of the selected master node.

[0157] In some embodiments of the present application, based on the aforementioned scheme, the management device 1200 of the distributed cluster also includes: a selection unit, configured to reselect the master node from the registered distributed nodes if an abnormality is detected in the master node; the sending unit is also configured to: send a second notification message to the distributed nodes in the distributed cluster according to the reselected master node, and the second notification message is used to indicate the reselected master node.

[0158] In some embodiments of the present application, based on the aforementioned scheme, the receiving unit 1202 is also configured to: receive a callback function registered after the distributed node is started, the callback function including a fourth callback function for notifying the distributed node that it is selected as a master node, a fifth callback function for notifying the master node that it is changed to a slave node, and a sixth callback function for notifying the master node in the distributed cluster of changes; the process in which the sending unit 1204 sends a first notification message to the distributed nodes in the distributed cluster includes: calling the fourth callback function to send a notification message to the selected master node, and calling the sixth callback function to send a notification message to other nodes other than the master node; the process in which the sending unit 1204 sends a second notification message to the distributed nodes in the distributed cluster according to the reselected master node includes: calling the fifth callback function to send a notification message to the original master node, and calling the fourth callback function to send a notification message to the reselected master node.

[0159] Fig.13 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0160] It should be noted that Fig.13 The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0161] like Fig.13 As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1302 or the program loaded from the storage part 1308 to the random access memory (RAM) 1303, such as executing the method described in the above embodiment. In RAM 1303, various programs and data required for system operation are also stored. CPU 1301, ROM 1302 and RAM 1303 are connected to each other through bus 1304. Input / output (I / O) interface 1305 is also connected to bus 1304.

[0162] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0163] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1309, and / or installed from a removable medium 1311. When the computer program is executed by a central processing unit (CPU) 1301, various functions defined in the system of the present application are executed.

[0164] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0165] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and a computer program.

[0166] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not constitute limitations on the units themselves in some cases.

[0167] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more computer programs, and when the above one or more computer programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0168] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0169] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0170] Those skilled in the art will readily come up with other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0171] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A distributed cluster management method, characterized in that: When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster, and the database also stores the execution status of each task. The management method of the distributed cluster includes: Reading the execution status of each task stored in the database according to a set period, wherein the execution status stored in the database is updated according to the processing status of the task by the distributed nodes; Acquire unfinished tasks in the database according to the execution status of each task; The acquired unfinished tasks are distributed to the distributed nodes in the distributed cluster for processing.

2. The distributed cluster management method according to claim 1, characterized in that: Acquiring unfinished tasks in the database according to the execution status of each task, including: According to the execution status of each task, unprocessed tasks and target tasks whose execution time exceeds a set time threshold are obtained; The acquired unprocessed tasks and the target tasks are used as the unfinished tasks.

3. The distributed cluster management method according to claim 1, characterized in that: Distributing the acquired unfinished tasks to the distributed nodes in the distributed cluster for processing, including: Obtaining a node status of a distributed node in the distributed cluster, wherein the node status includes a node load condition; In order of node load from low to high, the unfinished processed tasks are distributed to a set number of distributed nodes for processing.

4. The distributed cluster management method according to claim 1, characterized in that: The management method further comprises at least one of the following steps: Calling a first callback function to transmit a message to be sent to a slave node in the distributed cluster to the slave node, wherein the first callback function is used for the master node in the distributed cluster to send a message to the slave node; Receiving a message from a slave node sent by a cluster management framework calling a second callback function, wherein the second callback function is used to receive a message sent from the slave node to the master node; The slave node status information in the distributed cluster is received and sent by the cluster management framework calling a third callback function, where the third callback function is used to send the slave node status information to the master node.

5. The distributed cluster management method according to any one of claims 1 to 4, characterized in that: The distributed cluster management method also includes: When receiving the pending task sent by the task creator, storing the pending task in the database; If it is determined that the task to be processed can be executed according to the current state of the distributed node, the task to be processed is executed, and the execution state of the task to be processed stored in the database is set to a state of being processed; The execution status of the task to be processed stored in the database is updated according to the execution status of the task to be processed.

6. The distributed cluster management method according to claim 5, characterized in that: The distributed cluster management method also includes: If it is determined that the task to be processed cannot be executed according to the current state of the distributed node, the execution state of the task to be processed stored in the database is set to an unfinished processing state.

7. A distributed cluster management method, characterized in that: When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster, and the database also stores the execution status of each task. The management method of the distributed cluster includes: Receiving task distribution information sent by a master node in the distributed cluster, the task distribution information including unfinished tasks acquired by the master node from the database according to a set period, and information of slave nodes to which the unfinished tasks are to be distributed; According to the information of the slave nodes to which the unfinished tasks are to be distributed, the unfinished tasks are distributed to the corresponding slave nodes for processing.

8. The distributed cluster management method according to claim 7, characterized in that: Before receiving the task distribution information sent by the master node in the distributed cluster, the distributed cluster management method further includes: Receiving a registration request sent by a distributed node in the distributed cluster; A master node is selected according to information of registered distributed nodes, and a first notification message is sent to the distributed nodes in the distributed cluster, where the first notification message is used to indicate information of the selected master node.

9. The distributed cluster management method according to claim 8, characterized in that: The distributed cluster management method also includes: If an abnormality is detected in the master node, a new master node is selected from the registered distributed nodes; A second notification message is sent to the distributed nodes in the distributed cluster according to the reselected master node, where the second notification message is used to indicate the reselected master node.

10. The distributed cluster management method according to claim 9, characterized in that: The distributed cluster management method also includes: Receive a callback function registered after the distributed node is started, the callback function including a fourth callback function for notifying the distributed node that it is selected as a master node, a fifth callback function for notifying the master node to change to a slave node, and a sixth callback function for notifying the master node in the distributed cluster of a change; Sending a first notification message to the distributed nodes in the distributed cluster includes: calling the fourth callback function to send a notification message to the selected master node, and calling the sixth callback function to send a notification message to other nodes other than the master node; Sending a second notification message to the distributed nodes in the distributed cluster according to the reselected master node includes: calling the fifth callback function to send a notification message to the original master node, and calling the fourth callback function to send a notification message to the reselected master node.

11. A distributed cluster management method, characterized in that: When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster, and the database also stores the execution status of each task. The management method of the distributed cluster includes: Receiving a target task distributed by a master node in the distributed cluster, wherein the target task is an unfinished task obtained by the master node from the database; Execute the target task, and set the execution status of the target task stored in the database to a processing status; The execution status of the target task stored in the database is updated according to the execution status of the target task.

12. The distributed cluster management method according to claim 11, characterized in that: The management method further comprises at least one of the following steps: Calling a seventh callback function to transmit a message that needs to be sent to the master node or a message that needs to be forwarded by the master node to other slave nodes to the master node, wherein the seventh callback function is used for sending a message from a slave node in the distributed cluster to the master node; The cluster management framework calls an eighth callback function to send a message from the master node, where the eighth callback function is used to receive a message sent by the master node to the slave node.

13. A distributed cluster management device, characterized in that: When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster, and the database also stores the execution status of each task. The management device of the distributed cluster includes: A reading unit, configured to read the execution status of each task stored in the database according to a set period, wherein the execution status stored in the database is updated according to the processing status of the task by the distributed node; An acquisition unit, configured to acquire unfinished tasks in the database according to the execution status of each task; The distribution unit is configured to distribute the acquired unfinished processed tasks to the distributed nodes in the distributed cluster for processing.

14. A distributed cluster management device, characterized in that: When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster, and the database also stores the execution status of each task. The management device of the distributed cluster includes: A receiving unit, configured to receive task distribution information sent by a master node in the distributed cluster, wherein the task distribution information includes unfinished tasks obtained by the master node from the database according to a set period, and information of slave nodes to which the unfinished tasks are to be distributed; The sending unit is configured to distribute the unfinished tasks to corresponding slave nodes for processing according to the information of the slave nodes to which the unfinished tasks are to be distributed.

15. A distributed cluster management device, characterized in that: When each distributed node in the distributed cluster receives a task sent by a task creator, it stores the task sent by the task creator in a database corresponding to the distributed cluster, and the database also stores the execution status of each task. The management device of the distributed cluster includes: A receiving unit, configured to receive a target task distributed by a master node in the distributed cluster, wherein the target task is an unfinished task obtained by the master node from the database; an execution unit, configured to execute the target task and set the execution state of the target task stored in the database to a processing state; An updating unit is configured to update the execution status of the target task stored in the database according to the execution status of the target task.

16. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the distributed cluster management method according to any one of claims 1 to 12 is implemented.

17. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more computer programs, which, when executed by the one or more processors, enables the electronic device to implement the distributed cluster management method as described in any one of claims 1 to 12.

Citation Information

Cited By

  • State detection method and device for graph database task

    CN120743887A