Progressive cluster operation and maintenance system

Abstract

The embodiment of the invention provides a progressive cluster operation and maintenance system, and relates to the technical field of computers. The system comprises a central node, a service module and an operation and maintenance module, wherein the operation and maintenance module is configured on the central node; the operation and maintenance module is used for receiving a task execution instruction from the client and determining a task execution module of the task execution instruction; and the operation and maintenance module is used for calling the service interface of the service module corresponding to the task execution instruction to execute the module operation of the task execution module, obtaining a task execution result corresponding to the task execution instruction, and displaying the task execution result. According to the progressive cluster operation and maintenance system, the operation and maintenance module can be introduced and used for the massive cluster node operation and maintenance system, the service module is deployed on each cluster in the multiple clusters of the progressive cluster operation and maintenance system, and the safety and efficiency of large-scale cluster operation and maintenance can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a progressive cluster operation and maintenance system. Background Art

[0002] In the field of cluster operation and maintenance, Ansible tools are generally used to implement small-scale cluster operation and maintenance. However, in large-scale cluster operation and maintenance, Ansible tools have certain limitations, such as concurrent processing is limited by the number of nodes, high network requirements, the need to install Python on nodes and there are compatibility issues, troublesome debugging, no operation recording and playback functions, etc. There is a problem of poor universality and certain limitations. Summary of the invention

[0003] Based on this, it is necessary to provide a progressive cluster operation and maintenance system to address the above technical issues.

[0004] In a first aspect, the present application provides a progressive cluster operation and maintenance system. The system includes: a central node, a service module and an operation and maintenance module; wherein:

[0005] The operation and maintenance module is configured on the central node;

[0006] The operation and maintenance module is used to receive a task execution instruction from a client and determine a task execution module for the task execution instruction;

[0007] The operation and maintenance module is used to call the service interface of the service module corresponding to the task execution instruction to execute the module operation of the task execution module, obtain the task execution result corresponding to the task execution instruction, and display the task execution result.

[0008] In one of the embodiments, the task execution instruction includes a plurality of tasks to be executed; and the operation and maintenance module is further used to add the plurality of tasks to be executed to a task queue.

[0009] In one of the embodiments, the plurality of tasks to be executed correspond to a plurality of clusters; the operation and maintenance module is further configured to obtain path information of service modules of each of the clusters from a central database of the central node.

[0010] In one of the embodiments, the service module includes a target cluster management service module; and the path information indicates a path from the central node to the target cluster management service module.

[0011] In one of the embodiments, the service module includes a forwarding service module; and there is at least one forwarding service module between the central node and the target cluster management service module.

[0012] In one of the embodiments, the operation and maintenance module is used to determine the current task to be executed among the multiple tasks to be executed; the operation and maintenance module is also used to call the service interface of the current service module corresponding to the current task to be executed according to the current path information corresponding to the current task to be executed, execute the module operation of the task execution module corresponding to the current task to be executed, obtain the task execution result of the current task to be executed, and display the task execution result.

[0013] In one of the embodiments, the task execution module includes a command module and a file module; the service interface of the service module includes an execution Shell command interface; the operation and maintenance module is used to call the execution Shell command interface to execute module operations of the command module and / or module operations of the file module.

[0014] In one embodiment, the task execution module includes a copy module and a download module; the service interface of the service module includes an upload file interface and a download file interface; the operation and maintenance module is used to call the upload file interface to execute the module operation of the copy module; and is used to call the download file interface to execute the module operation of the download module.

[0015] In one embodiment, the service module includes a scheduled task manager; the scheduled task manager is used to regularly collect node information of multiple cluster nodes in each of the multiple clusters.

[0016] In one of the embodiments, the system includes a callback plug-in display output interface; the operation and maintenance module is used to call the callback plug-in display output interface output and display the task execution result.

[0017] In the above-mentioned progressive cluster operation and maintenance system, the operation and maintenance module is configured on the central node; the operation and maintenance module is used to receive the task execution instruction from the client and determine the task execution module of the task execution instruction; the operation and maintenance module is used to call the service interface of the service module corresponding to the task execution instruction to perform the module operation of the task execution module, obtain the task execution result corresponding to the task execution instruction, and display the task execution result. In the system provided by the embodiment of the present application, an operation and maintenance module, i.e., the Xmops tool, is introduced for the massive cluster node operation and maintenance system, and a service module, i.e., the Agent tool, is deployed on each of the multiple clusters in the progressive cluster operation and maintenance system, which has the following beneficial effects:

[0018] (1) Compared with operation and maintenance tools such as Ansible and SaltStack, it is not necessary for all nodes to be able to connect to the central node. It only requires that the virtualized management node in each cluster can connect to the central node's network, which can greatly simplify the cluster construction work.

[0019] (2) Based on basic shell commands, multi-agent operations can greatly improve the efficiency of operating massive nodes compared to Ansible, and can greatly reduce deployment work and ensure the stability of massive node operations compared to SaltStack tools.

[0020] (3) The central tool and Agent are both developed based on compiled languages, which can further improve the operating efficiency compared to tools developed in Python such as Ansible and SaltStack. In addition, since it is based on basic shell commands, the module's dependence on the system is greatly reduced, and there is no need to consider the Python version on the node and the compatibility of Python2 and Python3. Most importantly, putting the core logic of the relevant modules in the tool itself can greatly improve debugging efficiency.

[0021] (4) For large-scale cluster management, each operation information must be recorded to facilitate fault tracing. Xmops will implement the Zset structure in Redis based on RocksDB and use it to save the information of each operation. Only key information can be saved, and the high-performance key-value library can greatly improve the efficiency of playback operations and reduce storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A structural block diagram of a progressive cluster operation and maintenance system provided in an embodiment of the present application;

[0024] Figure 2 A flowchart of a task processing using the Xmops tool provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of a cluster user authentication process of an Xmops tool provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] In an exemplary embodiment, Figure 1As shown, a progressive cluster operation and maintenance system is provided, which may include: a central node, a service module and an operation and maintenance module; wherein the operation and maintenance module is configured on the central node; the operation and maintenance module is used to receive a task execution instruction from a client and determine the task execution module of the task execution instruction; the operation and maintenance module is used to call the service interface of the service module corresponding to the task execution instruction to execute the module operation of the task execution module, obtain the task execution result corresponding to the task execution instruction, and display the task execution result.

[0028] Among them, the operation and maintenance module can be an Xmops tool, and the service module can be an Agent service. The progressive cluster operation and maintenance system includes multiple clusters, for example, cluster 1, cluster 2 and cluster 3, etc., and each of the multiple clusters includes multiple cluster nodes. Each cluster is composed of multiple nodes, network connections, storage systems and cluster management software, and each part cooperates with each other to jointly realize the functions of the cluster system. In an embodiment of the present application, an Agent service can be deployed on each of the multiple clusters of the progressive cluster operation and maintenance system to manage small-scale clusters (for example, a cluster containing 100-500 nodes). The Agent service is deployed on a virtual machine created for the cluster, and the virtual machine is all free of secrets for internal nodes and can access the central node. The internal nodes of the cluster do not need to directly communicate with the external environment. This makes the Agent a key bridge for communication and interaction between the internal nodes of the cluster and the central node. The Xmops tool on the central node can indirectly operate and manage any node in the cluster by communicating with the Agent. The Agent service can provide the following interface functions: (1) Shell command execution interface: Any Shell command can be executed on any node, which is the basis for implementing node operation and maintenance operations, such as executing system management commands, application start / stop commands, etc. (2) File upload interface: Files can be uploaded to any location on any node, which is convenient for deploying applications on the node, updating configuration files, etc. (3) File download interface: Files can be downloaded from any location on any node to obtain log files, data files, etc. on the node for troubleshooting, data analysis, etc. The Agent service also includes a scheduled task manager, which will regularly collect information about all nodes in the current cluster, including the node's resource usage (CPU, memory, disk, etc.), operating status (process status, service status, etc.), etc., and report this information to the MySQL database of the central node. In this way, the central node can understand the status of each node in the cluster in real time and provide data support for subsequent operation and maintenance decisions. In addition, an Xmops tool can be developed in the central node of the progressive cluster operation and maintenance system. The Xmops tool can obtain the Agent information of the management cluster through the central database, so as to achieve the purpose of manipulating any node in any cluster on the central node. The Xmops tool is deployed in the central node and developed based on the C language. It is compatible with all options of Ansible and some core modules, and the implementation of these core modules is based on the interface provided by the Agent. The core module, that is, the task execution module, can include but is not limited to the Shell module, the File module, the Copy module, the Fetch module and the Script module. Among them, the Xmops tool is deployed in the central node and developed based on the C language.It is compatible with all options of Ansible and some core modules, which are implemented based on the interface provided by Agent. Shell module: It is implemented with the help of Agent's remote execution Shell command interface. When the user executes Shell module related operations in Xmops, Xmops executes Shell commands on the target node by calling Agent's interface and obtains the execution results. File module: File deletion, creation, attribute change and other operations are completed by remotely calling Agent's execution Shell command interface in Xmops and using corresponding Shell commands (such as rm, touch, mkdir, chown, chgrp, chmod, etc.). Copy module: Use Agent's upload interface to upload files from the local to the temporary directory of the cluster node managed by Agent, and then call the execution Shell command interface to execute the mv command to move the files to the specified location. If the backup parameter in the copy module is yes, you also need to use the cp command to back up the source file. And according to the mode, owner, group and other parameters in the copy module, the commands of chmod, chown and chgrp are determined, and the Shell interface is called again to modify the file permissions, and finally the details of the final file (size, permissions, owner, etc.) are returned. Fetch module: Use the Agent download interface to download the file of the specified path on the cluster node to the local temporary directory, and then use the mv command to move it to the specified location interface locally to realize the function of getting the file from the node to the local. Script module: Similar to the copy module, first upload the script to the node, then call the shell command interface to remotely execute the script, delete the script after obtaining the execution result, and complete the script running operation on the node.

[0029] Furthermore, the user can trigger a task execution instruction on the client, and the task execution instruction may include multiple tasks to be executed, and the multiple tasks to be executed may correspond to multiple clusters. Next, the user can execute the task execution module corresponding to the task execution instruction in the Xmops tool (for example, execute the Shell module on 1000 nodes in 100 clusters, with the parameter being the hostname command). Specifically, the operation and maintenance module is used to receive the task execution instruction from the client and determine the task execution module of the task execution instruction; the operation and maintenance module is used to call the service interface of the service module corresponding to the task execution instruction to perform the module operation of the task execution module, obtain the task execution result corresponding to the task execution instruction, and display the task execution result.

[0030] In the system of this embodiment, an operation and maintenance module, namely, the Xmops tool, can be introduced for the massive cluster node operation and maintenance system, and a service module, namely, the Agent tool, is deployed on each of the multiple clusters in the progressive cluster operation and maintenance system, which can improve the security and efficiency of large-scale cluster operation and maintenance. The following beneficial effects are achieved:

[0031] (1) Compared with operation and maintenance tools such as Ansible and SaltStack, it is not necessary for all nodes to be able to connect to the central node. It only requires that the virtualized management node in each cluster can connect to the central node's network, which can greatly simplify the cluster construction work.

[0032] (2) Based on basic shell commands, multi-agent operations can greatly improve the efficiency of operating massive nodes compared to Ansible, and can greatly reduce deployment work and ensure the stability of massive node operations compared to SaltStack tools.

[0033] (3) The central tool and Agent are both developed based on compiled languages, which can further improve the operating efficiency compared to tools developed in Python such as Ansible and SaltStack. In addition, since it is based on basic shell commands, the module's dependence on the system is greatly reduced, and there is no need to consider the Python version on the node and the compatibility of Python2 and Python3. Most importantly, putting the core logic of the relevant modules in the tool itself can greatly improve debugging efficiency.

[0034] (4) For large-scale cluster management, each operation information must be recorded to facilitate fault tracing. Xmops will implement the Zset structure in Redis based on RocksDB and use it to save the information of each operation. Only key information can be saved, and the high-performance key-value library can greatly improve the efficiency of playback operations and reduce storage space.

[0035] In an exemplary embodiment, the task execution instruction includes a plurality of tasks to be executed; the operation and maintenance module is further used to add the plurality of tasks to be executed to a task queue.

[0036] In an exemplary embodiment, multiple tasks to be executed correspond to multiple clusters; the operation and maintenance module is further used to obtain path information of service modules of each cluster in the central database of the central node.

[0037] In an exemplary embodiment, the service module includes a target cluster management service module; the path information indicates a path from the central node to the target cluster management service module.

[0038] In an exemplary embodiment, the service module includes a forwarding service module; and there is at least one forwarding service module between the central node and the target cluster management service module.

[0039] Specifically, the user can initiate a task execution instruction through the command line interface or other interactive methods (e.g., graphical interface) of the Xmops tool. The task execution instruction may include multiple tasks to be executed, and the module identifier and task execution parameters of the task execution module of each task to be executed in the multiple tasks to be executed. The Xmops tool may first determine the multiple cluster nodes corresponding to the multiple tasks to be executed, and then, the tasks corresponding to each cluster node may be encapsulated into a separate task object, and the task object package of each task object includes the node identifier, task execution module and task execution parameters of the corresponding target cluster node, and then, the multiple task objects may be added to the task queue to wait for subsequent processing. Next, the Xmops tool may retrieve the complete path information of each cluster in the multiple clusters from the central database Redis. The path information represents the path from the central node to the target cluster management service module; the service module includes a forwarding service module; there is at least one forwarding service module between the central node and the target cluster management service module. The path information may include the connection sequence of multiple layers of forwarding agents, for example, "central node->forwarding agent1->forwarding agent2->cluster management agent1->...->cluster management agent100". The Xmops tool will parse the path information and convert it into an internally processable format so that the task object processing request can be accurately routed to the corresponding target cluster node. Furthermore, the Xmops tool starts the corresponding number of processes (assuming N processes) according to the number of processes specified by the -f option to process the tasks in the task queue. Each process obtains the task from the task queue and executes it according to the following steps: Connect to the target Agent: According to the node path information corresponding to the task, the request to connect to the node is transmitted to the specified Agent (after passing through the forwarding agent until the operation agent). First, establish a connection with the first forwarding agent in the path (if any), send a connection request and include task-related information. After the forwarding agent verifies the request, it establishes a connection and waits to receive subsequent operation instructions. Then, the Xmops tool sends the task data (including module operations and parameters) to the forwarding agent, and the forwarding agent forwards the data to the next agent according to the path information until it reaches the target cluster management agent. Execute module operation: After receiving the task, the target cluster management agent performs the corresponding operation according to the module type in the task. If it is a Shell module, the Agent calls the remote execution Shell command interface, executes the hostname command on the target node, and obtains the command execution result; if it is other modules (such as File module, Copy module, etc.), the corresponding interfaces (such as file operation interface, upload and download interface, etc.) are called respectively to complete the corresponding operations, such as execution file creation, deletion, copying, file attribute modification, file upload, download and move, etc.Result processing and return: After completing the module operation, the target Agent will pass the operation results (including command execution output, file operation status, etc.) to the upper-level forwarding Agent in the opposite path, and the final result will be returned to the Xmops tool. After receiving the result, the Xmops tool will perform necessary processing on the result, such as parsing and formatting, to make it conform to the expected output format.

[0040] In an exemplary embodiment, the process of implementing task processing using the Xmops tool can be described by taking the current task to be executed as an example. The operation and maintenance module is used to determine the current task to be executed among multiple tasks to be executed; the operation and maintenance module is also used to call the service interface of the current service module corresponding to the current task to be executed according to the current path information corresponding to the current task to be executed, perform the module operation of the task execution module corresponding to the current task to be executed, obtain the task execution result of the current task to be executed, and display the task execution result.

[0041] Specifically, Figure 2 As shown, the Xmops tool can determine the current task to be executed in the task queue, and the current task to be executed can be any task in the task queue. Furthermore, the current cluster node of the current task to be executed can be determined. Next, the complete path information to the current cluster node can be obtained from Redis. The complete path information is the path from the central node to the current cluster. There can be at least one forwarding agent between the central node and the current cluster, for example, forwarding agent 1, forwarding agent 2 and forwarding agent 3. In a possible implementation, in the complete path information of the current cluster, there can be forwarding agent 1 from the central node to the current cluster node. In this case, the task processing request of the current task to be executed can first reach the forwarding agent 1, and then, the forwarding agent 1 forwards the task processing request of the current task to be executed to the current management agent of the current cluster. The current management agent can be management agent 3, the current cluster can be cluster 3, and the management agent 3 can operate the current cluster node; further, the service interface of the Agent 3 service of the current cluster node can be called to execute the task execution module of the current task to be executed, obtain the task execution result of the current task to be executed, and return the task execution result to the Xmops tool.

[0042] In an exemplary embodiment, the task execution module includes a command module and a file module; the service interface of the service module includes an execution Shell command interface; and the operation and maintenance module is used to call the execution Shell command interface to execute module operations of the command module and / or module operations of the file module.

[0043] Among them, the command module can be a Shell module. Shell module: implemented with the help of the remote execution Shell command interface of Agent. When the user executes Shell module related operations in Xmops, Xmops executes Shell commands on the target node by calling the interface of Agent and obtains the execution results. The file module can be a File module. File module: operations such as file deletion, creation, and attribute change are completed by remotely calling the execution Shell command interface of Agent in Xmops and using corresponding Shell commands (such as rm, touch, mkdir, chown, chgrp, chmod, etc.). The execution Shell command interface can execute any Shell command on any node, which is the basis for realizing node operation and maintenance operations, such as executing system management commands, application start / stop commands, etc.

[0044] In an exemplary embodiment, the task execution module includes a copy module and a download module; the service interface of the service module includes an upload file interface and a download file interface; the operation and maintenance module is used to call the upload file interface to execute module operations of the copy module; and is used to call the download file interface to execute module operations of the download module.

[0045] Among them, the copy module can be a Copy module, which can use the Agent's upload interface to upload files from the local to the temporary directory of the cluster node managed by the Agent, and then call the Shell command interface to execute the mv command to move the file to the specified location. If the backup parameter in the copy module is yes, the source file needs to be backed up with the cp command. And according to the mode, owner, group and other parameters in the copy module, the chmod, chown, and chgrp commands are determined to be executed, and the Shell interface is called again to modify the file permissions, and finally the details of the final file (size, permissions, owner, etc.) are returned. The download module can be a Fetch module, which uses the Agent's download interface to download the file of the specified path on the cluster node to the local temporary directory, and then uses the mv command to move it to the specified location interface locally to realize the function of getting the file from the node to the local. Script module: Similar to the copy module, first upload the script to the node, then call the shell command interface to remotely execute the script, delete the script after obtaining the execution result, and complete the script running operation on the node. The upload file interface can upload files to any location of any node, which is convenient for deploying applications on the node, updating configuration files, and other operations. The file download interface can download files from any location on any node, and is used to obtain log files, data files, etc. on the node for troubleshooting, data analysis, etc.

[0046] In an exemplary embodiment, the service module includes a scheduled task manager; the scheduled task manager is used to regularly collect node information of multiple cluster nodes in each of the multiple clusters.

[0047] The node information can include the node's resource usage (CPU, memory, disk, etc.), operating status (process status, service status, etc.), etc., and report this information to the central node's MySQL database. Based on this, the central node can understand the status of each node in the cluster in real time and provide data support for subsequent operation and maintenance decisions.

[0048] In an exemplary embodiment, the system includes a callback plug-in display output interface; an operation and maintenance module, which is used to call the callback plug-in display output interface output and display the task execution result.

[0049] After obtaining the execution result of any task, the Xmops tool will call the callback plug-in to display the output interface. The callback plug-in is responsible for displaying the results in a user-friendly way. For example, the operation results of each node are displayed in a clear format in the command line, including the status of success or failure, the detailed content of the execution result (such as the node host name returned by the hostname command), etc. If the operation involves multiple nodes, the results of each node will be displayed in turn, so that users can intuitively understand the execution status of the entire operation in the cluster. At the same time, the displayed result format and content are fully compatible with Ansible's output, which is convenient for users familiar with Ansible to quickly understand and use the Xmops tool.

[0050] Furthermore, during the entire operation, if any errors or abnormalities occur, the Xmops tool will perform corresponding error handling. For example, if a network failure occurs when connecting to the Agent, the Agent service is unavailable, or the target node is unresponsive, the Xmops tool will record detailed error information, including the error type, the time of occurrence, the nodes and tasks involved, etc. At the same time, depending on the severity of the error, some measures may be taken, such as trying to reconnect or execute the task a certain number of times, prompting the user with error information and providing possible solutions, etc. In addition, the Xmops tool will also record key information during the operation (such as user operation requests, task allocation, operation results, error information, etc.) into the log file for subsequent auditing, troubleshooting, and performance analysis. The level of detail and format of the log record can be adjusted according to the system configuration to ensure that sufficient information can be provided for operation and maintenance management.

[0051] In an exemplary embodiment, Figure 3As shown in the figure, the Xmops tool introduces cluster user authentication, which can save the user information in MySQL. When using the Xmops tool, you need to pass the cluster user authentication first. Only after the authentication is successful can you perform subsequent operations. The authentication success information can be written to the local through RocksDB. Each time the Xmops tool is operated, user information, executed task execution commands, task execution results, execution time and other information can be written to the local directory in the form of key-value pairs (Key-Value, KV). With the help of RocksDB's KV storage, a data structure similar to Zset is implemented. The time is used as a score, and the operation record is saved in the specified Key of RocksDB according to the Zset element.

[0052] In the system of this embodiment, when a failure occurs or an audit is required, the operation records can be searched and displayed according to the size of the scores in Zset. The largest score represents the latest operation record. A certain time period can also be selected to display the operations within that time period, which is convenient for operation and maintenance personnel to quickly locate problems and trace operation history, further ensuring the reliability and security of massive node operation and maintenance.

[0053] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0054] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0055] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment system can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned systems. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0056] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0057] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A progressive cluster operation and maintenance system, characterized in that: The system includes: a central node, a service module and an operation and maintenance module; wherein, The operation and maintenance module is configured on the central node; The operation and maintenance module is used to receive a task execution instruction from a client and determine a task execution module for the task execution instruction; The operation and maintenance module is used to call the service interface of the service module corresponding to the task execution instruction to execute the module operation of the task execution module, obtain the task execution result corresponding to the task execution instruction, and display the task execution result.

2. The system according to claim 1, characterized in that The task execution instruction includes a plurality of tasks to be executed; and the operation and maintenance module is further used to add the plurality of tasks to be executed to a task queue.

3. The system according to claim 2, characterized in that The plurality of tasks to be executed correspond to a plurality of clusters; the operation and maintenance module is further used to obtain path information of the service modules of each cluster from the central database of the central node.

4. The system according to claim 3, characterized in that The service module includes a target cluster management service module; the path information indicates a path from the central node to the target cluster management service module.

5. The system according to claim 4, characterized in that The service module includes a forwarding service module; there is at least one forwarding service module between the central node and the target cluster management service module.

6. The system according to claim 2, characterized in that The operation and maintenance module is used to determine the current task to be executed among the multiple tasks to be executed; The operation and maintenance module is also used to call the service interface of the current service module corresponding to the current task to be executed according to the current path information corresponding to the current task to be executed, execute the module operation of the task execution module corresponding to the current task to be executed, obtain the task execution result of the current task to be executed, and display the task execution result.

7. The system according to claim 1, characterized in that The task execution module includes a command module and a file module; the service interface of the service module includes an execution Shell command interface; the operation and maintenance module is used to call the execution Shell command interface to execute the module operation of the command module and / or the module operation of the file module.

8. The system according to claim 1, characterized in that The task execution module includes a copy module and a download module; the service interface of the service module includes an upload file interface and a download file interface; the operation and maintenance module is used to call the upload file interface to execute the module operation of the copy module; and is used to call the download file interface to execute the module operation of the download module.

9. The system according to claim 1, characterized in that The service module includes a timed task manager; the timed task manager is used to regularly collect node information of multiple cluster nodes of each of the multiple clusters.

10. The system according to any one of claims 1 to 9, characterized in that: The system includes a callback plug-in display output interface; the operation and maintenance module is used to call the callback plug-in display output interface to output and display the task execution result.

Images