An automated operation and maintenance method and system based on pipeline scheduling

Abstract

This invention relates to an automated operation and maintenance method and system based on pipeline scheduling, comprising: configuring a pipeline; generating tasks through a pipeline scheduling service; and executing tasks, including: the pipeline execution service obtaining node tasks from an execution queue and executing automated operation and maintenance operations according to the node tasks; the pipeline execution service sending the results of the automated operation and maintenance operations to the pipeline scheduling service; the pipeline scheduling service receiving the node task execution results; and proceeding to the next step based on the node task execution results and task status. This method and system can greatly expand automated operation and maintenance scenarios, reduce manual operations in various scenarios, and improve the level of automated operation and maintenance.

Description

Technical Field

[0001] This invention relates to the field of automated operation and maintenance technology, and in particular to an automated operation and maintenance method and system based on pipeline scheduling. Background Technology

[0002] Existing automated operation and maintenance tools, such as Ansible, can perform remote batch operation and maintenance on servers. By writing Ansible scripts or using command lines, and executing Ansible commands on a central host, corresponding operation and maintenance operations can be performed on multiple remote servers.

[0003] However, existing automation tools such as Ansible have the following shortcomings in providing automated operation and maintenance capabilities:

[0004] Task scheduling is not possible; a single maintenance task is executed simultaneously on all target hosts without any order.

[0005] If any step of an operation or maintenance task fails, the current operation or maintenance will be terminated immediately, and subsequent operations will not be executed, requiring manual intervention.

[0006] Operation and maintenance information is not persistently saved, and the operation and maintenance history cannot be viewed at any time. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention proposes an automated operation and maintenance method and system based on pipeline scheduling. The aim is to provide an automated operation and maintenance capability that enables serial control of batch operation and maintenance operations; allows for pausing and retrying operation and maintenance operations; and simultaneously saves all relevant information about the operation and maintenance operations, including execution results and operation information.

[0008] To achieve the above objectives, the technical solution adopted by the present invention includes:

[0009] An automated operation and maintenance method based on pipeline scheduling, characterized by comprising:

[0010] S1. Configure the production line;

[0011] S2. Generate tasks through pipeline scheduling service;

[0012] S3. Execute tasks, including:

[0013] S31. The pipeline execution service retrieves node tasks from the execution queue and performs automated operation and maintenance operations according to the node tasks;

[0014] S32. The pipeline execution service sends the results of automated operation and maintenance to the pipeline scheduling service.

[0015] S33. Pipeline scheduling service receives the task execution results from nodes;

[0016] S34. Proceed to the next step based on the node task execution results and task status.

[0017] Furthermore, step S2 includes the following sub-steps:

[0018] S21. Obtain the pipeline to be operated;

[0019] S22. Generate corresponding tasks according to the operation and maintenance mode; the operation and maintenance mode includes: serial mode and parallel mode;

[0020] S23. Generate a node task for the first node of each task;

[0021] S24. Set the task status to running and send the node task to the execution queue.

[0022] Furthermore, step S34 includes the following sub-steps:

[0023] S341. If the task status is paused, wait for manual operation;

[0024] S342. If the task status is running and the node task execution result is successful, then calculate the value of the next node task.

[0025] S343. If the task status is running and the node task execution result is failure, set the task result to failure and wait for a certain period of time.

[0026] Further, step S341 includes:

[0027] If the task status is set to running manually, then proceed to step S342;

[0028] If the task status is manually set to canceled, the execution will end.

[0029] Further, step S342 includes:

[0030] If a next node exists, proceed to step S24;

[0031] If no next node exists, execution ends.

[0032] Further, step S22 includes:

[0033] Determine whether the operation and maintenance is performed in serial or parallel mode;

[0034] If the operation and maintenance is in serial mode, the incoming host data will be serialized to generate a single task.

[0035] If the operation and maintenance is in parallel mode, then generate a batch of tasks with the same number of incoming hosts;

[0036] The task is an abstract set of automated operation and maintenance activities, including: the pipeline used by the task, the task status, the host to be executed, the start time, and the end time.

[0037] Furthermore, the pipeline is a pipeline formed by task execution steps, with each step being a node, and the pipeline containing one or more nodes.

[0038] Furthermore, each node represents the operation performed in each step, and each node contains several node tasks.

[0039] Furthermore, the node task is the process of each node's operation and maintenance, including the operation content, task status, and operation result.

[0040] This invention also relates to an automated operation and maintenance system based on pipeline scheduling, characterized in that it includes:

[0041] Pipeline configuration module, used to configure pipelines;

[0042] The task generation module is used to generate tasks through the pipeline scheduling service.

[0043] The task execution module is used to execute tasks.

[0044] The present invention also relates to a computer-readable storage medium, characterized in that the storage medium stores a computer program, which, when executed by a processor, implements the above-described automated operation and maintenance method based on pipeline scheduling.

[0045] The present invention also relates to an electronic device, characterized in that it includes a processor and a memory;

[0046] The memory is used to store pipelines and tasks;

[0047] The processor is used to execute the above-mentioned automated operation and maintenance method based on pipeline scheduling by calling pipelines and tasks.

[0048] The present invention also relates to a computer program product, including a computer program and / or instructions, characterized in that, when the computer program and / or instructions are executed by a processor, they implement the steps of the above-described automated operation and maintenance method based on pipeline scheduling.

[0049] The beneficial effects of this invention are as follows:

[0050] The automated operation and maintenance (O&M) method and system based on pipeline scheduling described in this invention provides an automated O&M capability, enabling serial control of batch O&M operations; pausing and retrying O&M operations; and saving all relevant information about the O&M operations, including execution results and operation information. Serial-parallel control: Pipeline scheduling allows for the control of serial and parallel O&M operations, improving automated O&M efficiency, such as improving efficiency in application polling scenarios; O&M operation control: Pipeline scheduling controls the pausing and retrying behavior of O&M operations, reducing manual operations in abnormal scenarios and improving automated O&M efficiency; Historical review: Supports the review of historical automated O&M operations. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of an automated operation and maintenance method based on pipeline scheduling according to the present invention.

[0052] Figure 2 This is a schematic diagram of an automated operation and maintenance system based on pipeline scheduling according to the present invention. Detailed Implementation

[0053] To better understand the content of this invention, a detailed description will be provided in conjunction with the accompanying drawings and embodiments.

[0054] To meet the needs of various operation and maintenance scenarios, this invention provides an automated operation and maintenance method and system based on pipeline scheduling. This method and system can greatly expand the scope of automated operation and maintenance scenarios, reduce manual operations in many scenarios, and improve the level of automated operation and maintenance.

[0055] Task: An abstract collection of automated operation and maintenance tasks, including the pipeline used by the task, task status, host to be executed, start time, end time, etc.

[0056] Pipeline: A pipeline is a series of steps in task execution, where each step is a node, and the pipeline contains one or more nodes.

[0057] Node: The operations performed in each step.

[0058] Node task: The process of operation and maintenance of each node, including operation content, task status, operation results, etc.

[0059] The first aspect of this invention relates to a process flow as follows: Figure 1 The method shown is an automated operation and maintenance method based on pipeline scheduling, including:

[0060] S1. Configure the production line;

[0061] Once the pipeline is configured, tasks are generated through the pipeline scheduling service.

[0062] S2. Generate tasks through pipeline scheduling service;

[0063] Step S2 includes the following sub-steps:

[0064] S21. Obtain the pipeline to be operated;

[0065] S22. Generate corresponding tasks according to the operation and maintenance mode; the operation and maintenance mode includes: serial mode and parallel mode;

[0066] Step S22 includes:

[0067] Determine whether the operation and maintenance is performed in serial or parallel mode;

[0068] If the operation and maintenance is in serial mode, the incoming host data will be serialized to generate a single task.

[0069] If the operation and maintenance is in parallel mode, then generate a batch of tasks with the same number of incoming hosts;

[0070] The task is an abstract set of automated operation and maintenance activities, including: the pipeline used by the task, the task status, the host to be executed, the start time, and the end time.

[0071] S23. Generate a node task for the first node of each task;

[0072] S24. Set the task status to running and send the node task to the execution queue.

[0073] S3. Execute tasks, including:

[0074] S31. The pipeline execution service retrieves node tasks from the execution queue and performs automated operation and maintenance operations according to the node tasks;

[0075] S32. The pipeline execution service sends the results of automated operation and maintenance to the pipeline scheduling service.

[0076] S33. Pipeline scheduling service receives the task execution results from nodes;

[0077] S34. Proceed to the next step based on the node task execution results and task status.

[0078] Step S34 includes the following sub-steps:

[0079] S341. If the task status is paused, wait for manual operation;

[0080] Step S341 includes:

[0081] If the task status is set to running manually, then proceed to step S342;

[0082] If the task status is manually set to canceled, the execution will end.

[0083] S342. If the task status is running and the node task execution result is successful, then calculate the value of the next node task.

[0084] Step S342 includes:

[0085] If a next node exists, proceed to step S24;

[0086] If no next node exists, execution ends.

[0087] S343. If the task status is running and the node task execution result is failure, set the task result to failure and wait for a certain period of time.

[0088] Task execution process:

[0089] 1. Obtain the pipeline to be used and generate corresponding tasks according to different modes. If it is parallel mode, proceed to step 2; if it is serial mode, proceed to step 3.

[0090] 2. Generate multiple tasks in batches based on the number of hosts provided. Proceed to step 4;

[0091] 3. Serialize the incoming host data to generate a task, proceed to step 4;

[0092] 4. For the first node of each task, generate a node task, proceeding to step 5;

[0093] 5. Set the task status to running and send the node task to the execution queue;

[0094] 6. The pipeline execution service retrieves node tasks from the queue and performs automated operation and maintenance operations according to the node tasks, proceeding to step 7;

[0095] 7. The pipeline execution service sends the results of automated operation and maintenance to the pipeline scheduling service;

[0096] 8. The pipeline scheduling service receives the task execution results from the nodes. It proceeds to the next step based on the execution results and task status. If the task status is paused, proceed to step 9.

[0097] If the task status is running and the node task result is successful, proceed to step 10;

[0098] If the task status is running and the node task result is failure, proceed to step 11;

[0099] 9. The task awaits manual intervention. When the manual intervention sets the task to running again, proceed to step 10; when the manual intervention sets the task to cancel, proceed to step 12.

[0100] 10. Calculate the task value of the next node. If a next node exists, proceed to step 5; otherwise, proceed to step 12.

[0101] 11. Set the task result to failure and wait for a certain period of time. If the task is retried manually within the time limit, set the task to running, regenerate the current node task, and proceed to step 5; otherwise, proceed to step 12.

[0102] 12. End.

[0103] Another aspect of this invention relates to an automated operation and maintenance system based on pipeline scheduling, the structure of which is as follows: Figure 2 As shown, it includes:

[0104] Pipeline configuration module, used to configure pipelines;

[0105] The task generation module is used to generate tasks through the pipeline scheduling service.

[0106] The task execution module is used to execute tasks.

[0107] By using this system, the aforementioned computational processing methods can be executed and the corresponding technical effects can be achieved.

[0108] Embodiments of the present invention also provide a computer-readable storage medium capable of implementing all the steps of the methods in the above embodiments, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, implements all the steps of the methods in the above embodiments.

[0109] Embodiments of the present invention also provide an electronic device for performing the above-described method. As an implementation device for the method, the electronic device includes at least a processor and a memory. In particular, the memory stores data and related computer programs required for performing the method, such as pipelines and tasks. The processor calls the data and programs in the memory to execute all the steps of the method and obtain the corresponding technical effect.

[0110] Preferably, the electronic device may include a bus architecture, which may include any number of interconnected buses and bridges. The bus will include various circuits linked together by one or more processors and memories. The bus may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter may be the same element, i.e., a transceiver, providing a unit for communicating with various other systems over a transmission medium. The processor is responsible for managing the bus and general processing, while the memory may be used to store data used by the processor during operation.

[0111] Additionally, the electronic device may further include components such as a communication module, an input unit, an audio processor, a display, and a power supply. The processor (or controller, operating control) used may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device; the memory may be one or more of a buffer, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices, which can store the aforementioned data information, and may also store programs for executing the information, and the processor can execute the program stored in the memory to achieve information storage or processing, etc.; the input unit is used to provide input to the processor, for example, it can be a button or touch input device; the power supply is used to provide power to the electronic device; the display is used to display images and text, for example, it can be an LCD display. The communication module is a transmitter / receiver that transmits and receives signals via an antenna. The communication module (transmitter / receiver) is coupled to the processor to provide input signals and receive output signals, which can be the same as in conventional mobile communication terminals. Based on different communication technologies, multiple communication modules can be incorporated into the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) is also coupled to a speaker and microphone via an audio processor to provide audio output through the speaker and receive audio input from the microphone, thereby enabling typical telecommunications functions. The audio processor can include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor is coupled to a central processing unit, enabling on-device recording via the microphone and on-device playback of stored sound via the speaker.

[0112] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0116] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An automated operation and maintenance method based on pipeline scheduling, characterized in that, include: S1. Configure the production line; S2. Generate tasks through pipeline scheduling service; S3. Execute tasks, including: S31. The pipeline execution service retrieves node tasks from the execution queue and performs automated operation and maintenance operations according to the node tasks; S32. The pipeline execution service sends the results of automated operation and maintenance to the pipeline scheduling service. S33. Pipeline scheduling service receives the task execution results from nodes; S34. Proceed to the next step based on the node task execution results and task status; Step S2 includes the following sub-steps: S21. Obtain the pipeline to be operated; S22. Generate corresponding tasks according to the operation and maintenance mode; the operation and maintenance mode includes: serial mode and parallel mode; S23. Generate a node task for the first node of each task; S24. Set the task status to running and send the node task to the execution queue; Step S22 includes: Determine whether the operation and maintenance is performed in serial or parallel mode; If the operation and maintenance is in serial mode, the incoming host data will be serialized to generate a single task. If the operation and maintenance is in parallel mode, then the same number of tasks as the number of incoming hosts are generated in batches; the task is an abstract set of automated operation and maintenance operations, including: the pipeline used by the task, the task status, the host to be executed, the start time and the end time.

2. The method as described in claim 1, characterized in that, Step S34 includes the following sub-steps: S341. If the task status is paused, wait for manual operation; S342. If the task status is running and the node task execution result is successful, then calculate the value of the next node task. S343. If the task status is running and the node task execution result is failure, set the task result to failure and wait for a certain period of time.

3. The method as described in claim 2, characterized in that, Step S341 includes: If the task status is set to running manually, then proceed to step S342; If the task status is manually set to canceled, the execution will end.

4. The method as described in claim 2, characterized in that, Step S342 includes: If a next node exists, proceed to step S24; If no next node exists, execution ends.

5. The method as described in claim 1, characterized in that, The pipeline is a pipeline formed by task execution steps, where each step is a node, and the pipeline contains one or more nodes.

6. The method as described in claim 1, characterized in that, Each node represents the operations performed in each step, and each node contains several node tasks.

7. The method as described in claim 1, characterized in that, The node task is the process of each node's operation and maintenance, including the operation content, task status, and operation result.

8. An automated operation and maintenance system based on pipeline scheduling, characterized in that, include: Pipeline configuration module, used to configure pipelines; The task generation module is used to generate tasks through a pipeline scheduling service. The process of generating tasks through the pipeline scheduling service includes: obtaining the pipeline to be operated; generating corresponding tasks according to the operation and maintenance mode; the operation and maintenance mode includes serial mode and parallel mode; generating a node task for the first node of each task; setting the task status to running; and sending the node task to the execution queue. The task generation module is also used to determine whether the operation and maintenance operation is in serial mode or parallel mode. If the operation and maintenance operation is in serial mode, the incoming host is serialized to generate a task. If the operation and maintenance is in parallel mode, then the same number of tasks as the number of incoming hosts will be generated in batches; the task is an abstract set of automated operation and maintenance operations, including: the pipeline used by the task, the task status, the host to be executed, the start time and the end time. The task execution module is used to execute tasks, wherein the task execution includes: the pipeline execution service obtains node tasks from the execution queue and performs automated operation and maintenance operations according to the node tasks; The pipeline execution service sends the results of automated operation and maintenance to the pipeline scheduling service; the pipeline scheduling service receives the node task execution results; and proceeds to the next step based on the node task execution results and task status.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the automated operation and maintenance method based on pipeline scheduling as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, Including processor and memory; The memory is used to store pipelines and tasks; The processor is configured to execute the automated operation and maintenance method based on pipeline scheduling as described in any one of claims 1 to 7 by invoking pipelines and tasks.

11. A computer program product, comprising a computer program and / or instructions, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the steps of the automated operation and maintenance method based on pipeline scheduling as described in any one of claims 1 to 7.

Images