Intelligent Operation and Maintenance System and Method Based on Database Virtual Software

Abstract

The present invention discloses an intelligent operation and maintenance system and method based on database virtual software, including: writing a task configuration file and an operation and maintenance script according to the operation and maintenance tasks; generating a new enhanced script by arranging and compiling the operation and maintenance script files defined in the configuration file according to the task flow; transmitting the enhanced script and the medium to the specified paths of all remote hosts; selecting the corresponding task node configuration file according to the current software architecture scenario; according to the host node numbers described in the task node configuration file, describing the corresponding relationship between the host node numbers and IPs in the node information configuration file, as well as the account passwords used for operation and maintenance; initiating a call to execute the scripts on all host nodes that need to be executed for the current task, and generating a log file. The present invention simplifies the development difficulty of the operation and maintenance program by abstracting the execution steps of the operation and maintenance script into the configuration of the task flow arrangement. At the same time, it enhances the script to improve the development efficiency of the operation and maintenance program.

Description

Technical Field

[0001] The present invention belongs to the technical field of computers, and relates to an intelligent operation and maintenance system and method based on database virtual software. Background Technique

[0002] With the popularity of distributed system architectures, especially the microservices architecture style, it can meet the rapidly growing business needs of software development teams. However, at the same time, the number of service nodes is also increasing rapidly, making the operation and maintenance operations such as system installation and upgrade of the entire system become increasingly cumbersome and complex. The existing software operation and maintenance methods generally include: writing operation and maintenance scripts containing multiple tasks and executing them sequentially on each node in a specific software architecture scenario; but this will cause the operation and maintenance scripts of large software systems to be very complex and difficult to maintain; at the same time, different operation and maintenance scripts need to be developed to meet different customer scenarios (architecture scenarios); moreover, as the system architecture evolves, the scripts will become more and more complex. This undoubtedly greatly increases the operation and maintenance costs and cannot guarantee the execution efficiency and success rate of operation and maintenance. Summary of the Invention

[0003] The purpose of the present invention is to solve the problems in the prior art that software operation and maintenance scripts are complex and difficult to maintain, and different operation and maintenance scripts are developed in the face of different architecture scenarios, resulting in the increasing bloat of operation and maintenance scripts, and to provide an intelligent operation and maintenance system and method based on database virtual software.

[0004] To achieve the above object, the present invention adopts the following technical solutions:[[]]

[0005] An intelligent operation and maintenance system based on database virtual software includes: a script writing module, a task scheduling module, a script enhancement module, a file transfer module, a remote execution module, and a report generation module;

[0006] The script writing module is used to write a task configuration file and an operation and maintenance script according to the operation and maintenance tasks;

[0007] The task scheduling module is used to read and parse the task configuration file to obtain a task process scheduling configuration file, a task node configuration file, and a node information configuration file;

[0008] The script enhancement module is used to generate a new enhanced script according to the operation and maintenance script file defined in the task process scheduling configuration file, add the definition of variables at the front of the new script, and execute the enhanced new script when executing the operation and maintenance task;

[0009] The transmission file module configures the enhancement scripts and required media defined in the task process orchestration configuration file, and transfers the enhancement scripts and media to the specified paths of all remote hosts; selects the corresponding task node configuration file according to the current software architecture scenario; and according to the host node numbers described in the task node configuration file, describes the correspondence between the host node numbers and IPs, as well as the account passwords used for operation and maintenance in the node information configuration file.

[0010] The remote execution script module initiates the call and execution of the scripts on all host nodes that need to be executed for the current task.

[0011] The report generation module obtains the standard output and standard error output returned after each task execution, and prints them to the console in real time and records them in the log file.

[0012] A further improvement of the present invention lies in:

[0013] Furthermore, the script writing module includes: a splitting sub-module and a writing sub-module; the splitting sub-module is used to split the operation and maintenance process of the target system to obtain several operation and maintenance tasks; the writing sub-module is used to write the task configuration file and operation and maintenance scripts according to the operation and maintenance tasks.

[0014] Furthermore, the operation and maintenance tasks include: environment detection, dependent media check, basic environment configuration, and installation service.

[0015] Furthermore, the task configuration files include: a task process orchestration configuration file, a task node configuration file, and a node information configuration file; the task process orchestration configuration file describes the execution order of each task and whether concurrency is performed; the task node configuration file describes the node identifier, connection information, and authentication information; the node information configuration file describes the IP address, port number, and resource allocation of the node.

[0016] Furthermore, the script enhancement module generates a new enhanced script. Specifically, it adds the definition of variables at the very front of the new script. When writing the operation and maintenance script, there is no need to redefine variables and variable assignments; the variable names added to the enhanced script come from the script input parameter variable names defined in the task process orchestration configuration file, and the variable values are obtained from the automatic parsing of the standard output of the previously executed script; the parsing method is: after each task is executed, all variable names and values are parsed according to the standard output format of the operation and maintenance script defined in the task process orchestration configuration file, and cached in the variable pool in the form of key-value. If the subsequent task outputs the same variables as the previous task, the variable values are directly updated.

[0017] Further, if a task fails during the execution of the report generation module, it is determined whether to continue executing subsequent tasks or end the entire operation and maintenance based on the value of the "ignore_failed" field of this task configured in the task process orchestration; after all tasks are completed, the final operation and maintenance report is generated.

[0018] Further, the remote execution script module initiates the call and execution of the scripts on all host nodes that need to execute the current task. Specifically: the job flow is obtained by parsing the configuration file, and each task is executed according to the task sequence described in the job flow; before each task is executed, an enhanced script needs to be generated first, and the enhanced script and the medium are transferred to the target host node; tasks in a serial relationship must wait for the previous task to end before the next task can be executed; tasks in a parallel relationship can be executed simultaneously.

[0019] Further, the job flow is obtained by parsing the configuration file. Specifically: the task process data obtained by parsing the task process orchestration configuration file is used as the main data, and the data obtained by parsing the task node configuration file and the node information configuration file is used as supplementary data to obtain the task flow data.

[0020] An intelligent operation and maintenance method based on database virtual software includes:

[0021] Write a task configuration file and an operation and maintenance script according to the operation and maintenance tasks;

[0022] Read and parse the task configuration file to obtain the task process orchestration configuration file, the task node configuration file, and the node information configuration file;

[0023] Generate a new enhanced script according to the operation and maintenance script file defined in the task process orchestration configuration file, add the definition of variables at the front of the new script, and execute the enhanced new script when executing the operation and maintenance tasks;

[0024] According to the enhanced script and the required medium defined in the task process orchestration configuration file, transfer the enhanced script and the medium to the specified path of all remote hosts; select the corresponding task node configuration file according to the current software architecture scenario; according to the host node numbers described in the task node configuration file, describe the corresponding relationship between the host node numbers and IPs in the node information configuration file, as well as the account passwords used for operation and maintenance;

[0025] Initiate the call and execution of the scripts on all host nodes that need to execute the current task;

[0026] Obtain the standard output and standard error output returned after each task is executed, and print them to the console in real time and record them in the log file.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] By abstracting the execution steps of the operation and maintenance scripts into the configuration of task process orchestration, the present invention reduces hard coding. Through a small amount of configuration, the serial and parallel dependency relationships between tasks are described, making the operation and maintenance process flexible and configurable, and the configuration is simple enough to simplify the development difficulty of operation and maintenance programs. At the same time, by separating the configuration of task execution steps and task execution objects (nodes), the change points are separated, enabling the configuration of each operation and maintenance task in a complex and changeable software architecture to be both simple and flexible in different scenarios. The present invention performs enhancement processing on the script through a script enhancement module, allowing developers to directly use variables when writing scripts without the need to define variables, thereby improving the development efficiency of operation and maintenance programs. The automatic assignment method of task script input parameters implemented in the execution engine greatly simplifies the cumbersome configuration work of operation and maintenance script variables, that is, it improves the development efficiency of operation and maintenance programs. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0030] Figure 1 It is a schematic structural diagram of an intelligent operation and maintenance system for a virtual software of the present invention;

[0031] Figure 2 It is a schematic diagram of the process for implementing operation and maintenance;

[0032] Figure 3 It is a schematic diagram of the construction of a task process orchestration configuration file task-flow.yaml;

[0033] Figure 4 It is a schematic diagram of a fresh installation and an upgrade installation of task-flow.yaml;

[0034] Figure 5 It is a schematic diagram of the construction of a task node configuration file task-node.yaml.

[0035] Figure 6 It is a schematic diagram of the IP address corresponding to each node number in a node information configuration file base.cfg. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. Components of the embodiments of the present invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0037] Therefore, the detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0038] It should be noted that like reference numerals and letters denote like items in the following drawings. Therefore, once an item is defined in one drawing, it does not require further definition and explanation in subsequent drawings.

[0039] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper", "lower", "horizontal", "inner", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, or the orientations or positional relationships in which the inventive product is customarily placed during use, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In addition, terms such as "first", "second", etc. are only used for descriptive distinction and should not be construed as indicating or implying relative importance.

[0040] In addition, if the term "horizontal" appears, it does not mean that the component is required to be absolutely horizontal, but it can be slightly inclined. For example, "horizontal" only means that its direction is more horizontal relative to "vertical", and does not mean that the structure must be completely horizontal, but it can be slightly inclined.

[0041] In the description of the embodiments of the present invention, it should also be noted that unless otherwise clearly specified and limited, if terms such as "set", "installed", "connected", "coupled" are understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0042] The present invention will be further described in detail below with reference to the accompanying drawings:

[0043] SeeFigure 1 , the present invention discloses an intelligent operation and maintenance system based on database virtual software, including: a script writing module, a task scheduling module, a script enhancement module, a file transfer module, a remote execution module, and a report generation module;

[0044] The script writing module is used to write a task configuration file and an operation and maintenance script according to the operation and maintenance tasks;

[0045] The task scheduling module is used to read and parse the task configuration file to obtain a task process scheduling configuration file, a task node configuration file, and a node information configuration file;

[0046] The script enhancement module is used to generate a new enhanced script according to the operation and maintenance script file defined in the task process scheduling configuration file, add the definition of variables at the front of the new script, and execute the enhanced new script when executing the operation and maintenance task;

[0047] The file transfer module is based on the enhanced script and the required media defined in the task process scheduling configuration file, and transfers the enhanced script and the media to the specified paths of all remote hosts; selects the corresponding task node configuration file according to the current software architecture scenario; according to the host node number described in the task node configuration file, describes the correspondence between the host node number and the IP in the node information configuration file, as well as the account password used for operation and maintenance;

[0048] The remote execution script module initiates a call to execute the scripts on all host nodes that need to be executed for the current task;

[0049] The report generation module obtains the standard output and standard error output returned after each task is executed, and prints them to the console in real time and records them in the log file.

[0050] The script writing module includes: a splitting sub-module and a writing sub-module; the splitting sub-module is used to split the operation and maintenance process of the target system to obtain several operation and maintenance tasks; the writing sub-module is used to write a task configuration file and an operation and maintenance script according to the operation and maintenance tasks.

[0051] The operation and maintenance tasks include: environment detection, dependent media check, basic environment configuration, and installation service.

[0052] The task configuration file includes: a task process scheduling configuration file, a task node configuration file, and a node information configuration file; the task process scheduling configuration file describes the execution order of each task and whether concurrency is performed; the task node configuration file describes the node identifier, connection information, and authentication information: the node information configuration file describes the IP address, port number, and resource allocation of the node.

[0053] The script enhancement module generates a new enhanced script. Specifically, it adds variable definitions at the very beginning of the new script. When writing the operation and maintenance script, there is no need to re-define variables and assign values to variables. The variable names added to the enhanced script come from the script input parameter variable names defined in the task flow orchestration configuration file, and the variable values are obtained from the automatic parsing of the standard output of the previously executed script. The parsing method is as follows: after each task is executed, all variable names and values are parsed according to the standard output format of the script defined in the task flow orchestration configuration file and cached in the variable pool in the form of key-value. If the subsequent task outputs the same variable as the previous task, the variable value is directly updated.

[0054] If a task fails during the execution of the report generation module, it is determined whether to continue executing the subsequent tasks or end the entire operation and maintenance according to the value of the "ignore_failed" field of this task in the task flow orchestration configuration. After all tasks are executed, the final operation and maintenance report is generated.

[0055] The remote execution script module initiates the call and execution of the scripts on all host nodes that need to be executed for the current task. Specifically, it parses the configuration file to obtain the job flow and executes each task according to the task order described by the job flow. Before each task is executed, an enhanced script needs to be generated first, and the enhanced script and media are transferred to the target host node. Tasks in a serial relationship must wait for the previous task to finish executing before the next task can be executed. Tasks in a parallel relationship can execute tasks simultaneously.

[0056] Parse the configuration file to obtain the job flow. Specifically, the task flow data obtained by parsing the task flow orchestration configuration file is used as the main data, and the data obtained by parsing the task node configuration file and the node information configuration file is used as supplementary data to obtain the task flow data.

[0057] An intelligent operation and maintenance method based on database virtual software includes:

[0058] Write a task configuration file and an operation and maintenance script according to the operation and maintenance tasks;

[0059] Read and parse the task configuration file to obtain the task flow orchestration configuration file, the task node configuration file, and the node information configuration file;

[0060] Generate a new enhanced script according to the operation and maintenance script file defined in the task flow orchestration configuration file, add variable definitions at the very beginning of the new script, and when executing the operation and maintenance task, execute the enhanced new script;

[0061] Arrange the enhancement scripts and required media defined in the configuration file according to the task process, and transfer the enhancement scripts and media to the specified paths of all remote hosts; select the corresponding task node configuration file according to the current software architecture scenario; according to the host node numbers described in the task node configuration file, describe the correspondence between the host node numbers and IPs in the node information configuration file, as well as the account passwords used for operation and maintenance.

[0062] Initiate the call and execution of the scripts on all host nodes that need to be executed for the current task.

[0063] Obtain the standard output and standard error output returned after each task execution, and print them to the console in real time and record them in the log file.

[0064] Embodiment:

[0065] The present invention discloses an intelligent operation and maintenance method based on database virtual software, including: writing a configuration file and an operation and maintenance script → developing an execution engine → implementing operation and maintenance.

[0066] Among them, referring to Figure 2 , the key action for implementing operation and maintenance is to start the execution engine, and the execution steps of the execution engine are:

[0067] (1) Read and parse the configuration file

[0068] (2) Execute scheduling tasks according to the parsed task flow. The execution process of each task is divided into: generating an enhancement script, transferring files to the target node, remotely calling the operation and maintenance script corresponding to the task, obtaining the output after the script execution and parsing the output, and adding the parsed variables to the variable pool to be used for assigning values to the input parameters of subsequent scripts.

[0069] (3) If a task fails during execution, determine whether to continue executing subsequent tasks or end the entire operation and maintenance according to the value of the "ignore_failed" field of this task configured in the task process arrangement.

[0070] (4) After all tasks are executed, generate the final operation and maintenance report.

[0071] I. The specific steps for writing the configuration file are as follows:

[0072] (1) Analyze the operation and maintenance actions of the entire software system, formulate a reasonable operation and maintenance process, and split it into individual operation and maintenance tasks. For example: environment detection, dependent media check, basic environment configuration, installation of service A, installation of service B, etc.

[0073] (2) Write operation and maintenance scripts according to the split operation and maintenance tasks. The written scripts should be as reusable as possible. It is convenient for subsequent reuse during operation and maintenance process configuration.

[0074] (3) Write a task process orchestration configuration file according to the operation and maintenance tasks. The operation and maintenance process configuration file supports describing information such as the execution order of each task and whether it is concurrent.

[0075] (4) Write a task node configuration file that needs to be executed for the operation and maintenance tasks according to different software architecture scenarios.

[0076] II. Develop an execution engine

[0077] Parse the configuration: Parse the task process orchestration configuration file, the task node configuration file, and the node information configuration file.

[0078] The parsing process can be understood as a process in which the program reads the data of the 3 configuration files into the memory for calculation. After reading the task process orchestration configuration file, the obtained is the task process data. Then, taking the task process data as the main data and the task node configuration and node information configuration data as its supplementary data, the three types of data are sorted out to obtain a complete task flow data. The task flow can also be called the task process, which is a concept in the process engine. The operation and maintenance execution engine executes tasks in sequence according to the sequence of each task described in the task flow and the relationship between concurrency and serialization. This process can also be called scheduling. The task flow data also contains the operation and maintenance script path, script type, medium, input parameters of the script that must be available when each task is executed, the handling method (whether to ignore) after the script execution fails, and whether to execute concurrently.

[0079] Schedule operation and maintenance tasks: Schedule operation and maintenance tasks serially or in parallel according to the execution process described in the task process orchestration configuration file.

[0080] Automatic script enhancement processing: Automatically generate a new enhanced script according to the operation and maintenance script file defined in the task process orchestration configuration file. When executing the operation and maintenance tasks, instead of directly executing the original script, the enhanced new script is executed. In the new script, variable definitions are added at the very front. When developers write operation and maintenance scripts, for variable values that need to be calculated after the previous operation and maintenance tasks are completed, they do not need to define variables and assign values to the variables themselves. They can be directly used just like using operating system environment variables, greatly improving the development efficiency. The variable names of the input parameters come from the script input parameter variable names defined in the task process orchestration configuration file, and the variable values are obtained from the automatic parsing of the standard output of the previously executed script. The parsing method is as follows: After each task is executed, all variable names and values are parsed according to the standard output format of the operation and maintenance script defined in the task process orchestration configuration file and cached in the variable pool in the form of key-value. If subsequent tasks output the same variables as the previous tasks, the variable values are directly updated. This automatic assignment method of task script input parameters greatly simplifies the operation and maintenance configuration work.

[0081] Transfer files and media: Arrange the task scripts and required media defined in the configuration file according to the task process, and transfer the enhanced scripts and media to the specified paths on all remote hosts. The hosts to be transferred are all the host nodes that need to be executed for the current task described in the task node configuration file.

[0082] There are two types of files to be transferred:

[0083] The path of the operation and maintenance script is derived from the task flow data parsed by the execution engine. And the task flow data is derived from the configuration file. Therefore, the path of the operation and maintenance script is configured in the script field of the task process orchestration configuration file task-flow.yaml. See Figure 3 。

[0084] The path of the media also comes from the configuration in task-flow.yaml, the mediaPath field. See Figure 3 。

[0085] Remote execution script: Remotely initiate a call to execute the scripts on all hosts that need to be executed for the current task.

[0086] The call is to execute an ssh command on the node where the execution engine is located, but the same ssh command needs to be executed for each target node required for the current task. That is to say, a task may be executed on multiple nodes.

[0087] The conditions for the call are: Parse the configuration file to obtain the job flow, and execute each task according to the task order described in the job flow. Before each task is executed, an enhanced script needs to be generated first, and then the enhanced script and media are transferred to the target host (node). Tasks in a serial relationship must wait for the previous task to finish before the next one can be executed. Parallel tasks can be executed simultaneously

[0088] Generate execution report: Streamingly obtain the standard output and standard error output returned after each task is executed, and print them to the console in real time and record them in the log file. After a task is executed, automatically parse the Json-formatted output to obtain variables for subsequent tasks to use, and confirm success / failure based on the return code. If it fails, the execution ends. After all tasks are executed, automatically summarize the execution result information of all tasks to form a report, which is convenient for operation and maintenance personnel to intuitively view the operation and maintenance results.

[0089] III. Implement operation and maintenance

[0090] (1) Select the corresponding task node configuration file according to the current software architecture scenario (the same set of software may have different architecture scenarios. For example, for a service, whether it is a single instance or a cluster, and the hardware requirements such as the number of cpu cores and memory size of each service for the host can all be flexibly changed according to business needs).

[0091] (2) According to the host node numbers described in the task node configuration file, describe the correspondence between the node numbers and IPs in the node information configuration file, as well as the account passwords used for operation and maintenance.

[0092] As Figure 5 shown, the base.cfg configuration file describes the mapping relationship between the numbers and IPs of the 3 hosts (nodes) that all need operation and maintenance.

[0093] For example, node1 is the node numbered 1, with an IP of 192.168.1.10, and this number corresponds to the Figure 6 nodeNumber field in Figure 5 The configuration in Figure 5 then indicates that the tasks to be executed by the node with nodeNumber 1 are: taskA, taskB, taskD. And the task names correspond to the Figure 3 name field of each task in

[0094] (3) Run the execution engine, initiate operation and maintenance, view the real-time logs on the console, and observe the execution status of the operation and maintenance operations.

[0095] (4) If the execution fails, view the operation and maintenance report to quickly locate and solve the problem; after solving the problem, the execution engine supports continuing to execute until all tasks are successfully executed.

[0096] During the process of configuring the file, elaborate on the configuration file, specifically:

[0097] 1. Task flow orchestration configuration file task-flow.yaml

[0098] Complete the orchestration of all subtasks included in the operation and maintenance task flow through a configuration file. Include information such as the name, type, whether to execute concurrently, the media on which the task execution depends, the path of the task execution script, input parameters, etc. This configuration file can accurately describe all the operation and maintenance tasks that need to be executed, as well as the sequential or concurrent execution relationship between tasks, and whether to continue execution or directly interrupt the task after failure. Specific configuration items are as Figure 3 shown.

[0099] In addition, according to different operation and maintenance purposes, different process configurations can also be written. For example, there can be different operation and maintenance steps for a fresh installation and an upgrade installation. As Figure 4 shown.

[0100] 2. Task node configuration file task-node.yaml

[0101] For a large software system, the operation and maintenance operation sequence for each component is relatively fixed. That is to say, the task flow scheduling configuration file compiled according to the entire operation and maintenance process will not change frequently. However, in different operation scenarios, the target objects (nodes) for each subtask to execute may change.

[0102] For example, for a certain microservice A, when the business concurrency is 10,000, A needs to be deployed to nodes 1 and 2 in the cluster. But when the concurrency is 100,000, A needs to be deployed to nodes 1, 2, and 3. And microservice B that has an affinity with A can only be deployed to nodes 1, 2, and 3, while microservice C that has an anti-affinity with A cannot be deployed to nodes 1, 2, and 3. Each microservice has its own resource requirements (CPU, memory, etc.) for the operating environment. Considering that overall deployment resources should be saved as much as possible, in different business scenarios, the node positions for each service deployment should be considered as a whole.

[0103] As Figure 5 shown in the directory structure, according to the two dimensions of business scale and cluster size, different task node configuration files are planned. scale1 represents a scale with a concurrency of 10,000, and it can be further divided into 4 deployment scenarios below, that is, the deployment methods when the number of nodes is 1, 2, 3, and 6 respectively. See Figure 5 , design a configuration example task-node.yaml for a business scale of 100,000 and a cluster size of 3 nodes; in Figure 5 , task A and B have an affinity, and the execution nodes of the corresponding tasks are the same, while A and C have an anti-affinity, and the execution nodes are different.

[0104] The key point of the task-node.yaml configuration file is to express its structure, that is, what configuration items there are. Through this configuration item, it can be shown which operation and maintenance tasks need to be executed on each host (node). When the execution engine starts, this configuration file will be read together with the other two configuration files first and converted into a job flow containing complete operation and maintenance tasks.

[0105] Business scale: The business scale of the software currently under operation and maintenance. Since different business scales may have different software architecture methods, task-node.yaml can also be written in many copies for different business scenarios and the appropriate copy can be used according to the business scale situation of the current software during each operation and maintenance. Therefore, this field is mainly for the convenience of human reading and distinction, and can be set as a number representing the business scale size, or can also be set as a name string representing a specific business scenario.

[0106] Cluster size: The number of hosts (nodes) that need to be executed for the current operation and maintenance implementation.

[0107] Node number: The number of the node.

[0108] Memory size: The minimum memory size required for the host of this node under the current business scale scenario. After the execution engine starts, it will first check the resource configuration of each node before executing all tasks, including memory, CPU, disk space, etc. If the resources on the node are less than those defined in the configuration file, the operation and maintenance personnel will be prompted that the resource configuration is insufficient, and the operation and maintenance tasks will not be initiated.

[0109] Task name: The tasks field is an array (list), and the names of multiple tasks can be set, indicating that there are multiple tasks to be executed by this node.

[0110] 3. Node information configuration file base.cfg

[0111] The first two configuration files describe the overall execution steps of the operation and maintenance tasks, the dependent resources (media scripts, etc.), and the target nodes where each task needs to be executed. Finally, when implementing in a specific environment, the environmental information on-site needs to be obtained to configure the node information configuration file. This configuration is used to specify the IP address corresponding to each node number. As Figure 6 shown.

[0112] The terminal device provided by an embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0113] The computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention.

[0114] The terminal device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0115] The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0116] The memory can be used to store the computer program and / or module. By running or executing the computer program and / or module stored in the memory, and invoking the data stored in the memory, the processor implements various functions of the terminal device.

[0117] If the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0118] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intelligent operation and maintenance system based on database virtual software, characterized in that Including: A script writing module, a task scheduling module, a script enhancement module, a file transfer module, a remote execution module, and a report generation module; The script writing module is used to write a task configuration file and an operation and maintenance script according to the operation and maintenance tasks; The task scheduling module is used to read and parse the task configuration file to obtain a task flow orchestration configuration file, a task node configuration file, and a node information configuration file; The script enhancement module is used to generate a new enhanced script according to the operation and maintenance script file defined in the task flow orchestration configuration file, and execute the new enhanced script when performing the operation and maintenance task; In the new script, variable definitions are added at the very beginning. The variable names added to the enhanced script come from the script input parameter variable names defined in the task flow orchestration configuration file, and the variable values are obtained from the automatic parsing of the standard output of the previously executed script; The file transfer module transfers the enhanced script and the required media to the specified paths of all remote hosts according to the enhanced script and the required media defined in the task flow orchestration configuration file; selects the corresponding task node configuration file according to the current software architecture scenario; According to the host node numbers described in the task node configuration file, describe the corresponding relationship between the host node numbers and IPs in the node information configuration file, as well as the account passwords used for operation and maintenance; The remote execution script module initiates a call to execute the scripts on all host nodes that need to be executed for the current task; The remote execution script module initiates a call to execute the scripts on all host nodes that need to be executed for the current task. Specifically: parse the configuration file to obtain the job flow, and execute each task according to the task sequence described in the job flow; before each task is executed, an enhanced script needs to be generated first, and the enhanced script and the media are transferred to the target host node; tasks in a serial relationship must wait for the previous task to finish before the next task can be executed; tasks in a parallel relationship can execute tasks simultaneously; the specific process of parsing the configuration file to obtain the job flow is: the task flow data obtained by parsing the task flow orchestration configuration file is used as the main data, and the data obtained by parsing the task node configuration file and the node information configuration file is used as supplementary data to obtain the job flow data; The report generation module obtains the standard output and standard error output returned after each task is executed, and prints them to the console in real time and records them in the log file.

2. The intelligent operation and maintenance system based on database virtual software according to claim 1, characterized in that The script writing module includes: a splitting sub-module and a writing sub-module; the splitting sub-module is used to split the operation and maintenance process of the target system to obtain several operation and maintenance tasks; the writing sub-module is used to write a task configuration file and an operation and maintenance script according to the operation and maintenance tasks.

3. The intelligent operation and maintenance system based on database virtual software according to claim 2, characterized in that, The operation and maintenance tasks include: environment detection, dependent media check, basic environment configuration, and installation service.

4. The intelligent operation and maintenance system based on database virtual software according to claim 3, characterized in that, The task configuration file includes: a task flow orchestration configuration file, a task node configuration file, and a node information configuration file; the task flow orchestration configuration file describes the execution order of each task and whether concurrency is performed; the task node configuration file describes the node identifier, connection information, and authentication information; the node information configuration file describes the IP address, port number, and resource allocation of the node.

5. The intelligent operation and maintenance system based on database virtual software according to claim 4, characterized in that, The script enhancement module generates a new enhanced script. Specifically, it adds variable definitions at the very beginning of the new script. When writing operation and maintenance scripts, there is no need to re-define variables and assign values to variables. The parsing method is as follows: After each task is executed, all variable names and values are parsed from the standard output format of the operation and maintenance script defined in the task process orchestration configuration file and cached in the variable pool in the form of key-value pairs. If a subsequent task outputs the same variable as a previous task, the variable value is directly updated.

6. The intelligent operation and maintenance system based on database virtual software according to claim 5, wherein, If a task fails during the execution of the report generation module, it is determined whether to continue executing subsequent tasks or end the entire operation and maintenance based on the value of the "ignore_failed" field of this task configured in the task process orchestration. After all tasks are executed, the final operation and maintenance report is generated.

7. An intelligent operation and maintenance method based on database virtual software, characterized in that, It includes: Write task configuration files and operation and maintenance scripts according to operation and maintenance tasks; Read and parse the task configuration file to obtain the task process orchestration configuration file, task node configuration file, and node information configuration file; Generate a new enhanced script according to the operation and maintenance script file defined in the task process orchestration configuration file. When executing the operation and maintenance task, execute the new enhanced script; In the new script, variable definitions are added at the very front. The variable names added to the enhanced script come from the script input parameter variable names defined in the task process orchestration configuration file, and the variable values are derived from the automatic parsing of the standard output of the previously executed script; According to the enhanced script and required media defined in the task process orchestration configuration file, and transfer the enhanced script and media to the specified paths of all remote hosts; select the corresponding task node configuration file according to the current software architecture scenario; According to the host node numbers described in the task node configuration file, describe the corresponding relationship between the host node numbers and IPs in the node information configuration file, as well as the account passwords used for operation and maintenance; Initiate calls to execute the scripts on all host nodes that need to be executed for the current task; Specifically: Parse the configuration file to obtain the job flow, and execute each task according to the task order described by the job flow; before each task is executed, an enhanced script needs to be generated first, and the enhanced script and media are transferred to the target host node; tasks in a serial relationship must wait for the previous task to finish executing before the next task can be executed; tasks in a parallel relationship can execute tasks simultaneously; The specific method of parsing the configuration file to obtain the job flow is as follows: What is parsed from the task process orchestration configuration file is the task process data. Using the task process data as the main data and the data parsed from the task node configuration file and node information configuration file as supplementary data, the job flow data is obtained; Obtain the standard output and standard error output returned after each task is executed, and print them to the console in real time and record them in the log file.

Images