An automated inspection and self-healing system

Abstract

The present invention discloses an automated inspection and self-healing system, which belongs to the field of intelligent operation and maintenance technology. The system includes several agent modules, which are respectively installed on different target servers, collect the full amount of monitoring data of the target servers in real time, and execute the received optimal self-healing script; a console, which is used to configure inspection task parameters and establish an association relationship between each inspection indicator and its self-healing script; a scheduler, which is used to regularly trigger inspection tasks according to inspection task parameters, mark each abnormal inspection indicator and its monitoring object, and send the marking result to a decision maker; a decision maker, which is used to determine the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm, and distribute each optimal self-healing script to the corresponding agent module for execution. The present application can quickly and on a large scale inspect problems existing in the inspection server, and automatically recover the fault when a system fault is found, thereby ensuring the health and stability of the client server.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance technology, and in particular to an automated inspection and self-healing system. Background Art

[0002] With the continued expansion of the company's business and the increasing number of customers, the number of servers has also increased dramatically. This change has also made the server inspection tasks undertaken by the operations and maintenance team increasingly arduous. Traditional server inspections rely primarily on manual operations, with inspectors logging into each server one by one to check and record the status of various key indicators. However, when faced with large-scale server clusters, this model has exposed a series of significant problems.

[0003] First, manual inspections are costly. As the number of servers increases, companies have to devote more human resources to completing inspections, which directly leads to increased operational costs. Second, manual inspections are inefficient. A single inspector needs to inspect dozens or even more servers daily, each of which has numerous performance indicators and operational statuses that need to be monitored. This high-intensity work is not only time-consuming and labor-intensive, but also easily leads to inspector fatigue, which affects the accuracy and timeliness of inspections.

[0004] Furthermore, manual inspections also suffer from non-standardized inspection content and results. Different inspectors may perform inspections based on their own experience and understanding, resulting in a lack of uniformity and standardization in inspection content. Inspection results are often difficult to compare and analyze horizontally. This not only reduces the quality and value of inspection data but also greatly inconveniences subsequent troubleshooting and performance optimization efforts.

[0005] Therefore, faced with the growing number of servers and increasingly complex operation and maintenance environment, the traditional manual inspection model can no longer meet the company's needs for efficient, accurate and standardized operation and maintenance inspections. Summary of the Invention

[0006] The purpose of the present invention is to provide an automated inspection and self-healing system to solve the problems of high cost and low efficiency caused by manual inspection in the prior art, as well as the problems of non-standard inspection content and non-standard inspection results.

[0007] To achieve the above objectives, this application adopts the following technical solutions:

[0008] The present application provides an automated inspection and self-healing system, comprising several agent modules, a console, a scheduler, and a decision maker, wherein:

[0009] The plurality of agent modules are respectively installed on different target servers, and are used to collect full monitoring data of the target servers in real time and execute the received optimal self-healing script;

[0010] The console is used to configure inspection task parameters, including the inspection task name, execution period, inspection indicator set, and monitoring objects and abnormal thresholds for each inspection indicator, and to establish an association between each inspection indicator and its self-healing script;

[0011] The scheduler is configured to periodically trigger inspection tasks according to the inspection task parameters, and determine the inspection indicator values exceeding the abnormal threshold based on the full amount of monitoring data in the current inspection cycle, so as to mark each abnormal inspection indicator and its monitored object, and send the marking result to the decision maker;

[0012] The decision maker is used to determine the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm, and distribute each optimal self-healing script to the corresponding agent module for execution.

[0013] Preferably, the full monitoring data includes system resource usage data, network performance indicators, application status data, and security log and event data;

[0014] The inspection indicators include CPU usage, memory usage, disk usage, network bandwidth usage and network latency.

[0015] Preferably, the determining of the inspection index value exceeding the abnormal threshold based on the full amount of monitoring data in the current inspection cycle includes:

[0016] Obtain the full monitoring data within the current inspection cycle, and extract the value corresponding to each inspection indicator from the current full monitoring data based on the inspection indicator and its monitoring object;

[0017] The extracted inspection indicator values are compared with the corresponding abnormal thresholds, and all inspection indicators with values exceeding the abnormal thresholds are marked as abnormal inspection indicators.

[0018] Preferably, the console is further configured to set an initial weight for each self-healing script, wherein the initial weights are integers greater than or equal to 0.

[0019] Preferably, the determining of the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm includes:

[0020] If the current inspection is the first inspection, all self-healing scripts associated with each abnormal inspection indicator are determined according to the association relationship, and arranged in descending order according to the initial weight. The self-healing script with the largest weight is taken as the optimal self-healing script, and its weight is increased.

[0021] Preferably, the determining of the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm includes:

[0022] If the current inspection is not the first inspection, determine whether there was any abnormality in the previous inspection;

[0023] If an abnormality occurred during the previous inspection, and the abnormal inspection indicators and monitoring objects obtained during the current inspection are the same as those during the previous inspection, query all self-healing scripts whose latest weight is greater than or equal to 0, sort them in descending order, set the latest weight of the self-healing script with the largest weight to a negative value, continue to query self-healing scripts whose weight is greater than or equal to 0, sort them in descending order, select the self-healing script with the largest weight as the optimal self-healing script, and increase its weight;

[0024] If no abnormality was found in the last inspection or the abnormal inspection indicators and monitoring objects obtained in the current inspection are different from those in the last inspection, the optimal self-healing script is directly selected in descending order of weight, and the weight of the optimal self-healing script is increased.

[0025] Preferably, when the initial weights of all self-healing scripts corresponding to the abnormal inspection indicators are the same, a self-healing script is randomly selected as the optimal self-healing script.

[0026] Preferably, if the latest weights of all self-healing scripts corresponding to the abnormal inspection indicators are negative, the console prompts that manual intervention is required.

[0027] Preferably, the console is further used to generate an inspection report, which includes abnormal inspection indicators and their monitoring objects, the selected optimal self-healing script, script execution results, manual intervention prompts or normal status prompts.

[0028] Preferably, the system further includes a database for storing all full monitoring data and self-healing scripts as well as an association between each inspection indicator and its corresponding self-healing script.

[0029] The present invention has the following beneficial effects:

[0030] Compared with traditional manual inspection methods, this application greatly reduces labor costs and improves inspection efficiency. It can quickly and large-scale inspect problems in the server and automatically recover from faults when system failures are found, ensuring the health and stability of customer servers. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0032] Figure 1 This is a structural architecture diagram of an automated inspection and self-healing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] To make the technical solution of the present application clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. The terms "first", "second", etc. in the claims and description of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate. This is merely a way of distinguishing objects of the same attributes when describing the embodiments of the present application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, so that a process, method, system, product, or device comprising a series of units is not necessarily limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0034] like Figure 1 As shown, this embodiment provides an automated inspection and self-healing system, which includes several agent modules, a console, a scheduler, and a decision maker.

[0035] Among them, several agent modules are installed on different target servers respectively, which are used to collect the full amount of monitoring data of the target servers in real time and execute the received optimal self-healing script.

[0036] In this embodiment, the automated inspection and self-healing system includes a console, a scheduler, a decision maker, and one or more agent modules. The agent modules are installed on the target servers, and there is a one-to-one correspondence between the agent modules and the target servers. That is, the number of agent modules means the number of target servers.

[0037] The Agent in this embodiment is custom-developed using the Go language. Upon completion, an installation package is generated. Users can download the package and then manually install it on the target server. Furthermore, the Agent also includes some program code to collect full monitoring data of the target server in real time. This full monitoring data includes, but is not limited to, system resource usage data, network performance indicators, application status data, and security log and event data. Specifically, the system resource usage data focuses on the target server's hardware resource usage, including CPU usage (usually expressed as a percentage), memory usage (total memory, used memory, and available memory), disk usage (total disk capacity, used capacity, available capacity, and read / write speed), and system load. Network performance indicators focus on the target server's network connection and transmission, including bandwidth, throughput, network latency, and data transmission rate (PPS). Application status data focuses on the status and performance of applications running on the target server, including application response time, throughput, error rate, and resource usage. Security log and event data focuses on the target server's security status and event records, including login logs, system logs, application logs, and security event logs.

[0038] In the Go language, the github.com / shirou / gopsutil package is usually used to collect the CPU usage and memory usage of the target server. The following example shows this.

[0039] / / Use the cpu.Percent(0,false) function to get the CPU usage.

[0040] cpuPercentages,err:=cpu.Percent(0,false)

[0041] fmt.Printf("CPU usage:%.2f%%\n",cpuPercentages[0])

[0042] / / Use the mem.VirtualMemory() function to obtain memory usage.

[0043] virtualMemory,err:=mem.VirtualMemory()

[0044] fmt.Printf("Total memory:%vMB\n",virtualMemory.Total / 1024 / 1024)

[0045] The agent is also responsible for executing the optimal self-healing script distributed by the decision maker and sending the results of success or failure to the console so that the console can generate an inspection report.

[0046] The console is used to configure inspection task parameters, including the inspection task name, execution cycle, inspection indicator set, monitoring objects and abnormal thresholds for each inspection indicator, and to establish an association between each inspection indicator and its self-healing script.

[0047] In this embodiment, the console mainly provides users with a simple and easy-to-use UI management interface, on which users can configure inspection task parameters. In this embodiment, the inspection task parameters include the inspection task name, execution cycle, inspection indicator set, and monitoring objects and abnormal thresholds for each inspection indicator. Among them, the inspection indicators include but are not limited to CPU utilization, memory utilization, disk utilization, network bandwidth utilization and network latency. The abnormal threshold defines the abnormal inspection indicator determination rule. For example, the abnormal threshold of CPU utilization is set to 90%, and the corresponding abnormal inspection indicator determination rule is that when the CPU utilization of the target server is greater than 90%, its CPU utilization is an abnormal inspection indicator, and the fault needs to be restored by executing the self-healing script. It should be noted here that the inspection task parameters and inspection indicators are not fixed and can be changed according to the actual needs of the user. For example, the alias, calculation method, grouping, display status and sorting field of each inspection indicator can be set. Among them, the alias refers to the Chinese meaning when the inspection indicator is expressed in English. The calculation method defines the method of determining the inspection indicator value, such as using the average value to represent the CPU usage rate. The grouping is to divide the inspection indicators according to different inspection tasks. After all, the inspection indicators of different inspection tasks can be different. The display status defines the number of inspection indicator values displayed on the console interface, and the sorting field defines the sorting rules for the displayed data.

[0048] At the same time, users can also establish an association between inspection indicators and their self-healing scripts on the console interface. A self-healing script is a script that can be automatically executed to repair or alleviate system problems when specific conditions are triggered. It is usually used in conjunction with a monitoring system. When the monitoring system detects an anomaly or failure, it triggers the self-healing script to perform the corresponding recovery operation. In this embodiment, each inspection indicator corresponds to multiple self-healing scripts. These self-healing scripts can be Python scripts or shell scripts, all of which are pre-deployed in the system database. When an abnormal inspection indicator is found, the system will only select the optimal self-healing script from the multiple self-healing scripts corresponding to the abnormal inspection indicator and hand it over to the Agent for execution.

[0049] In one example, the abnormal threshold of the target server's CPU usage is set to 90%. The working principle of one of the self-healing scripts is to check the current CPU usage. When the CPU usage exceeds 90%, it obtains the process ID that consumes the most CPU resources and restarts the process. The script code is as follows.

[0050] #! / bin / bash

[0051] # Check the current CPU usage

[0052] cpu_usage=$(top -bn1 | grep "Cpu(s)" | sed "s / .*, *\([0-9.]*\)%*id.* / \1 / " | awk '{print 100 - $1}')

[0053] threshold=90.0

[0054] if (( $(echo "$cpu_usage > $threshold" | bc -l) )); then

[0055] echo "CPU usage is above ${threshold}%: ${cpu_usage}%"

[0056] #Get the process ID that consumes the most CPU resources

[0057] high_cpu_pid=$(ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -n 2| tail -n 1 | awk '{print $1}')

[0058] echo "High CPU process ID: $high_cpu_pid"

[0059] #Restart the process

[0060] service apache2 restart

[0061] fi

[0062] In another example, the abnormal threshold for the target server's disk usage is set to 85%. The working principle of one of the self-healing scripts is to detect and record the current disk usage. When the disk usage exceeds 85%, it frees up disk space by finding, archiving, and cleaning temporary files or log files. The script code is as follows.

[0063] #! / bin / bash

[0064] # Check the current disk usage

[0065] disk_usage=$(df / | grep / | awk '{ print $5}' | sed 's / % / / g')

[0066] threshold=85

[0067] if [ "$disk_usage" -gt "$threshold" ]; then

[0068] echo "Disk usage is above ${threshold}%: ${disk_usage}%"

[0069] #Clean up the / tmp directory

[0070] rm -rf / tmp / *

[0071] #Find and clean up older log files

[0072] find / var / log -name "*.log" -type f -mtime +7 -exec gzip {} \;

[0073] fi

[0074] Users can also configure the initial weight of each self-healing script in the console interface. The initial weight of all self-healing scripts corresponding to the same inspection indicator can be set to the same or different. However, in this embodiment, the initial weight must be an integer greater than or equal to 0 to distinguish it from self-healing scripts with negative weights that are later excluded by the system. It should be noted that the range of initial weights and the weight of self-healing scripts excluded by the system can be adjusted according to actual user needs, as long as the system can easily distinguish between the two.

[0075] If the initial weights of the self-healing scripts corresponding to the same inspection metric differ, the following rule applies: sort the self-healing scripts by importance. The more important the self-healing script, the smaller the initial weight. Importance isn't the only criterion; it can also refer to priority, determined by user needs. For example, if a user considers disk files to be most important, followed by memory files, and there are two self-healing scripts, one for cleaning disk files and the other for cleaning memory files, the initial weight of the disk cleaning script would be set to 1, and the initial weight of the memory cleaning script would be set to 2.

[0076] Furthermore, the console is also used to generate inspection reports, which include abnormal inspection indicators and their monitoring objects, the selected optimal self-healing script, script execution results, manual intervention prompts or normal status prompts.

[0077] After each inspection is completed, an inspection report will be generated on the console interface. If the current inspection finds abnormal inspection indicators and the optimal self-healing script is selected, the inspection report should include the abnormal inspection indicators and their corresponding monitoring objects, the selected optimal self-healing script and the script execution result. The script execution result is success or failure. If the current inspection finds abnormal inspection indicators and the optimal self-healing script is not selected, the inspection report should include the abnormal inspection indicators and their corresponding monitoring objects and manual intervention prompts. If the current inspection does not find any abnormalities, the inspection report will show that everything is normal.

[0078] Among them, the scheduler is used to regularly trigger inspection tasks according to the inspection task parameters, and determine the inspection indicator values that exceed the abnormal threshold based on the full monitoring data in the current inspection cycle, so as to mark each abnormal inspection indicator and its monitoring object, and send the marking results to the decision maker.

[0079] In this embodiment, the scheduler periodically executes specific inspection tasks according to the inspection task parameters configured by the user. After the inspection task is started, the scheduler will query the database for the full monitoring data of the target server collected by the agent during the inspection cycle, and then extract the value corresponding to each inspection indicator according to the inspection indicator and monitoring object corresponding to the executed inspection task, and then compare all the extracted inspection indicator values with their corresponding abnormal thresholds. If a certain inspection indicator value is greater than its abnormal threshold, the inspection indicator is marked as an abnormal inspection indicator, and the abnormal inspection indicator and its monitoring object are marked. Finally, the inspection result, i.e. the marking result, is sent to the decision maker.

[0080] The decision maker is used to determine the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm, and distribute each optimal self-healing script to the corresponding agent module for execution.

[0081] In this embodiment, the decision maker is mainly used to select the optimal self-healing script from multiple self-healing scripts corresponding to abnormal inspection indicators and send it to the Agent for execution.

[0082] Furthermore, if the current inspection is the first inspection, all self-healing scripts associated with each abnormal inspection indicator are determined according to the association relationship, and are arranged in descending order according to the initial weight. The self-healing script with the largest weight is taken as the optimal self-healing script, and its weight is increased.

[0083] If the current inspection is the first inspection performed by the scheduler, all self-healing scripts corresponding to the abnormal inspection indicators are first extracted based on the correlation between the user-configured inspection indicators and the self-healing scripts. Then, all self-healing scripts are sorted from large to small according to their initial weights. The self-healing script with the largest weight is selected as the optimal self-healing script, and its initial weight is increased. In this embodiment, increasing the weight means adding 1 to its weight. Finally, this optimal self-healing script is sent to the corresponding agent for execution.

[0084] Furthermore, if the current inspection is not the first inspection, it is determined whether an abnormality occurred in the previous inspection;

[0085] If an abnormality occurred during the previous inspection, and the abnormal inspection indicators and monitoring objects obtained during the current inspection are the same as those during the previous inspection, query all self-healing scripts whose latest weight is greater than or equal to 0, sort them in descending order, set the latest weight of the self-healing script with the largest weight to a negative value, continue to query self-healing scripts whose weight is greater than or equal to 0, sort them in descending order, select the self-healing script with the largest weight as the optimal self-healing script, and increase its weight;

[0086] If no abnormality was found in the last inspection or the abnormal inspection indicators and monitoring objects obtained in the current inspection are different from those in the last inspection, the optimal self-healing script is directly selected in descending order of weight, and the weight of the optimal self-healing script is increased.

[0087] If the current inspection is not the first inspection performed by the scheduler, and the last inspection also had an exception, it is first determined whether the abnormal inspection indicator and its monitoring object obtained in this inspection are the same as the last one.

[0088] If they are the same, it means that the optimal self-healing script selected in the last inspection is invalid. First, obtain all the self-healing scripts with the latest weight greater than or equal to 0 corresponding to the abnormal inspection indicator, and arrange them in order of weight from large to small. Then set the weight of the self-healing script with the largest weight to a negative value. This self-healing script is the optimal self-healing script selected in the last inspection. Setting its weight to a negative value means that this self-healing script is invalid. In this embodiment, its weight is set to -1. Then, continue to query all the self-healing scripts with the latest weight greater than or equal to 0 corresponding to the abnormal inspection indicator, and arrange them in order of weight from large to small. Then select the self-healing script with the largest weight as the optimal self-healing script, add 1 to its weight, and finally send this optimal self-healing script to the corresponding Agent for execution.

[0089] If the current inspection is not the first inspection performed by the scheduler, and there was no abnormality in the last inspection, or the abnormal inspection indicators and monitoring objects obtained in the last inspection are different from those of this inspection, the optimal self-healing script is directly selected from the multiple self-healing scripts corresponding to the current abnormal inspection indicators in descending order of weight. The weight of the optimal self-healing script is then increased by 1, and the optimal self-healing script is distributed to the corresponding Agent for execution.

[0090] It should be noted here that regardless of whether the current inspection is the first inspection, as long as the decision maker does not select the optimal self-healing script, it will prompt through the console interface that manual intervention is required. Manual intervention can be manually re-editing the content of the self-healing script with a weight of -1, which is excluded by the system. After editing, its weight is set to an integer greater than or equal to 0 for use in the next system inspection; it can also be manually adding a self-healing script that has not been set before in the console; it can also be manually restoring the fault on the corresponding target server.

[0091] This self-healing system also includes a database, which is mainly used to store all the full monitoring data collected by the agent and all the self-healing scripts, as well as the association between each inspection indicator and its corresponding self-healing script, to facilitate data query by the scheduler and decision maker.

[0092] Compared with the traditional manual inspection method, this embodiment greatly reduces labor costs and improves inspection efficiency. It can quickly and large-scale inspect problems existing in the server and automatically recover from the fault when a system fault is found, ensuring the health and stability of the client server.

[0093] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. An automated inspection and self-healing system, characterized in that: It includes several agent modules, consoles, schedulers, and decision makers, including: The plurality of agent modules are respectively installed on different target servers, and are used to collect full monitoring data of the target servers in real time and execute the received optimal self-healing script; The console is used to configure inspection task parameters and set an initial weight for each self-healing script. The initial weights are all integers greater than or equal to 0. The inspection task parameters include the inspection task name, execution period, inspection indicator set, and monitoring objects and abnormal thresholds for each inspection indicator. The console also establishes an association between each inspection indicator and its self-healing script. The scheduler is configured to periodically trigger inspection tasks according to the inspection task parameters, and determine the inspection indicator values exceeding the abnormal threshold based on the full amount of monitoring data in the current inspection cycle, so as to mark each abnormal inspection indicator and its monitored object, and send the marking result to the decision maker; The decision maker is used to determine the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm, and distribute each optimal self-healing script to the corresponding agent module for execution; The method of determining the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm includes: If the current inspection is the first inspection, all self-healing scripts associated with each abnormal inspection indicator are determined based on the association relationship, and are arranged in descending order according to the initial weight. The self-healing script with the largest weight is selected as the optimal self-healing script, and its weight is increased; If the current inspection is not the first inspection, determine whether there was any abnormality in the previous inspection; If an abnormality occurred during the previous inspection, and the abnormal inspection indicators and monitoring objects obtained during the current inspection are the same as those during the previous inspection, query all self-healing scripts whose latest weight is greater than or equal to 0, sort them in descending order, set the latest weight of the self-healing script with the largest weight to a negative value, continue to query self-healing scripts whose weight is greater than or equal to 0, sort them in descending order, select the self-healing script with the largest weight as the optimal self-healing script, and increase its weight; If no abnormality was found in the last inspection or the abnormal inspection indicators and monitoring objects obtained in the current inspection are different from those in the last inspection, the optimal self-healing script is directly selected in descending order of weight, and the weight of the optimal self-healing script is increased.

2. The automatic inspection and self-healing system according to claim 1, characterized in that: The full monitoring data includes system resource usage data, network performance indicators, application status data, and security log and event data; The inspection indicators include CPU usage, memory usage, disk usage, network bandwidth usage and network latency.

3. The automatic inspection and self-healing system according to claim 1, characterized in that: The determining of the inspection indicator value exceeding the abnormal threshold based on the full amount of monitoring data in the current inspection cycle includes: Obtain the full monitoring data within the current inspection cycle, and extract the value corresponding to each inspection indicator from the current full monitoring data based on the inspection indicator and its monitoring object; The extracted inspection indicator values are compared with the corresponding abnormal thresholds, and all inspection indicators with values exceeding the abnormal thresholds are marked as abnormal inspection indicators.

4. The automatic inspection and self-healing system according to claim 1, characterized in that: When the initial weights of all self-healing scripts corresponding to the abnormal inspection indicators are the same, a self-healing script is randomly selected as the optimal self-healing script.

5. The automatic inspection and self-healing system according to claim 1, characterized in that: If the latest weights of all the self-healing scripts corresponding to the abnormal inspection indicators are negative, the console prompts that manual intervention is required.

6. The automatic inspection and self-healing system according to claim 5, characterized in that: The console is also used to generate an inspection report, which includes abnormal inspection indicators and their monitoring objects, the selected optimal self-healing script, script execution results, manual intervention prompts or normal status prompts.

7. The automated inspection and self-healing system according to claim 1, characterized in that: The system further includes a database for storing all full monitoring data and self-healing scripts as well as an association relationship between each inspection indicator and its corresponding self-healing script.

Images