Automatic inspection self-healing system

Abstract

The invention discloses an automatic inspection self-healing system, which belongs to the technical field of intelligent operation and maintenance, and comprises a plurality of agent modules which are respectively installed on different target servers, collect full monitoring data of the target servers in real time, and execute a received optimal self-healing script; the console is used for configuring inspection task parameters and establishing an association relationship between each inspection index and the self-healing script of the inspection index; the scheduler is used for periodically triggering the inspection task according to the inspection task parameters, marking each abnormal inspection index and a monitoring object thereof, and sending a marking result to the decision maker; and the decision maker is used for determining an optimal self-healing script corresponding to each abnormal inspection index according to a preset optimal selection algorithm, and distributing each optimal self-healing script to the corresponding agent module for execution. According to the method, the problems of the server can be quickly inspected on a large scale, and the fault can be automatically recovered when the system fault is found, so that the health and stability of the client server are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance technology, and in particular to an automated inspection and self-healing system. Background Art

[0002] As the company's business continues to expand and the number of customers continues to increase, the number of servers has also surged. This change has also made the server inspection tasks undertaken by the operation and maintenance team increasingly arduous. Traditional server inspection work mainly relies on manual operations, that is, inspectors log in to each server one by one to check and record the status of various key indicators. However, when faced with large-scale server clusters, this model exposes a series of significant problems.

[0003] First, the cost of manual inspection is high. As the number of servers increases, enterprises have to invest more human resources to complete inspection tasks, which directly leads to an increase in operation and maintenance costs. Second, manual inspection is inefficient. An inspector needs to undertake the inspection of dozens or even more servers every day. Each server contains a large number of performance indicators and operating status that need to be monitored. This high-intensity work is not only time-consuming and laborious, but also easily leads to fatigue of inspectors, thus affecting the accuracy and timeliness of inspections.

[0004] In addition, manual inspections also have problems with non-standard inspection content and non-standard inspection results. Different inspectors may perform inspection tasks based on their own experience and understanding, resulting in a lack of uniformity and standardization in inspection content, and inspection results are often difficult to compare and analyze horizontally. This not only reduces the quality and value of inspection data, but also brings great inconvenience to subsequent troubleshooting and performance optimization work.

[0005] Therefore, faced with the growing number of servers and increasingly complex operation and maintenance environments, the traditional manual inspection model can no longer meet the company's needs for efficient, accurate and standardized operation and maintenance inspections. Summary of the invention

[0006] The purpose of the present invention is to provide an automated inspection and self-healing system to solve the problems of high cost and low efficiency caused by manual inspection in the prior art, as well as the problems of non-standard inspection content and non-standard inspection results.

[0007] To achieve the above objectives, this application adopts the following technical solutions:

[0008] The present application discloses an automated inspection and self-healing system, including several agent modules, a console, a scheduler, and a decision maker, wherein:

[0009] The plurality of proxy modules are respectively installed on different target servers, and are used to collect full monitoring data of the target servers in real time, and execute the received optimal self-healing script;

[0010] The console is used to configure inspection task parameters, which include inspection task name, execution cycle, inspection indicator set, monitoring object and abnormal threshold of each inspection indicator, and establish an association relationship between each inspection indicator and its self-healing script;

[0011] The scheduler is used to periodically trigger the inspection task according to the inspection task parameters, and determine the inspection indicator value exceeding the abnormal threshold based on the full amount of monitoring data in the current inspection cycle, so as to mark each abnormal inspection indicator and its monitoring object, and send the marking result to the decision maker;

[0012] The decision maker is used to determine the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm, and distribute each optimal self-healing script to a corresponding proxy module for execution.

[0013] Preferably, the full monitoring data includes system resource usage data, network performance indicators, application status data, and security log and event data;

[0014] The inspection indicators include CPU usage, memory usage, disk usage, network bandwidth usage and network latency.

[0015] Preferably, the determining of the inspection index value exceeding the abnormal threshold based on the full amount of monitoring data in the current inspection cycle includes:

[0016] Obtain the full monitoring data within the current inspection cycle, and extract the value corresponding to each inspection indicator from the current full monitoring data according to the inspection indicator and its monitoring object;

[0017] The extracted inspection index values ​​are compared with the corresponding abnormal thresholds, and all inspection indexes whose values ​​exceed the abnormal thresholds are marked as abnormal inspection indexes.

[0018] Preferably, the console is further used to set an initial weight for each self-healing script, and the initial weight is an integer greater than or equal to 0.

[0019] Preferably, determining the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm includes:

[0020] If the current inspection is the first inspection, all self-healing scripts associated with each abnormal inspection indicator are determined according to the association relationship, and arranged in descending order according to the initial weight, and the self-healing script with the largest weight is used as the optimal self-healing script, and its weight is increased.

[0021] Preferably, determining the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm includes:

[0022] If the current inspection is not the first inspection, determine whether there was an abnormality in the previous inspection;

[0023] When an abnormality occurs in the last inspection, and the abnormal inspection indicators and monitoring objects obtained in the current inspection are the same as those in the last inspection, query all self-healing scripts whose latest weight is greater than or equal to 0, and arrange them in order from large to small. Set the latest weight of the self-healing script with the largest weight to a negative value, continue to query self-healing scripts with a weight greater than or equal to 0, and arrange them in order from large to small. Select the self-healing script with the largest weight as the optimal self-healing script and increase its weight.

[0024] When no abnormality occurred in the last inspection or the abnormal inspection indicators and monitoring objects obtained in the current inspection are different from those in the last inspection, the optimal self-healing script is directly selected in descending order of weight, and the weight of the optimal self-healing script is increased.

[0025] Preferably, when the initial weights of all self-healing scripts corresponding to the abnormal inspection indicators are the same, a self-healing script is randomly selected as the optimal self-healing script.

[0026] Preferably, if the latest weights of all self-healing scripts corresponding to the abnormal inspection indicators are negative values, the console prompts that manual intervention is required.

[0027] Preferably, the console is also used to generate an inspection report, which includes abnormal inspection indicators and their monitoring objects, the selected optimal self-healing script, script execution results, manual intervention prompts or normal status prompts.

[0028] Preferably, the system further comprises a database for storing all full monitoring data and self-healing scripts as well as an association relationship between each inspection indicator and its corresponding self-healing script.

[0029] The present invention has the following beneficial effects:

[0030] Compared with traditional manual inspection methods, this application greatly reduces labor costs and improves inspection efficiency. It can quickly and large-scale inspect server problems and automatically recover from faults when system failures are found, ensuring the health and stability of customer servers. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 This is a structural architecture diagram of an automated inspection and self-healing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] To make the technical solution of the present application clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The terms "first", "second", etc. in the claims and specification of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is only to describe the distinction method used when describing objects with the same attributes in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that a process, method, system, product or device containing a series of units is not necessarily limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] like Figure 1 As shown, this embodiment provides an automated inspection and self-healing system, which includes several agent modules, a console, a scheduler, and a decision maker.

[0035] Among them, several agent modules are installed on different target servers respectively, which are used to collect the full amount of monitoring data of the target servers in real time and execute the received optimal self-healing script.

[0036] In this embodiment, the automated inspection and self-healing system includes a console, a scheduler, a decision maker, and one or more agent modules, wherein the agent module is installed on the target server, and there is a one-to-one correspondence between the agent module and the target server, that is, the number of agent modules means the number of target servers.

[0037] The Agent in this embodiment is custom-developed based on the Go language. After the development is completed, an installation package is generated. When using it, the user first downloads the installation package and then manually installs it on the target server. At the same time, some program codes are written inside the Agent to collect the full monitoring data of the target server in real time. The full monitoring data includes but is not limited to system resource usage data, network performance indicators, application status data, and security log and event data. Specifically, the system resource usage data mainly focuses on the hardware resource usage of the target server, including CPU usage (usually expressed as a percentage), memory usage (total memory, used memory and available memory), disk usage (total disk capacity, used capacity, available capacity and read and write speed) and system load; the network performance indicators mainly focus on the network connection and transmission of the target server, including bandwidth, throughput, network delay and transmission rate (PPS); the application status data mainly focuses on the status and performance of the application running on the target server, including application response time, throughput, error rate and resource occupancy; the security log and event data mainly focus on the security status and event records of the target server, including login logs, system logs, application logs and security event logs.

[0038] In Go language, the github.com / shirou / gopsutil package is usually used to collect the CPU usage and memory usage of the target server. The example is as follows.

[0039] / / Use the cpu.Percent(0,false) function to get the CPU usage.

[0040] cpuPercentages,err:=cpu.Percent(0,false)

[0041] fmt.Printf("CPU usage:%.2f%%\n",cpuPercentages[0])

[0042] / / Use mem.VirtualMemory() function to get memory usage.

[0043] virtualMemory,err:=mem.VirtualMemory()

[0044] fmt.Printf("Total memory:%vMB\n",virtualMemory.Total / 1024 / 1024)

[0045] The agent is also responsible for executing the optimal self-healing script distributed by the decision maker and sending the results of success or failure to the console so that the console can generate an inspection report.

[0046] The console is used to configure inspection task parameters, including inspection task name, execution cycle, inspection indicator set, monitoring object and abnormal threshold of each inspection indicator, and establish the association between each inspection indicator and its self-healing script.

[0047] In this embodiment, the console mainly provides users with a simple and easy-to-use UI management interface, on which users can configure inspection task parameters. In this embodiment, the inspection task parameters include the inspection task name, execution cycle, inspection indicator set, and monitoring objects and abnormal thresholds of each inspection indicator. Among them, the inspection indicators include but are not limited to CPU utilization, memory utilization, disk utilization, network bandwidth utilization and network latency. The abnormal threshold defines the abnormal inspection indicator determination rule. For example, the abnormal threshold of CPU utilization is set to 90%, and the corresponding abnormal inspection indicator determination rule is that when the CPU utilization of the target server is greater than 90%, its CPU utilization is an abnormal inspection indicator, and the fault needs to be restored by executing the self-healing script. It should be noted here that the inspection task parameters and inspection indicators are not fixed and can be changed according to the actual needs of users. For example, the alias, calculation method, grouping, display status and sorting field of each inspection indicator can be set. Among them, the alias refers to the Chinese meaning when the inspection indicator is expressed in English. The calculation method defines the method of determining the inspection indicator value, such as using the average value to represent the CPU usage rate. The grouping is to divide the inspection indicators according to different inspection tasks. After all, different inspection tasks can have different inspection indicators. The display status defines the number of inspection indicator values ​​displayed on the console interface, and the sorting field defines the sorting rules for the displayed data.

[0048] At the same time, users can also establish an association between the inspection indicator and its self-healing script on the console interface. A self-healing script is a script that can be automatically executed to repair or alleviate system problems when specific conditions are triggered. It is usually used in conjunction with a monitoring system. When the monitoring system detects an abnormality or failure, it triggers the self-healing script to perform the corresponding recovery operation. In this embodiment, each inspection indicator corresponds to multiple self-healing scripts. These self-healing scripts can be python scripts or shell scripts. They are all pre-deployed in the system database. When an abnormal inspection indicator is found, the system will only select the optimal self-healing script from the multiple self-healing scripts corresponding to the abnormal inspection indicator and hand it over to the Agent for execution.

[0049] In one example, the abnormal threshold of the target server's CPU usage is set to 90%. The working principle of one of the self-healing scripts is to check the current CPU usage. When the CPU usage exceeds 90%, the process ID that consumes the most CPU resources is obtained and the process is restarted. The script code is as follows.

[0050] #! / bin / bash

[0051] # Check the current CPU usage

[0052] cpu_usage=$(top -bn1 | grep "Cpu(s)" | sed "s / .*, *\([0-9.]*\)%*id.* / \1 / " | awk '{print 100 - $1}')

[0053] threshold=90.0

[0054] if (( $(echo "$cpu_usage > $threshold" | bc -l) )); then

[0055] echo "CPU usage is above ${threshold}%: ${cpu_usage}%"

[0056] #Get the process ID that consumes the most CPU resources

[0057] high_cpu_pid=$(ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head-n 2 | tail -n 1 | awk '{print $1}')

[0058] echo "High CPU process ID: $high_cpu_pid"

[0059] #Restart the process

[0060] service apache2 restart

[0061] fi

[0062] In another example, the abnormal threshold of the target server disk usage is set to 85%. The working principle of one self-healing script is to detect and record the current disk usage. When the disk usage exceeds 85%, the disk space is released by finding, archiving and cleaning temporary files or log files. The script code is as follows.

[0063] #! / bin / bash

[0064] # Check the current disk usage

[0065] disk_usage=$(df / | grep / | awk '{ print $5}' | sed 's / % / / g')

[0066] threshold=85

[0067] if [ "$disk_usage" -gt "$threshold" ]; then

[0068] echo "Disk usage is above ${threshold}%: ${disk_usage}%"

[0069] #Clean up the / tmp directory

[0070] rm -rf / tmp / *

[0071] #Find and clean up older log files

[0072] find / var / log -name "*.log" -type f -mtime +7 -exec gzip {} \;

[0073] fi

[0074] The user can also configure the initial weight of each self-healing script in the console interface. The initial weights of all self-healing scripts corresponding to the same inspection indicator can be set to be the same or different, but in this embodiment, the initial weight must be an integer greater than or equal to 0 to distinguish it from the self-healing scripts with negative weights that are later excluded by the system. It should be noted here that the value range of the initial weight and the weight of the self-healing scripts excluded by the system can be adjusted according to the actual needs of the user, as long as the system can easily distinguish between the two.

[0075] At the same time, when the initial weights of the self-healing scripts corresponding to the same inspection indicator are different, the following rules can be followed: sort the self-healing scripts by importance. The more important the self-healing script is, the smaller the initial weight is. The importance here is not the only criterion, it can also be a priority, which is determined by user needs. For example, if the user thinks that the files on the disk are the most important, and the files in the memory are second, and there are two self-healing scripts, one for cleaning disk files and the other for cleaning memory files, then the initial weight of the self-healing script for cleaning disk files is set to 1, and the initial weight of the self-healing script for cleaning memory files is set to 2.

[0076] Furthermore, the console is also used to generate inspection reports, which include abnormal inspection indicators and their monitoring objects, the selected optimal self-healing script, script execution results, manual intervention prompts or normal status prompts.

[0077] After each inspection is completed, an inspection report will be generated on the console interface. If the current inspection finds abnormal inspection indicators and the optimal self-healing script is selected, the inspection report should include the abnormal inspection indicators and their corresponding monitoring objects, the selected optimal self-healing script and the script execution result. The script execution result is success or failure. If the current inspection finds abnormal inspection indicators and the optimal self-healing script is not selected, the inspection report should include the abnormal inspection indicators and their corresponding monitoring objects and manual intervention prompts. If the current inspection does not find any abnormalities, the inspection report will show that everything is normal.

[0078] Among them, the scheduler is used to periodically trigger the inspection task according to the inspection task parameters, and determine the inspection indicator value that exceeds the abnormal threshold based on the full monitoring data in the current inspection cycle, so as to mark each abnormal inspection indicator and its monitoring object, and send the marking result to the decision maker.

[0079] In this embodiment, the scheduler periodically executes a specific inspection task according to the inspection task parameters configured by the user. After the inspection task is started, the scheduler will query the database for the full monitoring data of the target server collected by the Agent during the inspection cycle, and then extract the value corresponding to each inspection indicator according to the inspection indicator and monitoring object corresponding to the executed inspection task, and then compare all the extracted inspection indicator values ​​with their corresponding abnormal thresholds. If a certain inspection indicator value is greater than its abnormal threshold, the inspection indicator is marked as an abnormal inspection indicator, and the abnormal inspection indicator and its monitoring object are marked. Finally, the inspection result, i.e. the marking result, is sent to the decision maker.

[0080] The decision maker is used to determine the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm, and distribute each optimal self-healing script to the corresponding proxy module for execution.

[0081] In this embodiment, the decision maker is mainly used to select the best self-healing script from multiple self-healing scripts corresponding to the abnormal inspection indicators, and send it to the Agent for execution.

[0082] Furthermore, if the current inspection is the first inspection, all self-healing scripts associated with each abnormal inspection indicator are determined according to the association relationship, and are arranged in descending order according to the initial weight. The self-healing script with the largest weight is used as the optimal self-healing script, and its weight is increased.

[0083] If the current inspection is the first inspection performed by the scheduler, first extract all self-healing scripts corresponding to the abnormal inspection indicators according to the correlation between the inspection indicators configured by the user and the self-healing scripts, then arrange all the self-healing scripts from large to small according to the initial weights of their respective self-healing scripts, select the self-healing script with the largest weight as the optimal self-healing script, and increase its initial weight. In this embodiment, increasing the weight means adding 1 to its weight, and finally sending this optimal self-healing script to the corresponding Agent for execution.

[0084] Furthermore, if the current inspection is not the first inspection, it is determined whether an abnormality occurred in the previous inspection;

[0085] When an abnormality occurs in the last inspection, and the abnormal inspection indicators and monitoring objects obtained in the current inspection are the same as those in the last inspection, query all self-healing scripts whose latest weight is greater than or equal to 0, and arrange them in order from large to small. Set the latest weight of the self-healing script with the largest weight to a negative value, continue to query self-healing scripts with a weight greater than or equal to 0, and arrange them in order from large to small. Select the self-healing script with the largest weight as the optimal self-healing script and increase its weight.

[0086] When no abnormality occurred in the last inspection or the abnormal inspection indicators and monitoring objects obtained in the current inspection are different from those in the last inspection, the optimal self-healing script is directly selected in descending order of weight, and the weight of the optimal self-healing script is increased.

[0087] If the current inspection is not the first inspection performed by the scheduler, and the last inspection also had an abnormality, first determine whether the abnormal inspection indicator and its monitoring object obtained in this inspection are the same as the last one.

[0088] If they are the same, it means that the optimal self-healing script selected in the last inspection is invalid. First, obtain all the self-healing scripts with the latest weight greater than or equal to 0 corresponding to the abnormal inspection indicator, and arrange them in order from large to small weights. Then set the weight of the self-healing script with the largest weight to a negative value. This self-healing script is the optimal self-healing script selected in the last inspection. Setting its weight to a negative value means that this self-healing script is invalid. In this embodiment, its weight is set to -1. Then, continue to query all the self-healing scripts with the latest weight greater than or equal to 0 corresponding to the abnormal inspection indicator, and arrange them in order from large to small weights. Then select the self-healing script with the largest weight as the optimal self-healing script, add 1 to its weight, and finally send this optimal self-healing script to the corresponding Agent for execution.

[0089] If the current inspection is not the first inspection performed by the scheduler, and there was no abnormality in the last inspection, or the abnormal inspection indicators and monitoring objects obtained in the last inspection are different from this time, the optimal self-healing script is directly selected from the multiple self-healing scripts corresponding to the current abnormal inspection indicators in order from large to small weights, and then the weight of the optimal self-healing script is increased by 1, and the optimal self-healing script is distributed to the corresponding Agent for execution.

[0090] It should be noted here that no matter whether the current inspection is the first inspection or not, as long as the decision maker does not select the optimal self-healing script, it will prompt through the console interface that manual intervention is required. Manual intervention can be manually re-editing the content of the self-healing script with a weight of -1, which is excluded by the system. After editing, its weight is set to an integer greater than or equal to 0 for use in the next system inspection; it can also be manually adding a self-healing script that was not set before in the console; it can also be manually restoring the fault on the corresponding target server.

[0091] The self-healing system also includes a database, which is mainly used to store all the full monitoring data collected by the Agent and all the self-healing scripts, as well as the association between each inspection indicator and its corresponding self-healing script, so as to facilitate the scheduler and decision maker to perform data query.

[0092] Compared with the traditional manual inspection method, this embodiment greatly reduces labor costs and improves inspection efficiency. It can quickly and large-scale inspect problems existing in the server and automatically recover from faults when system faults are found, thus ensuring the health and stability of the client server.

[0093] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An automated inspection and self-healing system, characterized in that: It includes several agent modules, consoles, schedulers and decision makers, among which: The plurality of proxy modules are respectively installed on different target servers, and are used to collect full monitoring data of the target servers in real time, and execute the received optimal self-healing script; The console is used to configure inspection task parameters, which include inspection task name, execution cycle, inspection indicator set, monitoring object and abnormal threshold of each inspection indicator, and establish an association relationship between each inspection indicator and its self-healing script; The scheduler is used to periodically trigger the inspection task according to the inspection task parameters, and determine the inspection indicator value exceeding the abnormal threshold based on the full amount of monitoring data in the current inspection cycle, so as to mark each abnormal inspection indicator and its monitoring object, and send the marking result to the decision maker; The decision maker is used to determine the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm, and distribute each optimal self-healing script to a corresponding proxy module for execution.

2. The automatic inspection and self-healing system according to claim 1, characterized in that: The full monitoring data includes system resource usage data, network performance indicators, application status data, and security log and event data; The inspection indicators include CPU usage, memory usage, disk usage, network bandwidth usage and network latency.

3. The automatic inspection and self-healing system according to claim 1, characterized in that: The determining of the inspection index value exceeding the abnormal threshold based on the full amount of monitoring data in the current inspection cycle includes: Obtain the full monitoring data within the current inspection cycle, and extract the value corresponding to each inspection indicator from the current full monitoring data according to the inspection indicator and its monitoring object; The extracted inspection index values ​​are compared with the corresponding abnormal thresholds, and all inspection indexes whose values ​​exceed the abnormal thresholds are marked as abnormal inspection indexes.

4. The automatic inspection and self-healing system according to claim 1, characterized in that: The console is also used to set an initial weight for each self-healing script, and the initial weight is an integer greater than or equal to 0.

5. The automatic inspection and self-healing system according to claim 4, characterized in that: Determining the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm includes: If the current inspection is the first inspection, all self-healing scripts associated with each abnormal inspection indicator are determined according to the association relationship, and arranged in descending order according to the initial weight, and the self-healing script with the largest weight is used as the optimal self-healing script, and its weight is increased.

6. The automatic inspection and self-healing system according to claim 5, characterized in that: Determining the optimal self-healing script corresponding to each abnormal inspection indicator according to a preset optimal selection algorithm includes: If the current inspection is not the first inspection, determine whether there was an abnormality in the previous inspection; When an abnormality occurs in the last inspection, and the abnormal inspection indicators and monitoring objects obtained in the current inspection are the same as those in the last inspection, query all self-healing scripts whose latest weight is greater than or equal to 0, and arrange them in order from large to small. Set the latest weight of the self-healing script with the largest weight to a negative value, continue to query self-healing scripts with a weight greater than or equal to 0, and arrange them in order from large to small. Select the self-healing script with the largest weight as the optimal self-healing script and increase its weight. When no abnormality occurred in the last inspection or the abnormal inspection indicators and monitoring objects obtained in the current inspection are different from those in the last inspection, the optimal self-healing script is directly selected in descending order of weight, and the weight of the optimal self-healing script is increased.

7. The automatic inspection and self-healing system according to claim 6, characterized in that: When the initial weights of all self-healing scripts corresponding to the abnormal inspection indicators are the same, a self-healing script is randomly selected as the optimal self-healing script.

8. The automatic inspection and self-healing system according to claim 7, characterized in that: If the latest weights of all self-healing scripts corresponding to the abnormal inspection indicators are negative values, the console prompts that manual intervention is required.

9. The automatic inspection and self-healing system according to claim 8, characterized in that: The console is also used to generate an inspection report, which includes abnormal inspection indicators and their monitoring objects, the selected optimal self-healing script, script execution results, manual intervention prompts or normal status prompts.

10. The automatic inspection and self-healing system according to claim 1, characterized in that: The system also includes a database, which is used to store all full monitoring data and self-healing scripts and the association relationship between each inspection indicator and its corresponding self-healing script.

Images