A cloud computing-based computer fault detection method and system

Abstract

The application discloses a kind of based on cloud computing's electronic computer fault detection method and system, including the virtual environment of constructing include diversified fault scene, and carry out prediction and obtain the expected running data corresponding to performance index in virtual environment, obtain the real-time running data of system, the comprehensive deviation value of expected running data and real-time running data is calculated in combination with scene weighting factor and time attenuation coefficient, and according to comprehensive deviation value, the influence grade of fault scene to service continuity is combined to calculate priority score, and generate the trigger task queue sorted by priority score;The fault detection task corresponding to the highest priority score of the fault scene is simulated by virtual environment, and simulation running data is obtained;Pretrained fault classification model is used to match the fault of simulation running data, and output fault type and fault positioning information.The application can realize accurate detection to electronic computer fault.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a computer fault detection method and system based on cloud computing. BACKGROUND

[0002] Computer system reliability research is a core direction in the field of modern information technology, and is directly related to the stable operation of critical infrastructure and data security. With the increase in the complexity of computer systems, fault detection and diagnosis has become a key link to ensure system high availability.

[0003] In one prior art, fault detection is mainly performed using power-on self-test and hardware diagnostic software, error and warning information in system logs is viewed through system-provided event viewer or log analysis tools, network connection status of the computer is checked using network diagnostic tools, and function tests are performed on the operating system and application programs to check for software faults.

[0004] However, some diagnostic software in the prior art needs to run in a specific environment, and for some complex hardware faults or intermittent faults, it may not be able to accurately detect or locate, resulting in low accuracy of computer fault detection. SUMMARY

[0005] The present application provides a computer fault detection method and system based on cloud computing to solve the problem of low accuracy of computer fault detection in the prior art.

[0006] In a first aspect, to solve the above technical problems, the present application provides a computer fault detection method based on cloud computing, comprising:

[0007] A virtual environment containing diverse fault scenarios is constructed by virtualization technology, system running logs and hardware performance indicators are obtained, and expected running data corresponding to the performance indicators in the virtual environment is obtained by prediction;

[0008] Real-time running data of the system is obtained, and a comprehensive deviation value is obtained by calculation in combination with a scenario weighting factor, a time decay coefficient, and the expected running data;

[0009] A priority score is calculated according to the magnitude by which the comprehensive deviation value exceeds a preset deviation threshold and the influence level of the fault scenario on business continuity, and a trigger task queue sorted by priority score is generated according to the priority score; the trigger task queue is a set of fault detection tasks containing trigger conditions, priority scores, and target fault scenarios;

[0010] The fault detection task with the highest priority is extracted from the trigger task queue, and the corresponding fault scenario simulation is performed in the virtual environment to obtain simulation running data of the fault scenario.

[0011] The pre-trained fault classification model is used for fault mode matching on the simulation running data, and fault type and fault positioning information are output.

[0012] Preferably, the virtual environment containing diversified fault scenarios is constructed by a virtualization technology, system running logs and hardware performance indicators are obtained, and expected running data corresponding to the performance indicators in the virtual environment are obtained by prediction, comprising:

[0013] According to the virtualization resources of the cloud computing platform, a containerization technology is used to deploy simulation nodes, and a multi-level hardware structure and a heterogeneous operating system are configured;

[0014] Hardware and software fault scenarios are injected and coupled by a tool and a fault injection framework to obtain a virtual environment containing diversified fault scenarios;

[0015] The running logs of each node in the virtual environment are obtained by a log collection tool, and the hardware performance indicators are collected in real time by a monitoring tool;

[0016] According to the node running logs and the hardware performance indicators, key state features are extracted by a feature extraction algorithm to obtain a state feature set;

[0017] According to the state feature set, structured features are processed by XGBoost to predict the expected running state of each node in the virtual environment, and expected running data are generated.

[0018] Preferably, the real-time running data of the system are obtained, combined with a scenario weighting factor, a time decay coefficient, and the expected running data to calculate a comprehensive deviation value, comprising:

[0019] According to the deployed containerized nodes, the running logs containing timestamps are extracted by a log collection tool to obtain the real-time running data of the system;

[0020] According to the influence of the fault scenarios on the business, an initial weight is allocated and the weight is dynamically updated based on real-time business load or external events to obtain a scenario weighting factor;

[0021] The time interval from the occurrence of the event to the current calculation is obtained, and the time decay coefficient is calculated according to the time interval and a preset decay rate;

[0022] According to the real-time running data, the expected running data, the scenario weighting factor, and the time decay coefficient, a comprehensive deviation value is calculated.

[0023] Preferably, the calculation formula of the time decay coefficient is:

[0024] β=e -λt

[0025] wherein β is a time decay coefficient, λ is a decay rate, and S is a unit of time -1 , t is a time interval from the event occurrence to the current calculation, and S is a unit of time;

[0026] The calculation formula of the comprehensive deviation value is:

[0027]

[0028] wherein Δ is a comprehensive deviation value; w is a scene weighting factor; β is a time decay coefficient; Now is a real-time running value; Pre is an expected running value; and Now and Pre are both values between 0 and 1.

[0029] Preferably, the priority score is calculated according to the magnitude of the comprehensive deviation value exceeding a preset deviation threshold and the influence level of the fault scene on service continuity, and a trigger task queue sorted by the priority score is generated according to the priority score, including:

[0030] The deviation magnitude is obtained by calculating according to the comprehensive deviation value and in combination with a preset deviation threshold;

[0031] The influence level of different fault scenes on service continuity is divided to obtain a weight score corresponding to different levels;

[0032] The priority score of different fault scenes is calculated according to the deviation magnitude and the weight score;

[0033] A fault detection task containing a trigger condition, a priority, and a target fault scene is generated, all fault detection tasks are sorted according to the priority score, and a trigger task queue is obtained.

[0034] Preferably, the fault detection task with the highest priority is extracted from the trigger task queue, and a corresponding fault scene simulation is performed in a virtual environment to obtain simulation running data of the fault scene, including:

[0035] The fault detection task with the highest priority score is obtained from the trigger task queue to obtain a fault scene to be simulated;

[0036] According to the fault scene to be simulated, corresponding running parameters are configured and loaded in a virtual environment to obtain a virtual environment of the fault scene to be simulated;

[0037] The fault scene to be simulated is simulated in the virtual environment, and a monitoring tool is used to collect running logs and hardware performance indicators of the containerized node in real time to obtain simulation running data;

[0038] After the simulation is completed, the virtual environment is reset for the next fault simulation.

[0039] Preferably, the pre-trained fault classification model is used to match the fault mode of the simulation operation data, output the fault type and fault positioning information, including:

[0040] According to the simulation operation data, a log collection tool is used to extract the operation log of the containerized node in the simulation operation data, and a log analysis tool is used for structured processing to obtain structured simulation operation data;

[0041] The pre-trained fault classification model is used to match the structured simulation operation data to obtain a fault mode matching result;

[0042] According to the fault mode matching result, a rule engine is used to classify the matched fault mode, and a preset fault type mapping table is used to output the corresponding fault type;

[0043] The node identifier of the containerized node in the simulation operation data is extracted to obtain fault positioning information containing the node name and IP address.

[0044] Preferably, the training process of the pre-trained fault classification model includes:

[0045] Obtain system historical fault data, diversified fault data coupled with virtual environment and open source fault data as pre-training data of the model;

[0046] Label the corresponding fault type and positioning information in the different fault data in the pre-training data;

[0047] According to the pre-training data, combined with the fault type and the positioning information, a double-channel deep learning architecture is used to train the model;

[0048] Calculate the accuracy, recall rate and F1 score indicators as evaluation results;

[0049] According to the evaluation results, the model is adjusted by using a regularization method to obtain a trained fault classification model.

[0050] Preferably, the accuracy, recall rate and F1 score indicators are calculated by the following formula:

[0051] The calculation formula of the accuracy Accuracy is:

[0052] Accuracy=(TP+TN) / (TP+TN+FP+FN)

[0053] The calculation formula of the recall rate Recall is:

[0054] Recall=TP / (TP+FN)

[0055] The calculation formula of the F1 score F1-Score is:

[0056]

[0057] Wherein, TP is a true example, TN is a true negative example, FP is a false positive example, FN is a false negative example, F1-Score is the harmonic mean of precision and recall, and precision Precison=TP / (TP+FP).

[0058] In a second aspect, the present application provides a cloud computing-based electronic computer fault detection system, comprising:

[0059] A virtual environment construction module is configured to construct a virtual environment containing diversified fault scenarios through a virtualization technology, acquire system running logs and hardware performance indicators, and obtain expected running data corresponding to the performance indicators in the virtual environment through prediction.

[0060] A running deviation calculation module is configured to acquire real-time running data of the system, combine a scenario weighting factor, a time attenuation coefficient, and the expected running data to perform calculation, and obtain a comprehensive deviation value.

[0061] A trigger task generation module is configured to calculate a priority score according to a magnitude at which the comprehensive deviation value exceeds a preset deviation threshold and an influence level of a fault scenario on business continuity, and generate a trigger task queue sorted according to the priority score according to the priority score; the trigger task queue is a collection of fault detection tasks containing a trigger condition, a priority score, and a target fault scenario.

[0062] A fault scenario simulation module is configured to extract a fault detection task with the highest priority from the trigger task queue, and perform simulation of a corresponding fault scenario in the virtual environment to acquire simulation running data of the fault scenario.

[0063] A matching result output module is configured to perform fault mode matching on the simulation running data by using a pre-trained fault classification model, and output fault type and fault positioning information.

[0064] In a third aspect, the present application further provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the cloud computing-based electronic computer fault detection method according to any one of the above aspects when executing the computer program.

[0065] In a fourth aspect, the present application further provides a computer readable storage medium comprising a stored computer program, wherein the computer readable storage medium controls a device in which the computer readable storage medium is located to execute the cloud computing-based electronic computer fault detection method according to any one of the preceding aspects when the computer program is executed.

[0066] Compared with the prior art, the present application has the following beneficial effects:

[0067] 1. The present application combines cloud computing and virtualization technology to build a virtual environment containing various fault scenarios, simulates the expected running value of each node indicator data of the computer by combining containerized nodes and XGBoost prediction analysis, calculates the comprehensive deviation value of each node indicator data by combining real-time running value, marks the node corresponding to the indicator data whose comprehensive deviation value exceeds the preset deviation threshold as a potential fault node, generates a fault detection task and simulates it in the virtual environment, and according to the simulation running data and in combination with the fault classification model, the fault type can be more accurately matched and the fault location can be located, thereby improving the accuracy of fault detection.

[0068] 2. The virtual environment of the present application contains various fault scenarios, can fully simulate the fault conditions that may occur during system operation, can quickly and accurately locate the fault node according to the monitoring running data of each containerized node deployed, and improves the efficiency of fault detection.

[0069] 3. The present application quickly and accurately detects the fault type and fault location information of the electronic computer, ensures that the system can be quickly detected after the fault occurs, then gets timely and effective treatment, and recovers to the normal operation state as soon as possible, reduces the system downtime and data loss risk caused by the fault, guarantees the stable operation of the key infrastructure and the safety of the data, and improves the overall reliability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 A flowchart of a cloud computing-based electronic computer fault detection method of the present application.

[0071] Figure 2 A module schematic diagram of a cloud computing-based electronic computer fault detection system of the present application. DETAILED DESCRIPTION

[0072] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0073] With reference to Figure 1 The first embodiment of the present application provides a cloud computing-based electronic computer fault detection method flowchart, comprising the following steps:

[0074] S11, a virtual environment containing diversified fault scenarios is constructed through a virtualization technology, system running logs and hardware performance indicators are obtained, and expected running data corresponding to the performance indicators in the virtual environment is obtained through prediction;

[0075] S12, real-time running data of the system is obtained, and a comprehensive deviation value is obtained through calculation in combination with a scenario weighting factor, a time attenuation coefficient and the expected running data;

[0076] S13, a priority score is calculated according to the amplitude at which the comprehensive deviation value exceeds a preset deviation threshold and the influence level of the fault scenario on business continuity, and a trigger task queue sorted according to the priority score is generated according to the priority score; the trigger task queue is a set of fault detection tasks containing a trigger condition, a priority score and a target fault scenario;

[0077] S14, the fault detection task with the highest priority is extracted from the trigger task queue, and a corresponding fault scenario simulation is performed in the virtual environment to obtain simulation running data of the fault scenario;

[0078] S15, a pre-trained fault classification model is used to perform fault mode matching on the simulation running data, and fault type and fault positioning information are output.

[0079] In step S11, a virtual environment containing diversified fault scenarios is constructed through a virtualization technology, system running logs and hardware performance indicators are obtained, and expected running data corresponding to the performance indicators in the virtual environment is obtained through prediction, comprising:

[0080] According to the virtualization resources of the cloud computing platform, a containerization technology is used to deploy simulation nodes, and a multi-level hardware structure and a heterogeneous operating system are configured;

[0081] Hardware and software fault scenarios are injected and coupled through a tool and a fault injection framework to obtain a virtual environment containing diversified fault scenarios;

[0082] Obtain the running logs of each node in the virtual environment through a log collection tool, and use a monitoring tool to collect hardware performance indicators in real time;

[0083] According to the node running log and the hardware performance indicator, a key state feature is extracted through a feature extraction algorithm to obtain a state feature set;

[0084] According to the state feature set, structured features are processed through XGBoost to predict the expected running state of each node in the virtual environment, and expected running data is generated.

[0085] By constructing a virtual environment, when subsequent nodes deviate from running, fault simulation can be performed to obtain simulation running data, so that the fault classification model can match the fault type without occupying computer resources during the fault.

[0086] Among them, the cloud computing platform can be selected based on mainstream cloud services such as AWS EKS, Aliyun ACK, AzureKubernetes Service or private cloud platforms such as OpenStack; the virtualization resources can be virtual machines, networks, storage volumes, etc. The application of the containerization technology can be divided into container orchestration, using Kubernetes K8s to manage containerized simulation nodes, defining Pod, Deployment, Service and other resources through declarative YAML files or lightweight containers, using Docker or containerd runtime, and deploying container images of simulation nodes such as Ubuntu, CentOS, Alpine, etc.

[0087] The multi-level hardware limits container resources such as CPU or memory through Kubernetes' Resource Quotas and Limit Ranges to simulate different hardware specifications; the heterogeneous operating systems can use QEMU or Firecracker micro-virtual machines to be nested in containers to simulate specific hardware architectures, such as ARM nodes coexisting with x86 nodes.

[0088] The hardware and software fault scenarios are injected through tools and a fault injection framework and are coupled, wherein the software fault can use Chaos Mesh or Litmus to inject Pod-level faults such as process termination and network delay in the K8s cluster; the hardware fault can limit CPU or memory through cgroups or use a fault-injection kernel module to simulate disk IO errors; wherein, fault coupling can be achieved through scripting such as Python or Ansible to coordinate multiple fault injection tools to realize "cascading faults", for example, while triggering network packet loss, limit the CPU quota of a certain container.

[0089] The acquisition of the running logs of each node in the virtual environment by the log collection tool and the real-time collection of the hardware performance indicators by the monitoring tool can be achieved by the following steps: collecting operating system logs and application logs such as Nginx error logs and middleware logs such as MySQL slow query logs by a log aggregation tool such as Fluentd and Logstash; and deploying a monitoring agent such as Prometheus Node Exporter to collect indicators such as CPU usage, memory occupation, disk I / O, and network throughput in real time.

[0090] The extraction of key state features by the feature extraction algorithm to obtain the state feature set can be achieved by the following steps:

[0091] The collected node running logs are cleaned to remove meaningless characters, blank lines, and repeated log records; for example, using a regular expression to match and filter out garbled codes and irrelevant symbols in the logs, and converting the logs into a standard text format for subsequent processing; the hardware performance indicator data is subjected to missing value filling and smoothing processing; for example, for occasionally missing CPU usage data points, the average value of the adjacent data points can be used for filling; for memory occupation data with fluctuations, a moving average method is used for smoothing processing to reduce the influence of data noise.

[0092] For log data, a text feature extraction algorithm can be used; for example, using the Bag of Words model to convert log text into a word frequency vector, and by counting the frequency of occurrence of each keyword such as "error"

[0093] “warning” “start” “stop” and the like, a high-dimensional word frequency vector space is formed, and each dimension corresponds to the number of occurrences of a keyword.

[0094] For hardware performance indicator data, a statistical feature extraction method can be used; for example, calculating the mean, variance, maximum, and minimum of CPU usage; for memory occupation, extracting the average value and growth rate of the occupation rate; for network performance indicators, extracting the bandwidth utilization rate, packet loss rate, and delay.

[0095] The features extracted from the log data and the hardware performance indicator data are fused to form a comprehensive state feature set; for example, the word frequency vector of the log and the statistical features of the hardware performance indicators are spliced to form a multi-dimensional feature vector containing text features and numerical features. Dimensionality reduction techniques such as principal component analysis (PCA) can be used to reduce the dimensionality of the fused features to remove redundant features and reduce computational complexity.

[0096] The generating of the expected running data is to learn the input features by XGBoost to build multiple trees, to predict the state feature set of each node in the current virtual environment, to obtain the expected running state of each node, and to generate the corresponding expected running data according to the expected running state. For example, for the expected running data of the normal state, the system running log and the hardware performance index data conforming to the distribution of the historical normal running can be generated based on the data distribution; for the expected running data of the fault state, the corresponding data mode under the historical fault scene can be referred to for generation.

[0097] In step S12, real-time running data of the system is acquired, and a comprehensive deviation value is obtained by calculation in combination with a scene weighting factor, a time decay coefficient and the expected running data, including:

[0098] According to the deployed containerized node, a log collection tool is combined to extract running logs containing timestamps, and real-time running data of the system is acquired;

[0099] According to the influence of the fault scene on the business, an initial weight is allocated and the weight is dynamically updated based on real-time business load or external events to obtain a scene weighting factor;

[0100] A time interval from the occurrence of the fault to the current calculation is acquired, and a time decay coefficient is calculated according to the time interval and a preset decay rate;

[0101] According to the real-time running data, the expected running data, the scene weighting factor and the time decay coefficient, a comprehensive deviation value is calculated.

[0102] By calculating the comprehensive deviation value of the index data of each node, a fault detection task is generated for the node exceeding the preset deviation threshold, simulation running data is obtained after simulation in the virtual environment, and the fault type is matched according to the fault classification model, so that the node where the fault occurs can be quickly obtained, the fault positioning information is obtained, and the fault detection accuracy is improved.

[0103] The real-time running data of the system can be acquired by the following steps: Filebeat is used as a log collection tool, a configuration file of Filebeat is configured inside the container, and a log file path to be monitored and a log output format are specified; the log collection tool reads log files generated by each application program or system service in real time during the running of the container; the log collection tool transmits the collected running logs with timestamps to a centralized log processing system or a data storage platform, and performs preliminary formatting and arrangement on the logs to ensure the consistency and integrity of the data; the running data of the system is acquired in real time from the centralized log processing system or the data storage platform.

[0104] For example, the access log of the Web server contains the timestamp of the client request in each record, similar to the format "[2024-10-01 10:00:00] Received request from client IP 192.168.1.100"; the log of the application may also record a timestamped log entry such as "[2024-10-01 10:05:30] Application started"; Filebeat transmits the collected logs to the Logstash data processing pipeline tool over the network, parses, filters and converts the logs in Logstash, extracts key information, and stores it uniformly in the Elasticsearc distributed search engine and analysis engine; by querying the latest log data in Elasticsearch, the real-time CPU usage, memory occupation, network request response time and other performance indicators of each containerized node can be obtained, as well as the state information of whether the application has abnormal or error.

[0105] The implementation of the scenario weighting factor according to the impact of the fault scenario on the business, assigning an initial weight and dynamically updating the weight based on real-time business load or external events is as follows: determine the weight definition, the business impact level is determined according to the impact of the fault scenario on the business, such as the weight of 0.9 corresponding to the core service downtime, and the weight of 0.3 corresponding to the edge service delay to assign the initial weight; dynamically adjust the rules based on real-time business load, such as weight increase during peak hours; or external events, such as increasing the weight of related scenarios when a security vulnerability occurs.

[0106] It should be noted that the decay rate λ needs to be adjusted according to different faults. For high-frequency abnormalities, such as multiple disk IO errors in a short period of time, increase the decay rate to quickly reduce the impact of old abnormalities; for low-frequency critical events, such as database master node downtime, slow down the decay rate to extend its impact time.

[0107] For example, the last 5 minutes of data are retained, and the data outside the time window is zeroed, and if a node has CPU overload for 3 times in 10 seconds, λ is adjusted from 0.1 to 0.3 to accelerate the decay of old data.

[0108] The time interval from the occurrence of the fault to the current calculation can be obtained by the system timer.

[0109] It should be noted that the calculation formula of the time decay coefficient is:

[0110] β=e -λt

[0111] Where β is the time decay coefficient, and λ is the decay rate, with the unit of S-1 t is the time interval from the event occurrence to the current calculation, in seconds;

[0112] The formula for calculating the comprehensive deviation value is:

[0113]

[0114] where Δ is the comprehensive deviation value; w is the scene weighting factor; β is the time decay coefficient; Now is the real-time running value; Pre is the expected running value; and both Now and Pre are values between 0 and 1, expressed as percentages.

[0115] For example, the real-time CPU usage is 98%, the expected value is 70%, the weight is 0.9, the decay coefficient is 0.8, the CPU deviation is calculated to be 0.288; the real-time disk IOPS is 50, the expected value is 200, the weight is 0.5, the decay coefficient is 0.6: the disk deviation is calculated to be 0.225, and the total deviation value is 0.513. According to the comprehensive deviation value result, the weight and decay coefficient can be adjusted inversely, for example, if a certain type of fault occurs frequently, the weight is increased, and reinforcement learning such as Q-Learning is used to automatically optimize the adjustment strategy.

[0116] In step S13, a priority score is calculated according to the magnitude by which the comprehensive deviation value exceeds a preset deviation threshold and the impact level of the fault scene on business continuity, and a trigger task queue sorted by priority score is generated according to the priority score; the trigger task queue is a collection of fault detection tasks containing trigger conditions, priority scores, and target fault scenes, including:

[0117] According to the comprehensive deviation value, and in combination with a preset deviation threshold, a deviation magnitude is obtained;

[0118] The impact levels of different fault scenes on business continuity are divided to obtain weight scores corresponding to different levels;

[0119] According to the deviation magnitude and the weight scores, priority scores of different fault scenes are calculated;

[0120] Fault detection tasks containing trigger conditions, priority scores, and fault scenes are generated, all fault detection tasks are sorted according to the priority scores, and a trigger task queue is obtained.

[0121] By calculating the priority score, it can be ensured that the most urgent and most impactful faults are processed by the system resources, and corresponding fault detection tasks containing trigger conditions, priority scores, and target fault scenes are generated, so that simulation running data are obtained for fault matching.

[0122] The magnitude of the comprehensive deviation value exceeding the preset deviation threshold can be obtained by converting the difference between the comprehensive deviation value and the preset deviation threshold by the preset deviation threshold into a percentage. For example, the preset deviation threshold is set to 0.5, and the actual deviation value is 0.8, and the deviation magnitude is calculated to be 60%. The node corresponding to the index data of the comprehensive deviation value exceeding the preset deviation threshold is marked as a potential fault node, and a fault detection task is generated to enter the trigger task queue.

[0123] It should be noted that the business continuity impact level of the fault scenario can be divided into the following levels: impact level 1 is core service unavailable, such as payment, database, and the corresponding weight score is 5; impact level 2 is secondary service degradation, such as log service, and the corresponding weight score is 3; impact level 3 is non-critical service delay, such as monitoring alarm, and the corresponding weight score is 1.

[0124] It should be noted that the priority score PS can be calculated by the following formula:

[0125] PS = a x DS + b x BIL

[0126] Where PS is the priority score, a and b are weight coefficients, a is 0.6 by default, b is 0.4, which can be dynamically adjusted according to the business scenario, DS is the deviation magnitude, and BIL is the weight score corresponding to the business continuity impact level.

[0127] The priority sorting can use a max-heap data structure, and the priority scores are sorted from high to low. When a new task node is inserted at the end of the heap, the heap structure is adjusted by comparing and swapping node values to make the top node of the heap always have the maximum priority score corresponding to the fault detection task. The trigger task queue is a collection of fault detection tasks containing trigger conditions, priority scores, and target fault scenarios.

[0128] In step S14, the fault detection task with the highest priority is extracted from the trigger task queue, and the corresponding fault scenario simulation is performed in the virtual environment to obtain the simulation running data of the fault scenario, including:

[0129] The fault detection task with the highest priority score is obtained from the trigger task queue, and the fault scenario to be simulated is obtained.

[0130] According to the fault scenario to be simulated, the corresponding running parameters are configured and loaded in the virtual environment to obtain the virtual environment of the fault scenario to be simulated.

[0131] The fault scenario to be simulated is simulated in the virtual environment, and the monitoring tool is used to collect the running logs and hardware performance indicators of the containerized node in real time to obtain the simulation running data.

[0132] After the simulation is completed, the virtual environment is reset for the next fault simulation.

[0133] By generating corresponding fault detection tasks according to the priority scores of each node exceeding the preset deviation threshold and performing fault simulation, simulation running data is obtained, which can be matched using the fault classification model to accurately detect the type of fault.

[0134] Among them, the fault detection task with the highest priority score can directly extract the task at the top of the trigger task queue to obtain the fault scenario to be simulated.

[0135] According to the fault detection task, the corresponding running parameters are configured and loaded in the virtual environment to obtain the virtual environment of the fault scenario to be simulated. According to the fault scenario determined in the fault detection task, the corresponding simulation configuration is generated, and the corresponding running parameters are configured and loaded in the virtual environment to simulate the system state under the fault scenario. For example, for a database server disk I / O fault, the simulation configuration may include reducing the I / O performance of the virtual disk, setting a higher read-write delay, etc. The virtualization tool provides virtualization services for the cloud platform, which will create or adjust the virtual environment according to the generated simulation configuration. This involves setting hardware parameters of the virtual machine, such as disk I / O rate, network bandwidth, etc., and software configuration, such as operating system settings, application parameters, etc., to ensure that the virtual environment can accurately simulate the expected fault scenario.

[0136] The running logs and hardware performance indicators of the containerized nodes are collected in real time using monitoring tools to obtain simulation running data. The running logs and CPU usage, memory occupation, disk I / O, network traffic, etc. hardware performance indicators of the containerized nodes can be collected by deploying Filebeat log collection tools and hardware performance monitoring tools such as Prometheus Node Exporter, Grafana Agent, etc. to obtain simulation running data.

[0137] The virtual environment is reset, which can call the snapshot management module of the virtualization environment, such as VMware's Snapshots API, to roll back the system state to the base image before simulation and securely erase residual temporary data such as coredump files, cache logs, etc. for the next simulation.

[0138] In step S15, the pre-trained fault classification model is used to match the fault pattern of the simulation running data, and the fault type and fault positioning information are output, including:

[0139] According to the simulation operation data, a log collection tool is used to extract the operation log of the containerized node in the simulation operation data, and a log analysis tool is used for structured processing to obtain structured simulation operation data;

[0140] A pre-trained fault classification model is used to match the structured simulation operation data to obtain a fault mode matching result;

[0141] According to the fault mode matching result, a rule engine is used to classify the matched fault mode, and a preset fault type mapping table is used to output the corresponding fault type;

[0142] The node identifier of the containerized node in the simulation operation data is extracted to obtain fault location information containing the node name and IP address.

[0143] Through the accurate matching of the fault classification model on the simulation operation data, the accurate fault type is obtained, and according to the source node of the simulation operation data, the positioning information of the fault is accurately extracted, thereby improving the accuracy of fault detection.

[0144] According to the simulation operation data, a log collection tool is used to extract the operation log of the containerized node, and a log analysis tool is used for structured processing to obtain structured simulation operation data, which can be realized by the following steps: using Filebeat log collection tool to extract the operation log which has been collected and saved as simulation operation data, and using Logstash, Fluentd and other log analysis tools to extract the key information in the log through regular expression, grok analysis mode and other methods, and convert it to structured format such as JSON, CSV, etc.

[0145] For example, in Logstash, the grok plug-in is configured, and the parsing rules are defined to match the timestamp, log level, message content and other fields in the log. For the log entry "[2024-10-01 10:00:00]ERROR: Payment processing failed for order ID 12345 due to connection timeout", the grok analysis can convert it to structured data in JSON format:

[0146]

[0147] The matching of the structured simulation running data by using the pre-trained fault classification model is to extract features from the structured simulation running data, and the model identifies key features in the data, such as error codes, warning information in logs, and abnormal values in hardware performance indicators, according to existing fault data; the extracted features are compared and matched with corresponding features in real-time running data to obtain a fault mode matching result.

[0148] According to the fault mode matching result, a rule engine is used to classify the matched fault mode, and a corresponding fault type is output through a pre-set fault type mapping table. A suitable rule engine such as Drools, CLIPS, etc. can be selected and configured, and rules for classifying faults based on fault mode features and business logic are defined in the rule engine; the fault mode matching result is input into the rule engine, and the rule engine evaluates the input fault mode matching result according to the pre-defined rules to check whether the features in the matching result meet the conditions specified in the rules, for example, a rule can be defined as "if the fault mode contains database connection timeout and the affected component is order service, then classify the fault as order service database connection fault".

[0149] The node identifier of the containerized node in the simulation running data is extracted to obtain fault location information containing node name and IP address. The unique identifier, node name and IP address of the containerized node are extracted to form complete fault location information, for example, the output result of the fault type combined with the fault location information can be represented as: order service database connection fault occurred at node n1 with IP address 192.168.1.101.

[0150] It should be noted that the fault type mapping table maps the classification result in the rule engine to a specific fault type description, which can be a simple dictionary or database table, where the key is the classification identifier of the rule engine and the value is the corresponding fault type name.

[0151] For example, the mapping table may contain the following mapping relationship: classification identifier 1 corresponds to network connection fault; classification identifier 2 corresponds to hardware performance fault; classification identifier 3 corresponds to application fault; when the rule engine outputs the classification result, the result is used as the key to query the fault type mapping table to obtain the corresponding fault type name.

[0152] It is worth noting that the pre-trained fault classification model can be obtained by the following steps: obtaining system historical fault data, virtual environment coupled diversified fault data and open source fault data as pre-training data of the model; labeling the corresponding fault type and positioning information in the different fault data in the pre-training data; training the model using a dual-channel deep learning architecture according to the pre-training data, combining the fault type and the positioning information; calculating the accuracy, recall rate and F1 score as evaluation results; adopting a regularization method to adjust the model according to the evaluation results to obtain the trained fault classification model.

[0153] Wherein, the pre-training data can be obtained by extracting system historical fault database information and various open source fault information, and inputting into the virtual environment for coupling, obtaining diversified fault data including system historical fault data, open source fault data and virtual environment coupled fault data as model pre-training data, and labeling the corresponding fault type and positioning information according to the obtained fault information.

[0154] It should be noted that the dual-channel deep learning architecture is divided into two channels, the first channel mainly processes the structured features of the pre-training data, uses a fully connected layer Dense Layer as the basis, and performs weight calculation and nonlinear transformation on the structured features; the second channel mainly processes text data such as system logs, and can use a one-dimensional convolutional neural network architecture.

[0155] It is worth noting that the evaluation indicators of the model can be divided into accuracy, recall rate and F1 score, and the corresponding calculation formulas are as follows:

[0156] The calculation formula of accuracy is:

[0157] Accuracy = (TP + TN) / (TP + TN + FP + FN)

[0158] The calculation formula of recall rate is:

[0159] Recall = TP / (TP + FN)

[0160] The calculation formula of F1 score is:

[0161]

[0162] Wherein, TP is true positive, TN is true negative, FP is false positive, FN is false negative, and F1 score is the harmonic mean of precision and recall, precision = TP / (TP + FP).

[0163] The regularization method is adopted to prevent overfitting, and L1 or L2 regularization methods can be adopted, the L1 regularization makes some weights of the model to be 0 by adding an absolute value weight item in the loss function, and plays a role of feature selection; the L2 regularization limits the size of the weight by adding a square weight item, and prevents the model from being too complex.

[0164] In summary, the cloud computing and virtualization technology are fused, the virtual environment containing various fault scenes is constructed, the expected running value of each node index data of the computer is simulated and calculated by combining the containerized node and the XGBoost prediction analysis, then the comprehensive deviation value of each node index data is calculated by combining the real-time running value, the fault detection task of the node corresponding to the index data whose comprehensive deviation value exceeds the preset deviation threshold is generated and simulated in the virtual environment, the fault type is accurately matched according to the simulation running data through the fault classification model, and therefore the accuracy of the computer fault detection is improved.

[0165] With reference to Figure 2 The second embodiment of the present application provides a cloud computing-based computer fault detection system, which comprises:

[0166] A virtual environment construction module is configured to construct a virtual environment containing diversified fault scenes by using a virtualization technology, acquire system running logs and hardware performance indexes, and obtain expected running data corresponding to the performance indexes in the virtual environment through prediction;

[0167] An operation deviation calculation module is configured to acquire real-time running data of the system, and calculate a comprehensive deviation value by combining a scene weighting factor, a time attenuation coefficient and the expected running data;

[0168] A task triggering generation module is configured to calculate a priority score according to the amplitude of the comprehensive deviation value exceeding a preset deviation threshold and the influence level of the fault scene on business continuity, and generate a task triggering queue sorted according to the priority score according to the priority score; the task triggering queue comprises a set of fault detection tasks of the target fault scene, a triggering condition and the priority score.

[0169] A fault scene simulation module is configured to extract the fault detection task with the highest priority from the task triggering queue, and execute corresponding fault scene simulation in the virtual environment to acquire simulation running data of the fault scene.

[0170] A matching result output module is configured to adopt a pre-trained fault classification model to perform fault mode matching on the simulation running data, and output fault type and fault positioning information.

[0171] It should be noted that the cloud computing-based electronic computer fault detection system provided by the embodiments of the present application is used to execute all process steps of the cloud computing-based electronic computer fault detection method provided by the above embodiments, and the working principles and beneficial effects of the two are one-to-one correspondence, thus not being described again.

[0172] The embodiments of the present application also provide an electronic device. The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an XGBoost prediction analysis program. The processor implements the steps in the above various cloud computing-based electronic computer fault detection method embodiments when executing the computer program, for example Figure 1 The processor implements the functions of the modules / units in the above various device embodiments when executing the computer program, for example, a running deviation calculation module.

[0173] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device.

[0174] The electronic device can be a desktop computer, a notebook, a palm computer, and a smart tablet, etc. The electronic device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above components are only examples of the electronic device and do not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the above, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, etc.

[0175] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the electronic device, and connects all parts of the electronic device through various interfaces and lines.

[0176] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the electronic device by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0177] The modules / units integrated in the electronic device can be stored in a computer readable storage medium if they are realized in the form of software function units and sold or used as independent products. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0178] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate units can or can not be physically separate, and the units displayed as units can or can not be physical units, i.e. can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. In addition, the connection relationship between the modules in the apparatus embodiment provided by the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0179] The above specific embodiments further illustrate the purpose, technical scheme and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the protection scope of the present application. It is particularly pointed out that any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A cloud computing-based electronic computer failure detection method, characterized by, The method comprises the following steps: constructing a virtual environment containing diversified fault scenarios through virtualization technology, obtaining system running logs and hardware performance indicators, and predicting to obtain expected running data corresponding to the performance indicators in the virtual environment; obtaining real-time running data of the system, combining a scenario weighting factor, a time decay coefficient and the expected running data to perform calculation, and obtaining a comprehensive deviation value; calculating a priority score according to the magnitude that the comprehensive deviation value exceeds a preset deviation threshold and the influence level of the fault scenario on business continuity, and generating a trigger task queue sorted by the priority score according to the priority score; the trigger task queue is a set of fault detection tasks containing a trigger condition, a priority score and a target fault scenario; extracting a fault detection task with the highest priority from the trigger task queue, and performing simulation of the corresponding fault scenario in the virtual environment to obtain simulation running data of the fault scenario; adopting a pre-trained fault classification model to perform fault mode matching on the simulation running data, and outputting fault type and fault positioning information; wherein the step of constructing a virtual environment containing diversified fault scenarios through virtualization technology, obtaining system running logs and hardware performance indicators, and predicting to obtain expected running data corresponding to the performance indicators in the virtual environment comprises the following steps: deploying simulation nodes by using containerization technology according to virtualization resources of a cloud computing platform, and configuring a multi-level hardware structure and a heterogeneous operating system; injecting and coupling hardware and software fault scenarios through a tool and a fault injection framework to obtain a virtual environment containing diversified fault scenarios; obtaining running logs of each node in the virtual environment through a log collection tool, and collecting hardware performance indicators in real time by using a monitoring tool; extracting key state features by using a feature extraction algorithm according to the node running logs and the hardware performance indicators to obtain a state feature set; predicting expected running states of each node in the virtual environment by using XGBoost to process structured features according to the state feature set, and generating expected running data; wherein the calculation formula of the comprehensive deviation value is: ; wherein, is a comprehensive bias value; is a scenario weighting factor; is a time decay coefficient; is real-time operational data; expected operational data; and is a value between 0 and 1. are values between 0 and 1.

2. The cloud computing based electronic computer failure detection method according to claim 1, wherein, the step of obtaining real-time running data of the system, combining a scenario weighting factor, a time decay coefficient and the expected running data to perform calculation, and obtaining a comprehensive deviation value comprises the following steps: extracting running logs containing time stamps by using a log collection tool according to deployed containerized nodes to obtain real-time running data of the system; allocating an initial weight according to the influence of the fault scenario on business, and dynamically updating the weight based on real-time business load or external events to obtain a scenario weighting factor; obtaining a time interval from fault occurrence to current calculation, and calculating a time decay coefficient according to the time interval and a preset decay rate; calculating a comprehensive deviation value according to the real-time running data, the expected running data, the scenario weighting factor and the time decay coefficient.

3. The cloud computing based electronic computer failure detection method according to claim 2, wherein, the calculation formula of the time decay coefficient is: ; wherein, is a time decay coefficient, is a decay rate in , is the time interval from the occurrence of the event to the current computation in .

4. The cloud-computing-based electronic computer failure detection method according to claim 1, characterized by, the step of calculating a priority score according to the magnitude that the comprehensive deviation value exceeds a preset deviation threshold and the influence level of the fault scenario on business continuity, and generating a trigger task queue sorted by the priority score according to the priority score comprises the following steps: According to the comprehensive deviation value, a deviation amplitude is calculated in combination with a preset deviation threshold value; Different fault scenarios are classified according to the influence on the service continuity, and a weight score corresponding to each level is obtained; According to the deviation amplitude and the weight score, a priority score of each fault scenario is calculated; A fault detection task containing a trigger condition, a priority score and a fault scenario is generated, all fault detection tasks are sorted according to the priority score, and a trigger task queue is obtained.

5. The cloud-computing-based electronic computer failure detection method according to claim 1, characterized by, The fault detection task with the highest priority is extracted from the trigger task queue, and a corresponding fault scenario simulation is performed in a virtual environment to obtain simulation running data of the fault scenario, including: The fault detection task with the highest priority score is obtained from the trigger task queue, and a fault scenario to be simulated is obtained; According to the fault scenario to be simulated, corresponding running parameters are configured and loaded in the virtual environment, and a virtual environment of the fault scenario to be simulated is obtained; The fault scenario to be simulated is simulated in the virtual environment, and a monitoring tool is used to collect running logs and hardware performance indicators of the containerized node in real time to obtain simulation running data; After the simulation is completed, the virtual environment is reset for the next fault simulation.

6. The cloud-computing-based electronic computer failure detection method according to claim 1, characterized by, The simulation running data is matched with a pre-trained fault classification model to output fault type and fault positioning information, including: According to the simulation running data, a log collection tool is used to extract the running logs of the containerized node in the simulation running data, and a log analysis tool is used for structured processing to obtain structured simulation running data; The structured simulation running data is matched with a pre-trained fault classification model to obtain a fault pattern matching result; According to the fault pattern matching result, a rule engine is used to classify the matched fault pattern, and a preset fault type mapping table is used to output the corresponding fault type; The node identifier of the containerized node in the simulation running data is extracted to obtain fault positioning information containing the node name and IP address.

7. The cloud-computing-based electronic computer failure detection method according to claim 6, characterized by, The training process of the pre-trained fault classification model includes: Obtain system historical fault data, virtual environment coupled diversified fault data and open source fault data as pre-training data of the model; Different fault data in the pre-training data are labeled with corresponding fault types and positioning information; According to the pre-training data, the fault types and the positioning information, a double-channel deep learning architecture is used to train the model; The accuracy, recall rate and F1 score are calculated as evaluation results; Regularization method is adopted to adjust the model according to the evaluation results to obtain the trained fault classification model.

8. The cloud-computing-based electronic computer failure detection method according to claim 7, characterized by, The accuracy, recall rate and F1 score are calculated by the following formula: The accuracy rate The calculation formula is: ; The recall rate The calculation formula is: ; The F1 score The formula for calculating F1 score is: ; wherein, is a true positive, is a true negative, is a false positive, is a false negative, is the harmonic mean of precision and recall, precision .

9. A cloud computing based electronic computer failure detection system, characterized by, The cloud computing-based electronic computer fault detection method according to any one of claims 1 to 8 is implemented, including: A virtual environment construction module is configured to construct a virtual environment containing diversified fault scenarios through a virtualization technology, acquire system running logs and hardware performance indexes, and obtain expected running data corresponding to the performance indexes in the virtual environment through prediction; An operation deviation calculation module is configured to acquire real-time running data of the system, combine a scenario weighting factor, a time attenuation coefficient, and the expected running data to perform calculation, and obtain a comprehensive deviation value; A trigger task generation module is configured to calculate a priority score according to a magnitude at which the comprehensive deviation value exceeds a preset deviation threshold and an influence level of a fault scenario on business continuity, and generate a trigger task queue sorted according to the priority score according to the priority score; the trigger task queue is a collection of fault detection tasks containing a trigger condition, a priority score, and a target fault scenario; A fault scenario simulation module is configured to extract a fault detection task with the highest priority from the trigger task queue, and perform simulation of a corresponding fault scenario in the virtual environment to acquire simulation running data of the fault scenario; A matching result output module is configured to perform fault mode matching on the simulation running data by using a pre-trained fault classification model, and output fault type and fault positioning information.

Images