Computer anomaly detection method and system

Abstract

The invention relates to the technical field of computer internal detection, and discloses a computer anomaly detection method and system, and the method comprises the steps: collecting data through a data collection module, carrying out the processing and feature extraction of the collected data through a specific algorithm, and selecting the processed and extracted features to train a model, a fault detection model is established, a random forest classifier is used for modeling a fault detection problem, the effectiveness of the model is verified through experiments, and a fault diagnosis model is established through a feature data set corresponding to fault features. Then training the fault diagnosis model according to the feature data set based on a machine learning method to obtain an optimized fault diagnosis model, and quickly and accurately carrying out fault analysis and fault identification on computer hardware according to the optimized fault diagnosis model, so that the efficiency of fault diagnosis and the reliability of system operation are improved; therefore, software and hardware faults can be accurately diagnosed in a complex computer system.

Description

Technical Field

[0001] The present application relates to the technical field of computer internal detection, for example, to a computer anomaly detection method and system. Background Art

[0002] Software and hardware failures are one of the main issues affecting computer performance and reliability, which can lead to system crashes, data loss, and computer unavailability. In complex computer systems, diagnosing software and hardware failures is an important and challenging task. The introduction of automated fault detection tools has become an effective way to reduce failure rates and improve system availability. However, due to design defects, existing fault detection tools cannot learn new fault features in a timely manner, which can cause problems such as missed detection and false detection.

[0003] Therefore, existing computer anomaly detection methods suffer from poor detection accuracy.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The present disclosure provides a computer anomaly detection method, the method comprising:

[0007] Collect server system performance indicators, log files, system configuration files and fault database data through the data acquisition module;

[0008] The data collected by the data acquisition module is processed and features are extracted using a preset feature extraction algorithm, and the processed and extracted features are selected as a training set to train the fault detection model;

[0009] Use random forest classifier as a modeling tool to build a fault detection model, and use the fault detection model to detect hardware abnormal results;

[0010] The performance of the fault detection model is tested using data not used in the training set, and the accuracy, precision, and recall of the fault detection model are calculated based on the comparison between the predicted and actual values.

[0011] In some embodiments, the server system performance indicators include CPU utilization, memory utilization, disk IO, network bandwidth, processes and threads. The server system performance indicators are monitored by the server monitoring tool of the data acquisition module, and the monitoring of CPU, memory, disk IO and network bandwidth indicators is achieved through plug-in expansion.

[0012] In some embodiments, the method may further include integrating the data collected by the data collection module, wherein the data integration algorithm function used in the integration process is:

[0013]

[0014] Among them, Y is the data of the data collection module, n is the number of integrated data sources involved in the data integration process, and m i is the number of data in the i-th integrated data source, exp is the exponential function, x ij is the vector form of the jth data in the i-th integrated data source, is the weighted transposed weight vector of the i-th integrated data source, x ik is the vector form of the kth data in the i-th integrated data source, μ i is the mean of the i-th integrated data source, σ i is the standard deviation of the i-th integrated data source, α i is the amplitude of the Gaussian sine function of the i-th integrated data source, β i is the Gaussian sinusoidal function phase shift of the i-th integrated data source, γ i is the constant term of the Gaussian sine function of the i-th integrated data source, δ i is the limit value of the Gaussian sine function of the i-th integrated data source, and ε is the correction value of the network data.

[0015] In some embodiments, the above-mentioned processing and feature extraction using a preset feature extraction algorithm includes:

[0016] Acquire multiple groups of data collected by the data collection module, wherein each group of data includes data elements corresponding to predetermined data attributes;

[0017] Determining key data attributes for extracting feature data from the predetermined data attributes;

[0018] A template vector is established according to the data element value domain of the key data attribute and the value range of the key data element of the feature data, wherein the key data element refers to the data element in the feature data corresponding to the key data attribute;

[0019] Feature data is extracted by applying the established template vector to the data elements corresponding to the key data attributes in the plurality of groups of data.

[0020] In some embodiments, the above-mentioned use of a random forest classifier as a modeling tool to establish a fault detection model includes:

[0021] Setting a first decision tree, a second decision tree and a third decision tree, and forming a random forest fault detection model based on the first decision tree, the second decision tree and the third decision tree;

[0022] Reading a fault database to obtain a historical fault data set, wherein the historical fault data set includes historical fault data and historical fault locations, and the historical fault data corresponds to the historical fault locations; traversing and selecting two classification points in the historical fault data set and dividing the historical fault data set into a first data set, a second data set, and a third data set, and marking the classification points as data set classification points;

[0023] The number of historical fault data in the first data set, the second data set and the third data set are obtained respectively, and marked as the first number, the second number and the third number respectively, and the number of data in the historical fault data set is marked as the number of historical fault data; the classification selection probability of the first data set, the second data set and the third data set is calculated respectively by the classification probability formula; if the classification selection probability is less than zero, it is set to 0.

[0024] In some embodiments, the method may further include:

[0025] The Gini impurity of the first data set, the second data set, and the third data set is calculated by using a preset Gini impurity formula to obtain the classification Gini impurity.

[0026] In some embodiments, the method may further include:

[0027] The mean square error (MSE) and root mean square error (RMSE) are selected as evaluation indicators to evaluate the detection accuracy of the trained fault detection model;

[0028] Select the mean square error MSE to satisfy the formula The root mean square error RMSE satisfies the formula

[0029] Where n is the number of samples; ym i Indicates the actual value; y si It represents the predicted value, which is the multi-mode prediction data generated by inputting the feature data in the test set into the trained fault detection model; i is the sequence number of the multi-mode prediction data and the actual value data; the smaller the mean square error MSE and the root mean square error RMSE, the higher the detection accuracy of the trained fault detection model.

[0030] In some embodiments, the method may further include selecting a coefficient of determination R 2As an evaluation indicator, the reference value of the trained fault detection model is evaluated. The calculation formula is:

[0031]

[0032] in, Represents the average value of the actual value, which is the average value of the actual value data in the test data; ym i Indicates actual value; Represents the average value of the predicted value, which is the average value of the multi-modal predicted data generated by inputting the feature data into the trained random forest model; si Represents the predicted value; the coefficient of determination R 2 The larger the value, the greater the goodness of fit, and the higher the degree of explanation of the independent variable on the dependent variable, which means that the reference value of the trained fault detection model is greater.

[0033] The disclosed embodiment provides a computer anomaly detection system, which includes a data acquisition module, a preprocessing unit, a model building unit, and a performance evaluation unit;

[0034] Data collection module, used to collect server system performance indicators, log files, system configuration files and fault database data;

[0035] A preprocessing unit is used to process and extract features from the data collected by the data collection module using a preset feature extraction algorithm, and select the processed and extracted features as a training set to train the fault detection model;

[0036] A model building unit, used to use a random forest classifier as a modeling tool to build a fault detection model, and detect hardware abnormal results through the fault detection model;

[0037] The performance evaluation unit is used to test the performance of the fault detection model using the unused data in the training set, and calculate the accuracy, precision, and recall of the fault detection model based on the comparison between the predicted value and the actual value.

[0038] An embodiment of the present disclosure provides a storage medium storing program instructions, which, when run, execute the computer anomaly detection method in the above embodiment.

[0039] The computer anomaly detection method and system provided by the embodiments of the present disclosure can achieve the following technical effects:

[0040] Data is collected through the data acquisition module. For the collected data, specific algorithms are used to process and extract features. The processed and extracted features are selected to train the model, and a fault detection model is established. The fault detection problem is modeled using a random forest classifier, and the effectiveness of the model is verified through experiments. The feature data set corresponding to the fault characteristics is used to build a fault diagnosis model, and then the fault diagnosis model is trained based on the feature data set based on the machine learning method to obtain an optimized fault diagnosis model. Based on the optimized fault diagnosis model, computer hardware fault analysis and fault identification are quickly and accurately performed, which is conducive to improving the efficiency of fault diagnosis as well as the safety and reliability of system operation, thereby realizing accurate diagnosis of software and hardware faults in complex computer systems.

[0041] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0043] Figure 1 It is a flowchart of a computer anomaly detection method provided by an embodiment of the present disclosure;

[0044] Figure 2 is a structural diagram of a computer anomaly detection system provided by an embodiment of the present disclosure;

[0045] Figure 3 It is a structural diagram of another computer anomaly detection system provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0047] The terms "first", "second", etc. in the embodiments of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0048] Unless otherwise stated, the term "plurality" means two or more.

[0049] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0050] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0051] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0052] The computer anomaly detection method and system provided by the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0053] Figure 1 It is a flowchart of a computer anomaly detection method provided by an embodiment of the present disclosure.

[0054] like Figure 1 As shown, the computer anomaly detection method may include:

[0055] S01, collecting server system performance indicators, log files, system configuration files and fault database data through a data collection module;

[0056] S02, using a preset feature extraction algorithm to process and extract features from the data collected by the data acquisition module, and selecting the processed and extracted features as a training set to train a fault detection model;

[0057] S03, using random forest classifier as a modeling tool, establishes a fault detection model, and detects hardware abnormal results through the fault detection model;

[0058] S04, using the unused data in the training set to test the performance of the fault detection model, and calculating the accuracy, precision and recall of the fault detection model based on the comparison between the predicted values ​​and the actual values.

[0059] It should be noted that S01 is the data collection process, S02 is the feature selection and preprocessing process, S03 is the detection model establishment process, and S04 is the performance evaluation process.

[0060] In the present disclosure, data is collected through a data acquisition module, and a specific algorithm is used to process and extract features from the collected data. The processed and extracted features are selected to train a model, and a fault detection model is established. A random forest classifier is used to model the fault detection problem, and the effectiveness of the model is verified through experiments. A feature data set corresponding to the fault features is used to build a fault diagnosis model, and then the fault diagnosis model is trained based on the feature data set based on a machine learning method to obtain an optimized fault diagnosis model. Based on the optimized fault diagnosis model, fault analysis and fault identification of computer hardware are quickly and accurately performed, which is beneficial to improving the efficiency of fault diagnosis and the safety and reliability of system operation, thereby achieving accurate diagnosis of software and hardware faults in complex computer systems.

[0061] In some embodiments, the server system performance indicators include CPU utilization, memory utilization, disk IO, network bandwidth, processes and threads. The server system performance indicators are monitored by the server monitoring tool of the data acquisition module, and the monitoring of CPU, memory, disk IO and network bandwidth indicators is achieved through plug-in expansion.

[0062] In some embodiments, Figure 1 The method may further include integrating the data collected by the data collection module, wherein the data integration algorithm function used in the integration process is:

[0063]

[0064] Among them, Y is the data of the data collection module, n is the number of integrated data sources involved in the data integration process, and m i is the number of data in the i-th integrated data source, exp is the exponential function, x ij is the vector form of the jth data in the i-th integrated data source, is the weighted transposed weight vector of the i-th integrated data source, x ij is the vector form of the kth data in the i-th integrated data source, μ i is the mean of the i-th integrated data source, σ i is the standard deviation of the i-th integrated data source, α i is the amplitude of the Gaussian sine function of the i-th integrated data source, β i is the Gaussian sinusoidal function phase shift of the i-th integrated data source, γ i is the constant term of the Gaussian sine function of the i-th integrated data source, δ i is the limit value of the Gaussian sine function of the i-th integrated data source, and ε is the correction value of the network data.

[0065] In some embodiments, using a preset feature extraction algorithm to process and extract features may include:

[0066] Acquire multiple groups of data collected by the data collection module, wherein each group of data includes data elements corresponding to predetermined data attributes;

[0067] Determining key data attributes for extracting feature data from the predetermined data attributes;

[0068] A template vector is established according to the data element value domain of the key data attribute and the value range of the key data element of the feature data, wherein the key data element refers to the data element in the feature data corresponding to the key data attribute;

[0069] Feature data is extracted by applying the established template vector to the data elements corresponding to the key data attributes in the plurality of groups of data.

[0070] Specifically, the above multiple sets of data can be represented by the following matrix T:

[0071]

[0072] Where m is the number of rows of T, n is the number of columns of T, Ci,j is any data element in the i-th row and j-th column of T, where i∈[1,m], j∈[1,n], m is a positive integer, n is a positive integer, and each row of the matrix T represents a set of data, and each column of the matrix T corresponds to a data attribute.

[0073] In some embodiments, using a random forest classifier as a modeling tool to establish a fault detection model may include:

[0074] Setting a first decision tree, a second decision tree and a third decision tree, and forming a random forest fault detection model based on the first decision tree, the second decision tree and the third decision tree;

[0075] Reading a fault database to obtain a historical fault data set, wherein the historical fault data set includes historical fault data and historical fault locations, and the historical fault data corresponds to the historical fault locations; traversing and selecting two classification points in the historical fault data set and dividing the historical fault data set into a first data set, a second data set, and a third data set, and marking the classification points as data set classification points;

[0076] The number of historical fault data in the first data set, the second data set and the third data set are obtained respectively, and marked as the first number, the second number and the third number respectively, and the number of data in the historical fault data set is marked as the number of historical fault data; the classification selection probability of the first data set, the second data set and the third data set is calculated respectively by the classification probability formula; if the classification selection probability is less than zero, it is set to 0.

[0077] Specifically, the above classification probability formula can be configured as:

[0078] Among them, P i is the classification selection probability of the i-th data set, A i is the i-th quantity, the i-th data set is the first data set, the second data set or the third data set, the i-th quantity is the first quantity, the second quantity or the third quantity, i is a constant and a positive integer, and Ng is the number of historical fault data.

[0079] In some embodiments, Figure 1 The method may also include:

[0080] The Gini impurity of the first data set, the second data set, and the third data set is calculated by using a preset Gini impurity formula to obtain the classification Gini impurity.

[0081] Specifically, the Gini impurity formula can be configured as: Among them, Gn is the classification Gini impurity, and C is the number of data sets; the classification Gini impurity of different data set classification points is traversed and calculated, and the data set classification point with the smallest classification Gini impurity is selected as the final classification point. The first data set, the second data set, and the third data set corresponding to the final classification point are respectively entered into the first decision tree, the second decision tree, and the third decision tree for data training.

[0082] In some embodiments, Figure 1 The method may also include:

[0083] The mean square error (MSE) and root mean square error (RMSE) are selected as evaluation indicators to evaluate the detection accuracy of the trained fault detection model;

[0084] Select the mean square error MSE to satisfy the formula The root mean square error RMSE satisfies the formula

[0085] Where n is the number of samples; ym i Indicates the actual value; y si It represents the predicted value, which is the multi-mode prediction data generated by inputting the feature data in the test set into the trained fault detection model; i is the sequence number of the multi-mode prediction data and the actual value data; the smaller the mean square error MSE and the root mean square error RMSE, the higher the detection accuracy of the trained fault detection model.

[0086] In some embodiments, Figure 1 The method may also include selecting a coefficient of determination R 2 As an evaluation indicator, the reference value of the trained fault detection model is evaluated. The calculation formula is:

[0087]

[0088] in, Represents the average value of the actual value, which is the average value of the actual value data in the test data; mi Indicates actual value; Represents the average value of the predicted value, which is the average value of the multi-modal predicted data generated by inputting the feature data into the trained random forest model; si Represents the predicted value; the coefficient of determination R 2 The larger the value, the greater the goodness of fit, and the higher the degree of explanation of the independent variable on the dependent variable, which means that the reference value of the trained fault detection model is greater.

[0089] Corresponding to the above method embodiment, the present disclosure also provides a computer anomaly detection system, such as Figure 2 shown.

[0090] Figure 2 20 is a schematic diagram of a computer anomaly detection system provided by an embodiment of the present disclosure, wherein the computer anomaly detection system may include a data acquisition module 201, a preprocessing unit 202, a model building unit 203, and a performance evaluation unit 204;

[0091] Data collection module 201, used to collect server system performance indicators, log files, system configuration files and fault database data;

[0092] The preprocessing unit 202 is used to process and extract features from the data collected by the data collection module using a preset feature extraction algorithm, and select the processed and extracted features as a training set to train the fault detection model;

[0093] A model building unit 203 is used to use a random forest classifier as a modeling tool to build a fault detection model, and detect hardware abnormality results through the fault detection model;

[0094] The performance evaluation unit 204 is used to test the performance of the fault detection model using unused data in the training set, and calculate indicators such as accuracy, precision and recall of the fault detection model based on the comparison between the predicted value and the actual value.

[0095] The server system performance indicators include CPU utilization, memory utilization, disk IO, network bandwidth, processes and threads. The server system performance indicators are monitored by the server monitoring tool of the data acquisition module, and the monitoring of CPU, memory, disk IO and network bandwidth indicators is achieved through plug-in expansion.

[0096] In some embodiments, the above-mentioned computer anomaly detection system can also be used to integrate the data collected by the data collection module 201, wherein the data integration algorithm function used in the integration process is:

[0097]

[0098] Among them, Y is the data of the data collection module, n is the number of integrated data sources involved in the data integration process, and m i is the number of data in the i-th integrated data source, exp is the exponential function, x ij is the vector form of the jth data in the i-th integrated data source, is the weighted transposed weight vector of the i-th integrated data source, x ik is the vector form of the kth data in the i-th integrated data source, μ i is the mean of the i-th integrated data source, σ i is the standard deviation of the i-th integrated data source, α i is the amplitude of the Gaussian sine function of the i-th integrated data source, β i is the Gaussian sinusoidal function phase shift of the i-th integrated data source, γ i is the constant term of the Gaussian sine function of the i-th integrated data source, δ i is the limit value of the Gaussian sine function of the i-th integrated data source, and ε is the correction value of the network data.

[0099] In some embodiments, the above-mentioned processing and extracting features using a preset feature extraction algorithm may include:

[0100] Acquire multiple groups of data collected by the data collection module, wherein each group of data includes data elements corresponding to predetermined data attributes;

[0101] Determining key data attributes for extracting feature data from the predetermined data attributes;

[0102] A template vector is established according to the data element value domain of the key data attribute and the value range of the key data element of the feature data, wherein the key data element refers to the data element in the feature data corresponding to the key data attribute;

[0103] Feature data is extracted by applying the established template vector to the data elements corresponding to the key data attributes in the plurality of groups of data.

[0104] In some embodiments, the use of the random forest classifier as a modeling tool to establish a fault detection model may include:

[0105] Setting a first decision tree, a second decision tree, and a third decision tree, and forming a random forest fault detection model based on the first decision tree, the second decision tree, and the third decision tree;

[0106] Reading a fault database to obtain a historical fault data set, wherein the historical fault data set includes historical fault data and historical fault locations, and the historical fault data corresponds to the historical fault locations; traversing and selecting two classification points in the historical fault data set and dividing the historical fault data set into a first data set, a second data set, and a third data set, and marking the classification points as data set classification points;

[0107] The number of historical fault data in the first data set, the second data set and the third data set are obtained respectively, and marked as the first number, the second number and the third number respectively, and the number of data in the historical fault data set is marked as the number of historical fault data; the classification selection probability of the first data set, the second data set and the third data set is calculated respectively by the classification probability formula; if the classification selection probability is less than zero, it is set to 0.

[0108] In some embodiments, the above-mentioned computer anomaly detection system can also be used to calculate the Gini impurity of the first data set, the second data set and the third data set through a preset Gini impurity formula to obtain the classification Gini impurity.

[0109] In some embodiments, the above-mentioned computer anomaly detection system can also be used to select mean square error MSE and root mean square error RMSE as evaluation indicators to evaluate the detection accuracy of the trained fault detection model;

[0110] Select the mean square error MSE to satisfy the formula The root mean square error RMSE satisfies the formula

[0111] Where n is the number of samples; ym i Indicates the actual value; y si It represents the predicted value, which is the multi-mode prediction data generated by inputting the feature data in the test set into the trained fault detection model; i is the sequence number of the multi-mode prediction data and the actual value data; the smaller the mean square error MSE and the root mean square error RMSE, the higher the detection accuracy of the trained fault detection model.

[0112] In some embodiments, the above-mentioned computer anomaly detection system can also be used to select the determination coefficient R 2 As an evaluation indicator, the reference value of the trained fault detection model is evaluated. The calculation formula is:

[0113]

[0114] in, Represents the average value of the actual value, which is the average value of the actual value data in the test data; ym i Indicates actual value; Represents the average value of the predicted value, which is the average value of the multi-modal predicted data generated by inputting the feature data into the trained random forest model; si Represents the predicted value; the coefficient of determination R 2 The larger the value, the greater the goodness of fit, and the higher the degree of explanation of the independent variable on the dependent variable, which means that the reference value of the trained fault detection model is greater.

[0115] Combination Figure 3 As shown, the embodiment of the present disclosure provides another computer anomaly detection system 300, including a processor 304 and a memory 301. Optionally, the system may also include a communication interface 302 and a bus 303. The processor 304, the communication interface 302, and the memory 301 may communicate with each other through the bus 303. The communication interface 302 may be used for information transmission. The processor 304 may call the logic instructions in the memory 301 to execute the computer anomaly detection method of the above embodiment.

[0116] In addition, the logic instructions in the memory 301 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0117] The memory 301 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 304 executes the function application and data processing by running the program instructions / modules stored in the memory 301, that is, implementing the computer anomaly detection method in the above embodiment.

[0118] The memory 301 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 301 may include a high-speed random access memory and may also include a non-volatile memory.

[0119] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute a computer anomaly detection method.

[0120] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0121] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0122] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, separate components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. As used in the description of the embodiments, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or including (comprising) refer to the existence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or these groups. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same or similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0123] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the systems, devices and units described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0124] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0125] The flowchart and block diagram in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A computer anomaly detection method, characterized in that: The method comprises: Collect server system performance indicators, log files, system configuration files and fault database data through the data acquisition module; The data collected by the data acquisition module is processed and features are extracted using a preset feature extraction algorithm, and the processed and extracted features are selected as a training set to train the fault detection model; Use random forest classifier as a modeling tool to build a fault detection model, and use the fault detection model to detect hardware abnormal results; The performance of the fault detection model is tested using data not used in the training set, and the accuracy, precision, and recall of the fault detection model are calculated based on the comparison between the predicted and actual values.

2. The method according to claim 1, characterized in that The server system performance indicators include CPU utilization, memory utilization, disk IO, network bandwidth, processes and threads. The server system performance indicators are monitored by the server monitoring tool of the data acquisition module, and the monitoring of CPU, memory, disk IO and network bandwidth indicators is achieved through plug-in expansion.

3. The method according to claim 1, characterized in that The method further includes integrating the data collected by the data collection module, wherein the data integration algorithm function used in the integration process is: Among them, Y is the data of the data collection module, n is the number of integrated data sources involved in the data integration process, and m i is the number of data in the i-th integrated data source, exp is the exponential function, x ij is the vector form of the jth data in the i-th integrated data source, is the weighted transposed weight vector of the i-th integrated data source, x ik is the vector form of the kth data in the i-th integrated data source, μ i is the mean of the i-th integrated data source, σ i is the standard deviation of the i-th integrated data source, α i is the amplitude of the Gaussian sine function of the i-th integrated data source, β i is the Gaussian sinusoidal function phase shift of the i-th integrated data source, γ i is the constant term of the Gaussian sine function of the i-th integrated data source, δ i is the limit value of the Gaussian sine function of the i-th integrated data source, and ε is the correction value of the network data.

4. The method according to claim 1, characterized in that: The process of using a preset feature extraction algorithm to process and extract features includes: Acquire multiple groups of data collected by the data collection module, wherein each group of data includes data elements corresponding to predetermined data attributes; Determining key data attributes for extracting feature data from the predetermined data attributes; A template vector is established according to the data element value domain of the key data attribute and the value range of the key data element of the feature data, wherein the key data element refers to the data element in the feature data corresponding to the key data attribute; Feature data is extracted by applying the established template vector to the data elements corresponding to the key data attributes in the plurality of groups of data.

5. The method according to claim 1, characterized in that: The random forest classifier is used as a modeling tool to establish a fault detection model, including: Setting a first decision tree, a second decision tree, and a third decision tree, and forming a random forest fault detection model based on the first decision tree, the second decision tree, and the third decision tree; Reading a fault database to obtain a historical fault data set, wherein the historical fault data set includes historical fault data and historical fault locations, and the historical fault data corresponds to the historical fault locations; traversing and selecting two classification points in the historical fault data set and dividing the historical fault data set into a first data set, a second data set, and a third data set, and marking the classification points as data set classification points; The number of historical fault data in the first data set, the second data set and the third data set are obtained respectively, and marked as the first number, the second number and the third number respectively, and the number of data in the historical fault data set is marked as the number of historical fault data; the classification selection probability of the first data set, the second data set and the third data set is calculated respectively by the classification probability formula; if the classification selection probability is less than zero, it is set to 0.

6. The method according to claim 5, characterized in that The method further comprises: The Gini impurity of the first data set, the second data set, and the third data set is calculated by using a preset Gini impurity formula to obtain the classification Gini impurity.

7. The method according to claim 1, characterized in that The method further comprises: The mean square error (MSE) and root mean square error (RMSE) are selected as evaluation indicators to evaluate the detection accuracy of the trained fault detection model; Select the mean square error MSE to satisfy the formula The root mean square error RMSE satisfies the Where n is the number of samples; y mi Indicates the actual value; y si It represents the predicted value, which is the multi-mode prediction data generated by inputting the feature data in the test set into the trained fault detection model; i is the sequence number of the multi-mode prediction data and the actual value data; the smaller the mean square error MSE and the root mean square error RMSE, the higher the detection accuracy of the trained fault detection model.

8. The method according to claim 1, characterized in that: The method further comprises selecting a coefficient of determination R 2 As an evaluation indicator, the reference value of the trained fault detection model is evaluated. The calculation formula is: in, Represents the average value of the actual value, which is the average value of the actual value data in the test data; mi Indicates actual value; Represents the average value of the predicted value, which is the average value of the multi-modal predicted data generated by inputting the feature data into the trained random forest model; si Represents the predicted value; the coefficient of determination R 2 The larger the value, the greater the goodness of fit, and the higher the degree of explanation of the independent variable on the dependent variable, which means that the reference value of the trained fault detection model is greater.

9. A computer anomaly detection system, characterized in that: The system includes a data acquisition module, a preprocessing unit, a model building unit and a performance evaluation unit; Data collection module, used to collect server system performance indicators, log files, system configuration files and fault database data; A preprocessing unit is used to process and extract features from the data collected by the data collection module using a preset feature extraction algorithm, and select the processed and extracted features as a training set to train the fault detection model; A model building unit, used to use a random forest classifier as a modeling tool to build a fault detection model, and detect hardware abnormal results through the fault detection model; The performance evaluation unit is used to test the performance of the fault detection model using the unused data in the training set, and calculate the accuracy, precision, and recall of the fault detection model based on the comparison between the predicted value and the actual value.

Images