Microservice fault detection method and device, storage medium and computer device

Abstract

The application discloses a micro-service fault detection method and device, a storage medium and a computer device, and mainly aims at improving the detection accuracy of micro-service faults. The method comprises the following steps: acquiring multi-modal data of a micro-service system in each time window; performing abnormality detection on the multi-modal data respectively to obtain abnormal events corresponding to the multi-modal data in each time window; determining an abnormal event sequence in each time window according to the abnormal events corresponding to the multi-modal data; performing feature extraction on the abnormal event sequence to obtain an abnormal feature vector; inputting the abnormal feature vector into a preset fault detection model for fault detection to obtain a fault detection result of the micro-service system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric power, and in particular to a micro-service fault detection method and device, a storage medium and computer equipment. BACKGROUND

[0002] Cloud native systems are usually implemented in a micro-service architecture, which allows developers to independently develop and deploy functional software units (micro-services). Complex applications can be decomposed into fine-grained, lightweight, elastic, independent and maintainable micro-services. Each lightweight micro-service is responsible for a simple function, and each function is provided by different micro-services developed by different teams. The micro-service architecture has the characteristics of high flexibility, reusability and scalability, enabling software development to be carried out in an agile manner. In order to ensure the reliability of the cloud native system, the operation and maintenance personnel need to detect micro-service faults according to the monitoring data.

[0003] At present, the micro-service system is usually detected for faults according to the collected log data or specific indicator data. However, for some resource-type faults, the log data in the micro-service system will not show abnormalities, and for some business-type faults, the indicator data will not show abnormalities, so this way of using single-dimensional data for fault detection cannot accurately detect the existing fault problems in the micro-service system. SUMMARY

[0004] The present application provides a micro-service fault detection method, device, storage medium and computer equipment, which can improve the detection accuracy of micro-service faults.

[0005] According to a first aspect of the present application, a micro-service fault detection method is provided, comprising:

[0006] Obtaining multi-modal data of a micro-service system under each time window;

[0007] Respectively detecting the multi-modal data for abnormalities to obtain abnormal events corresponding to the multi-modal data under each time window;

[0008] Determining an abnormal event sequence under each time window according to the abnormal events corresponding to the multi-modal data;

[0009] Extracting features from the abnormal event sequence to obtain an abnormal feature vector;

[0010] Inputting the abnormal feature vector into a pre-set fault detection model for fault detection to obtain a fault detection result of the micro-service system.

[0011] According to a second aspect of the present application, a micro-service fault detection device is provided, comprising:

[0012] an acquisition unit configured to acquire multi-modal data of a micro-service system in each time window;

[0013] an anomaly detection unit configured to perform anomaly detection on the multi-modal data respectively to obtain anomaly events corresponding to the multi-modal data in each time window respectively;

[0014] a determination unit configured to determine anomaly event sequences in the time windows according to the anomaly events corresponding to the multi-modal data respectively;

[0015] an extraction unit configured to perform feature extraction on the anomaly event sequences to obtain anomaly feature vectors;

[0016] a fault detection unit configured to input the anomaly feature vectors into a preset fault detection model to perform fault detection and obtain a fault detection result of the micro-service system.

[0017] According to a third aspect of the present application, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the following steps:

[0018] acquiring multi-modal data of a micro-service system in each time window;

[0019] performing anomaly detection on the multi-modal data respectively to obtain anomaly events corresponding to the multi-modal data in each time window respectively;

[0020] determining anomaly event sequences in the time windows according to the anomaly events corresponding to the multi-modal data respectively;

[0021] performing feature extraction on the anomaly event sequences to obtain anomaly feature vectors;

[0022] inputting the anomaly feature vectors into a preset fault detection model to perform fault detection and obtain a fault detection result of the micro-service system.

[0023] According to a fourth aspect of the present application, a computer device is provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program:

[0024] acquiring multi-modal data of a micro-service system in each time window;

[0025] performing anomaly detection on the multi-modal data respectively to obtain anomaly events corresponding to the multi-modal data in each time window respectively;

[0026] determining anomaly event sequences in the time windows according to the anomaly events corresponding to the multi-modal data respectively;

[0027] characteristic vector of the abnormal event is obtained by performing feature extraction on the abnormal event sequence;

[0028] The abnormal characteristic vector is input into a preset fault detection model to perform fault detection, and a fault detection result of the micro-service system is obtained.

[0029] The micro-service fault detection method, device, storage medium and computer equipment provided by the application can obtain multi-modal data of the micro-service system under each time window, and perform abnormal detection on the multi-modal data, obtain abnormal events corresponding to the multi-modal data under each time window, determine abnormal event sequences under each time window according to the abnormal events corresponding to the multi-modal data, perform feature extraction on the abnormal event sequences, obtain abnormal characteristic vectors, and finally input the abnormal characteristic vectors into a preset fault detection model to perform fault detection, and obtain a fault detection result of the micro-service system. By integrating multi-modal data and using multi-modal data and a preset fault detection model to detect micro-service faults, the detection accuracy of micro-service faults can be effectively improved, which is beneficial to quickly and accurately locating the root cause micro-service. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings, which are included to provide a further understanding of the application and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0031] Figure 1 A micro-service fault detection method flowchart provided by an embodiment of the application is shown;

[0032] Figure 2 Another micro-service fault detection method flowchart provided by an embodiment of the application is shown;

[0033] Figure 3 A call diagram of a cloud-native system provided by an embodiment of the application is shown;

[0034] Figure 4 A system architecture diagram provided by an embodiment of the application is shown;

[0035] Figure 5 A structure diagram of a micro-service fault detection device provided by an embodiment of the application is shown;

[0036] Figure 6 Another structure diagram of a micro-service fault detection device provided by an embodiment of the application is shown;

[0037] Figure 7An entity structure schematic diagram of a computer device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0038] In the following, the present application will be described in detail with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0039] At present, the way of using single-dimensional data for fault detection cannot accurately detect the fault problems existing in the micro-service system.

[0040] In order to solve the above problems, an embodiment of the present application provides a micro-service fault detection method, as shown in the figure, the method comprises the following steps. Figure 1

[0041] 101. Obtain multi-modal data of the micro-service system in each time window.

[0042] Among them, the time window can be set according to the actual business demand, and the multi-modal data includes log data, index data and call chain data.

[0043] The embodiment of the present application is mainly applicable to the scene of using multi-modal data for micro-service fault detection. The execution subject of the embodiment of the present application is a device or equipment capable of micro-service fault detection, which can be specifically set on the server side.

[0044] Before the present application is specifically described to integrate multi-modal data and use it for micro-service fault detection, the log data, index data and call chain data are first described. First, the log data records the detailed running time of the system, which specifically includes the system information (server information, network equipment information, operating system information, application software information) of the micro-service, and also includes the behavior information of the user and the business information of the network link. Therefore, it allows the developers and operation and maintenance engineers to supervise the system behavior through the log data and track the possible problems. In addition, the log semantics contains a large amount of information, for example, whether the log is associated with a fault, and what type of exception is contained in the log. Rich log information makes various system management and diagnosis tasks possible, such as analyzing statistical information, ensuring system security, discovering service faults and diagnosing fault root causes. Since the main purpose of the log is to record the state and important events of the system (including services and businesses, etc.) to help locate the problems of the system, the log data is an important information source for online monitoring and anomaly detection.

[0045] ​Secondly, Key Performance Indicator (KPI) refers to the monitoring indicators of service, system and other operation and maintenance objects, for example, CPU utilization, memory utilization, disk I / O, page access volume, page access delay, page access error number, etc. The form of storage of the indicator data is a sequence arranged in the order of time of occurrence, that is, streaming data aggregated at a predefined time interval (for example, 30 seconds or 1 minute), which is essentially a time series. The indicator data can be used to monitor abnormal behavior of the system.

[0046] Furthermore, the cloud native system is composed of multiple microservices. All calls between microservices triggered by the same user request share a unique identifier, and in the process of completing the business call, the microservice monitoring system records the call relationship between the related microservices in detail, and then connects all the recorded data into a tree link, and then obtains the call chain of the business call process of this request. In order to obtain the complete link monitoring of a call process, it is necessary to record the request state, delay and special information of different business applications into the call chain data, so the call chain data should include Trace, Span and Annotations.

[0047] Among them, Trace refers to the link process of a business request, and TraceId is usually used as the ID of this request call. In a business request process, a globally unique TraceId is generated at the beginning of the user request to identify this request, and this TraceId remains unchanged in this business request process and is continuously passed with each layer of call; Span refers to the call process of a microservice, and is usually identified by SpanId and ParentId. In a business call process, different microservices will be called, and a new SpanId will be generated to record each call of the current microservice, and the ParentId of the downstream node service is this SpanId. In this way, the level of the current request in the entire business call process can be located and its upstream and downstream node services can be determined through SpanId and ParentId; Annotations refers to the data information attached to each Span, which generally includes request call time, request state, request delay, and special information of different businesses.

[0048] As Figure 3As shown, the call chain data records the link information of each business call in the entire request process, including service call information, resource sharing information, service deployment information, etc., and the complete call topology graph of this request can be obtained accordingly. Through the topology graph, the system architect can determine the call relationship and dependency relationship between each microservice in the system, understand the deployment and overall structure of the system, monitor the running state of the microservice in real time, provide support for subsequent system maintenance and optimization, facilitate the discovery of system performance bottlenecks, analyze the call link data of the business request, determine whether there is invalid or incorrect call relationship, find the optimal call mode, and optimize the system performance. Moreover, when the system fails, the system fault can be found in time and effectively, and the problem root microservice can be located quickly and accurately.

[0049] For the embodiment of the application, the log data, index data and call link data of the microservice system in different time windows can be obtained through monitoring and recording, so as to detect microservice faults through the above-mentioned multi-modal data.

[0050] 102, respectively detecting the multi-modal data for abnormalities to obtain abnormal events corresponding to the multi-modal data in each time window.

[0051] The UniFind system architecture for microservice fault detection built by the embodiment of the application is as shown in Figure 4 The UniFind system architecture mainly includes a log module, an index module, a call chain module, an abnormal event sequence module, a feature extraction module, a model training module and a fault diagnosis module. The system architecture is used to detect microservice abnormal patterns in a cloud native system, find system faults and recover from faults in time. From a practical point of view, such a system architecture should be easy to implement and maintain a high fault discovery effect at a low time cost.

[0052] Before microservice fault detection is performed by using multi-modal data, data integration needs to be performed on the multi-modal data. In the process of data integration, the multi-modal data needs to be detected for abnormalities to determine abnormal events corresponding to the multi-modal data in each time window. When detecting, the log module will first perform log analysis on the log data, then perform abnormality detection on the analyzed log data to obtain abnormal events corresponding to the log data in each time window; the index module will first perform index screening to determine the detection index, then perform abnormality detection on the detection index data to obtain abnormal events corresponding to the index data in each time window; the call chain module will first perform time series conversion on the call chain data, then perform abnormality detection on the converted call chain data to obtain abnormal events corresponding to the call chain data in each time window. Thus, the log module, the index module and the call chain module can realize abnormality detection on the multi-modal data.

[0053] 103. Determine abnormal event sequences in the time windows according to the abnormal events corresponding to the multi-modal data respectively.

[0054] For the embodiment of the application, after obtaining the abnormal events corresponding to the log data, the index data and the call chain data respectively, the abnormal event sequence module integrates the abnormal events corresponding to the multi-modal data respectively to obtain the abnormal event sequences in the time windows.

[0055] 104. Perform feature extraction on the abnormal event sequences to obtain the abnormal feature vectors.

[0056] For the embodiment of the application, since the machine learning algorithm usually processes fixed-length input and output data, it cannot directly process the abnormal event sequences. Therefore, after obtaining the abnormal event sequences corresponding to the multi-modal data, the feature extraction module converts them into digital feature vectors, i.e., abnormal feature vectors. Specifically, the Bag of Words (BOW) model can be used to encode the abnormal event sequences into digital feature vectors.

[0057] 105. Input the abnormal feature vectors into a preset fault detection model to perform fault detection and obtain the fault detection result of the microservice system.

[0058] The fault detection result of the microservice system includes the existence of microservice faults and the non-existence of microservice faults. The existing microservice faults mainly include resource-type faults and business-type faults. Since the embodiment of the application integrates the multi-modal data related to resource-type faults and business-type faults, it can effectively detect all microservice faults in the system, thereby improving the microservice fault detection efficiency.

[0059] For the embodiment of the application, the preset fault detection model can be a gradient boosting classifier. Specifically, when performing fault detection, the model training module pre-trains a gradient boosting classifier, and then the fault detection module inputs the abnormal feature vectors into the gradient boosting classifier for classification, i.e., obtains the conclusion of the existence of microservice faults or the non-existence of microservice faults. It should be noted that the preset fault detection model in the embodiment of the application is not limited to the gradient boosting classifier, but can also be other classification models.

[0060] The microservice fault detection method provided by the embodiment of the application integrates multi-modal data and uses multi-modal data and a preset fault detection model to perform microservice fault detection, which can effectively improve the detection accuracy of microservice faults and is conducive to subsequent rapid and accurate root cause microservice positioning.

[0061] Further, in order to better illustrate the above microservice fault detection process, as a refinement and extension of the above embodiment, the embodiment of the application provides another microservice fault detection method, as shown in Figure 2 The method comprises the following steps:

[0062] 201. Obtain multi-modal data of a microservice system in each time window.

[0063] For the embodiment of the application, the specific process and steps of obtaining multi-modal data in each time window are exactly the same as those of step 101, and will not be repeated here.

[0064] 202. Analyze the log data to obtain parsed log data, and perform anomaly detection on the parsed log data to obtain abnormal events corresponding to the log data in each time window.

[0065] For the embodiment of the application, after obtaining the multi-modal data, the log module is first used to analyze and detect the abnormality of the log data. For the log analysis process, step 205 specifically comprises: determining the length and token corresponding to the log data; based on the length and token corresponding to the log data, using a preset fixed-depth tree log analysis algorithm to determine the log message template corresponding to the log data, and determining the parsed log data according to the log message template.

[0066] Since the log data is unstructured data, it is difficult to process directly, so it is necessary to parse the log data into structured data (template part + variable parameter) to obtain the log template, so as to facilitate subsequent analysis. In order to realize the automatic extraction of the log template, clustering-based, longest common subsequence-based, frequent item mining-based, heuristic-based and other algorithms can be used for log analysis. Taking the fixed-depth tree log analysis algorithm Drain in the clustering algorithm as an example, the parsing process of the log data will be described in detail. First, the original log data is preprocessed. Since the preset fixed-depth tree log analysis algorithm assumes that the log data belonging to the same log template has the same log length and token, the corresponding log template can be matched according to the log length and token corresponding to the obtained original log data, and the original log data can be parsed into structured data (template part and variable parameter) through the log template. Thus, basic information can be extracted from the log, which specifically includes: client information sending a request, timestamp and specific content of the request. All information will be aggregated in different dimensions according to a certain way, and then the parsed log data is obtained.

[0067] Further, after the log data is parsed, the parsed log data needs to be detected for abnormalities, and for the abnormality detection process, step 202 specifically includes: respectively determining whether the parsed log data under each time window exists a target keyword; and determining abnormal events corresponding to the parsed log data under each time window according to the determination result.

[0068] Specifically, since the parsed log data itself contains relevant semantic information, for example, "ERROR" and "Failure", the log data can be detected for abnormalities through keywords, that is, when such keywords appear in the parsed log data, the target keyword is extracted, it is determined that the log data has an abnormality, and the abnormal log data is taken as a log event (an abnormal event), and the form of the log event can be a binary tuple <timestamp, template id>, wherein the timestamp is used to record the time when the abnormal log event occurs, and the template id is used to record the log template corresponding to the abnormal event.

[0069] 203, determine the detection index corresponding to the micro-service system, determine the detection index data from the index data, and detect the detection index data for abnormalities to obtain abnormal events corresponding to the index data under each time window.

[0070] For the embodiment of the application, after the multi-modal data is acquired, the index module is used for index screening and abnormality detection, and for the index screening process, step 203 specifically includes: filtering out invalid indexes from each index to obtain remaining indexes; performing correlation analysis on the remaining indexes by using a Pearson correlation coefficient algorithm, and determining a detection index according to the correlation analysis result; and acquiring detection index data according to the detection index.

[0071] Since a lot of index data is recorded at the service level and the machine (physical machine and virtual machine) level, and the index data has many measurement standards, index screening is needed. Specifically, first, invalid indexes are filtered out, including indexes with single values, indexes with a large number of missing values, and linear indexes, then similarity analysis is performed on the remaining indexes, the index module calculates the correlation between indexes by using a Pearson correlation coefficient, indexes with high correlation are filtered out, and one of them is retained, and finally, the detection index is obtained.

[0072] Further, after the detection index is determined, the detection index data needs to be detected for abnormalities, and for the abnormality detection process, step 203 specifically includes: setting that the detection index data conforms to a normal distribution, determining a first average value and a first variance corresponding to the detection index data; determining a first standard deviation range according to the first average value and the first variance; and determining abnormal events corresponding to the detection index data under each time window based on the first standard deviation range.

[0073] In the embodiment of the present application, improved 3-sigma is adopted to detect abnormal events of the monitoring index data. Specifically, assuming that the monitoring index data in each time window conforms to normal distribution, a first average value and a first variance are calculated, and if a subsequent monitoring index value is not within a 3-sigma range, it is determined that an index event (an abnormal event) exists. Since different faults may cause abnormal events on the same index data, but the abnormal behaviors are different, some abnormal levels are much higher than the normal level, and some abnormal levels are much lower than the normal level, therefore, the embodiment of the present application adds abnormal behaviors to the abnormal detection method, which are respectively rising abnormal behavior and falling abnormal behavior. The form of the index event can be a binary group <timestamp, microservice + index + abnormal behavior>, wherein the timestamp records the time when the abnormal index event occurs, the microservice + index + abnormal behavior records which index of which microservice the abnormal event occurs on, and which abnormal behavior it belongs to.

[0074] 204. Time series conversion is performed on the call chain data to obtain converted call chain data, and abnormal detection is performed on the converted call chain data to obtain abnormal events corresponding to the call chain data in the time windows.

[0075] For the embodiment of the present application, after obtaining the multi-modal data, the call chain module is used to perform call chain data conversion and call chain data abnormal detection. For the time series conversion process of the call chain data, step 204 specifically includes: based on the calling party and the called party, the call chain data is respectively split and grouped to obtain a plurality of groups of split call chain data; and according to the field information involved in each group of split call chain data, converted call chain data is generated.

[0076] The embodiment of the present application splits the call chain data into a kind of time series data according to <caller-callee>. Specifically, the call chain data can be grouped by the caller service and the callee service to obtain a plurality of groups of split call chain data, and then a plurality of fields in each group of call chain data are determined. In different implementations of call chain records, the call chain data carries different fields, such as response time, status code, etc., so that converted call chain data can be obtained.

[0077] Further, after the converted call chain data is determined, it needs to be detected for abnormality. For this abnormal detection process, step 204 specifically includes: assuming that the converted call chain data conforms to normal distribution, a second average value and a second variance corresponding to the converted call chain data are determined; according to the second average value and the second variance, a second standard deviation range is determined; and based on the second standard deviation range, abnormal events corresponding to the monitoring index data in the time windows are determined.

[0078] The embodiment of the present application adopts improved 3-sigma to detect the abnormality of the converted call chain data. Specifically, for numerical fields such as response time and state code number, the time series data grouped by the caller service and the callee service is obtained through conversion, and the same abnormality detection method as the index data can be used to determine the call chain event (abnormal event). The form of the call chain event is a binary tuple <timestamp, caller+callee+abnormal behavior>, wherein the timestamp records the time when the abnormal call chain event occurs, and the caller+callee+abnormal behavior records the call between which two services the abnormal event occurs and the abnormal behavior.

[0079] 205、Integrate the abnormal events corresponding to the multi-modal data respectively under the same time window according to the timestamps to obtain the abnormal event sequence under each time window.

[0080] For the embodiment of the present application, after obtaining the abnormal events corresponding to the multi-modal data respectively under each time window, the abnormal event sequence module integrates the abnormal events corresponding to the multi-modal data respectively to obtain a unified representation, i.e., the abnormal event sequence. When a microservice fault occurs, there can be many abnormal events of different modalities on the system instance. These events reflect the abnormal information of the instance during the fault, and the integration of multi-source event information and the sorting according to the timestamps can obtain the abnormal event sequence in a period of time. Among them, the log event information is the template id, the index event information is the microservice+index+abnormal behavior, and the call chain event information is the caller+callee+abnormal behavior.

[0081] 206, Feature extraction is performed on the abnormal event sequence to obtain the abnormal feature vector.

[0082] For the embodiment of the present application, after obtaining the abnormal event sequence, the feature extraction module encodes the abnormal event sequence into a digital feature vector by using the Bag of Words (BOW). Since the machine learning algorithm often processes fixed-length input and output data, it cannot directly process the abnormal event sequence and must convert the abnormal event sequence into a digital feature vector. Specifically, the Bag of Words regards the abnormal event sequence as a set of abnormal events, and the occurrence of each abnormal event in the abnormal event sequence is independent and does not affect each other, and the grammar and other elements of the abnormal event sequence are ignored. This method is very simple and flexible, and can be used to extract various features from the abnormal event sequence, i.e., to map the abnormal event sequence to a new feature vector space.

[0083] 207, inputting the abnormal feature vector into a preset fault detection model to perform fault detection, and obtaining the fault detection result of the microservice system.

[0084] The preset fault detection model is a preset gradient boosting classifier. For the embodiment of the application, before microservice fault detection is performed, a preset gradient boosting classifier needs to be constructed. Specifically, a large amount of historical sample data can be collected, and the historical sample data is labeled according to the fault detection results corresponding to the historical sample data to obtain a sample training set, and finally the preset gradient boosting classifier is trained based on the sample training set.

[0085] In the microservice fault detection using the preset gradient boosting classifier, step 207 specifically includes: inputting the abnormal feature vector into the preset gradient boosting classifier for classification to obtain a classification result; and determining the fault detection result of the microservice system according to the classification result. Specifically, the abnormal feature vector is input into the preset gradient boosting classifier for classification to obtain a first probability value of the system having a microservice fault and a second probability value of the system not having a microservice fault, if the first probability value is greater than the second probability value, it is determined that the system has a microservice fault; and if the first probability value is less than the second probability value, it is determined that the system does not have a microservice fault.

[0086] Further, the embodiment of the application can also evaluate the UniFind system built. Specifically, the embodiment of the application uses the general public GAIA dataset in the intelligent operation and maintenance field, and obtains multi-modal data and fault intervals from July 4, 2021 to July 16, 2021 from the GAIA dataset. The time window of each fault discovery work is 10 minutes, according to the fault injection information provided by the GAIA dataset, that is, the fault label, the system is manually confirmed again whether it has a fault within 10 minutes. The data and fault labels from July 4, 2021 to July 13, 2021 for a total of 10 days are used as a training set to train the model. As shown in Table 1, the dataset contains more than 170 million rows of log data, more than 56 million pieces of index data, and more than 28 million rows of call chain data. Among them, the training set fault interval accounts for 75%, and the normal interval accounts for 25%. The data and fault labels from July 14, 2021 to July 16, 2021 for a total of three days are used as a test set to evaluate the fault discovery effect of the model, wherein the test set fault interval accounts for 78%, and the normal interval accounts for 22%. In the 13-day time span, the training set and the test set account for 80% and 20%, respectively. Therefore, the evaluation results on this dataset can better reflect the fault discovery effect of the UniFind system to verify the performance of the UniFind system.

[0087] Table 1: Dataset Details

[0088]

[0089] Further, in performance evaluation, the embodiment of the present application uses three commonly used machine learning evaluation indicators: precision, recall and F1 score. The above evaluation indicators are obtained by calculating four data in the result: true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN). The calculation method is as follows:

[0090]

[0091]

[0092]

[0093] Among them, the true class is all the fault intervals accurately judged to be found, the true negative class is all the normal intervals accurately judged to be found, the false positive class is all the normal intervals judged to be found as fault intervals (i.e. false alarm), and the false negative class is all the fault intervals not found by judgment (i.e. missed alarm). Thus, the performance of the UniFind system can be evaluated by calculating the above evaluation indicators.

[0094] Another microservice fault detection method provided by the embodiment of the present application can effectively improve the detection accuracy of microservice faults by integrating multi-modal data and using multi-modal data and a preset fault detection model for microservice fault detection, which is conducive to subsequent rapid and accurate root cause microservice positioning.

[0095] Further, as a specific implementation of Figure 1 , the embodiment of the present application provides a microservice fault detection device, as shown in Figure 5 , the device comprises an acquisition unit 31, an anomaly detection unit 32, a determination unit 33, an extraction unit 34 and a fault detection unit 35.

[0096] The acquisition unit 31 can be used to acquire multi-modal data of a microservice system in each time window.

[0097] The anomaly detection unit 32 can be used to perform anomaly detection on the multi-modal data respectively to obtain abnormal events corresponding to the multi-modal data in each time window respectively.

[0098] The determination unit 33 can be used to determine an abnormal event sequence in each time window according to the abnormal events corresponding to the multi-modal data respectively.

[0099] The extraction unit 34 can be configured to extract features of the abnormal event sequence to obtain the abnormal feature vector.

[0100] The fault detection unit 35 can be configured to input the abnormal feature vector into a preset fault detection model to perform fault detection, and obtain a fault detection result of the micro-service system.

[0101] In a specific application scenario, the multi-modal data includes log data, index data, and call chain data, and the anomaly detection unit 32 includes an analysis module 321, a screening module 322, a conversion module 323, and an anomaly detection module 324. Figure 6

[0102] The analysis module 321 can be configured to analyze the log data to obtain analyzed log data.

[0103] The screening module 322 can be configured to determine a detection index corresponding to the micro-service system, and determine detection index data from the index data.

[0104] The conversion module 323 can be configured to perform time series conversion on the call chain data to obtain converted call chain data.

[0105] The anomaly detection module 324 can be configured to perform anomaly detection on the analyzed log data, the detection index data, and the converted call chain data respectively, and obtain abnormal events corresponding to the log data, the index data, and the call chain data respectively in each time window.

[0106] Further, the analysis module 321 can be specifically configured to determine a length and a token corresponding to the log data, determine a log message template corresponding to the log data by using a preset fixed-depth tree log analysis algorithm based on the length and the token corresponding to the log data, and determine the analyzed log data according to the log message template.

[0107] The anomaly detection module 324 can be specifically configured to determine whether the analyzed log data in each time window contains a target keyword, and determine abnormal events corresponding to the analyzed log data in each time window according to the determination result.

[0108] Further, the screening module 322 can be specifically configured to filter out invalid indexes from each index to obtain remaining indexes, perform correlation analysis on the remaining indexes by using a Pearson correlation coefficient algorithm, determine detection indexes according to the correlation analysis result, and obtain detection index data according to the detection indexes.

[0109] ​The abnormality detection module 324 can further be specifically configured to set the detection index data to conform to a normal distribution, determine a first average value and a first variance corresponding to the detection index data, determine a first standard deviation range according to the first average value and the first variance, and determine an abnormal event corresponding to the detection index data in each time window based on the first standard deviation range.

[0110] Further, the conversion module 323 can be specifically configured to split and group the call chain data based on a calling party and a called party, to obtain split call chain data, and generate converted call chain data according to field information involved in each group of the split call chain data.

[0111] The abnormality detection module 324 can further be specifically configured to set the converted call chain data to conform to a normal distribution, determine a second average value and a second variance corresponding to the converted call chain data, determine a second standard deviation range according to the second average value and the second variance, and determine an abnormal event corresponding to the detection index data in each time window based on the second standard deviation range.

[0112] In a specific application scenario, the determination unit 33 can be specifically configured to integrate abnormal events corresponding to the multi-modal data in the same time window according to timestamps to obtain an abnormal event sequence in each time window.

[0113] In a specific application scenario, the fault detection unit 35 includes a classification module 351 and a determination module 352.

[0114] The classification module 351 can be configured to input the abnormal feature vector into the preset gradient boosting classifier for classification to obtain a classification result.

[0115] The determination module 352 can be configured to determine a fault detection result of the micro-service system according to the classification result.

[0116] It should be noted that other corresponding descriptions of each functional module involved in the micro-service fault detection device provided by the embodiments of the present application can be referred to the corresponding descriptions of the method shown in Figure 1 The method shown in the foregoing description will not be repeated here.

[0117] Based on the above as Figure 1Correspondingly, the method also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the following steps: obtaining multi-modal data of a micro-service system in each time window; performing abnormality detection on the multi-modal data respectively to obtain abnormal events corresponding to the multi-modal data in each time window respectively; determining abnormal event sequences in the time windows according to the abnormal events corresponding to the multi-modal data respectively; performing feature extraction on the abnormal event sequences to obtain abnormal feature vectors; and inputting the abnormal feature vectors into a preset fault detection model to perform fault detection, and obtaining a fault detection result of the micro-service system.

[0118] Based on the above embodiments of the method as shown in Figure 1 and the device as shown in Figure 5 , the embodiments of the present application also provide a physical structure diagram of a computer device, as shown in Figure 7 , which comprises a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor, wherein the memory 42 and the processor 41 are both arranged on a bus 43, and the processor 41 implements the following steps when executing the program: performing abnormality detection on the multi-modal data respectively to obtain abnormal events corresponding to the multi-modal data in each time window respectively; determining abnormal event sequences in the time windows according to the abnormal events corresponding to the multi-modal data respectively; performing feature extraction on the abnormal event sequences to obtain abnormal feature vectors; and inputting the abnormal feature vectors into a preset fault detection model to perform fault detection, and obtaining a fault detection result of the micro-service system.

[0119] The embodiments of the present application can effectively improve the detection accuracy of micro-service faults by integrating multi-modal data and using multi-modal data and a preset fault detection model to detect micro-service faults, which is conducive to rapidly and accurately locating the root cause micro-service subsequently.

[0120] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and alternatively, they can be realized by program codes executable by computing devices, so that they can be stored in storage devices and executed by computing devices, and in some cases, the steps shown or described can be executed in different orders, or they can be manufactured into individual integrated circuit modules, or multiple modules or steps among them can be manufactured into a single integrated circuit module. Therefore, the present application is not limited to any specific combination of hardware and software.

[0121] The above merely provides the preferred embodiments of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principles and technical scope of the present application shall fall into the scope of the present application.

Claims

1. A microservice fault detection method, characterized in that, The method comprises: acquiring multi-modal data of a micro-service system in each time window, wherein the multi-modal data comprises log data, index data and call chain data; performing abnormality detection on the multi-modal data respectively to obtain abnormal events corresponding to the multi-modal data in each time window respectively; determining abnormal event sequences in the time windows according to the abnormal events corresponding to the multi-modal data respectively; performing feature extraction on the abnormal event sequences to obtain abnormal feature vectors; inputting the abnormal feature vectors into a preset fault detection model for fault detection to obtain a fault detection result of the micro-service system; wherein the abnormality detection on the multi-modal data respectively to obtain abnormal events corresponding to the multi-modal data in each time window respectively comprises: parsing the log data to obtain parsed log data; determining detection indexes corresponding to the micro-service system and determining detection index data from the index data; performing time series conversion on the call chain data to obtain converted call chain data; performing abnormality detection on the parsed log data, the detection index data and the converted call chain data respectively to obtain abnormal events corresponding to the log data, the index data and the call chain data in each time window respectively; wherein the determination of abnormal event sequences in the time windows according to the abnormal events corresponding to the multi-modal data respectively comprises: integrating abnormal events corresponding to the multi-modal data in the same time window according to timestamps to obtain abnormal event sequences in the time windows.

2. The method of claim 1, wherein, The parsing of the log data to obtain parsed log data comprises: determining lengths and tokens corresponding to the log data; determining log message templates corresponding to the log data by using a preset fixed-depth tree log parsing algorithm based on the lengths and tokens corresponding to the log data, and determining the parsed log data according to the log message templates; the abnormality detection on the parsed log data to obtain abnormal events corresponding to the log data in each time window comprises: determining whether the parsed log data in each time window contains target keywords respectively; determining abnormal events corresponding to the parsed log data in each time window according to the determination results.

3. The method of claim 1, wherein, The determination of detection indexes corresponding to the micro-service system and the determination of detection index data from the index data comprises: filtering out invalid indexes from each index to obtain remaining indexes; performing correlation analysis on the remaining indexes by using a Pearson correlation coefficient algorithm, and determining detection indexes according to the correlation analysis results; acquiring detection index data according to the detection indexes; the abnormality detection on the detection index data to obtain abnormal events corresponding to the index data in each time window comprises: assuming that the detection index data conforms to a normal distribution, determining a first average value and a first variance corresponding to the detection index data; determining a first standard deviation range according to the first average value and the first variance; Based on the first standard deviation range, determine the abnormal events corresponding to the detection index data under the time windows.

4. The method of claim 1, wherein, The time series conversion of the call chain data obtains converted call chain data, including: Based on the caller and the callee, the call chain data is respectively split and grouped to obtain the split multiple groups of call chain data; According to the field information involved in each group of split call chain data, the converted call chain data is generated; The converted call chain data is detected for abnormalities to obtain the abnormal events corresponding to the call chain data under the time windows, including: Set the converted call chain data to conform to a normal distribution, determine the second average and the second variance corresponding to the converted call chain data; According to the second average and the second variance, a second standard deviation range is determined; Based on the second standard deviation range, determine the abnormal events corresponding to the detection index data under the time windows.

5. The method according to any one of claims 1 to 3, characterized in that, The preset fault detection model is a preset gradient boosting classifier, and the abnormal feature vector is input into the preset fault detection model for fault detection to obtain the fault detection result of the microservice system, including: The abnormal feature vector is input into the preset gradient boosting classifier for classification to obtain a classification result; According to the classification result, the fault detection result of the microservice system is determined.

6. A microservice fault detection apparatus, characterized by, Including: The acquisition unit is configured to acquire multi-modal data of a microservice system under each time window; wherein the multi-modal data includes log data, index data and call chain data; The anomaly detection unit is configured to detect the multi-modal data for abnormalities respectively to obtain abnormal events corresponding to the multi-modal data under each time window respectively; The determination unit is configured to determine an abnormal event sequence under each time window according to the abnormal events corresponding to the multi-modal data respectively; The extraction unit is configured to extract features from the abnormal event sequence to obtain an abnormal feature vector; The fault detection unit is configured to input the abnormal feature vector into a preset fault detection model for fault detection to obtain a fault detection result of the microservice system; Wherein, the multi-modal data is respectively detected for abnormalities to obtain the abnormal events corresponding to the multi-modal data under each time window, including: The log data is parsed to obtain parsed log data; Determine the detection index corresponding to the microservice system, and determine the detection index data from the index data; The call chain data is time series converted to obtain converted call chain data; The parsed log data, the detection index data and the converted call chain data are respectively detected for abnormalities to obtain the abnormal events corresponding to the log data, the index data and the call chain data under each time window respectively; Wherein, the abnormal event sequence under each time window is determined according to the abnormal events corresponding to the multi-modal data respectively, including: The abnormal events corresponding to the multi-modal data under the same time window are integrated according to the time stamp to obtain the abnormal event sequence under each time window.

7. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, which when executed by a processor, implements the steps of the method of any one of claims 1 to 5.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program, which when executed by a processor, implements the steps of the method of any one of claims 1 to 5.

Images