Fault detection method and device based on log data, computer equipment, medium and product

Abstract

The invention relates to a fault detection method and device based on log data, computer equipment, a medium and a product. The method comprises the steps of performing conversion processing on interference content in log data to obtain target data, establishing a neural network architecture based on a relation perception type, training the neural network architecture through sample data, extracting feature vectors of the target data through the trained neural network architecture, and outputting the feature vectors of the target data. And obtaining a fault detection result according to the feature vector. By adopting the method, the implicit relationship between the specific content in the fault log and the potential fault service source can be deeply mined, so that the fault features are accurately captured, and the accuracy of fault detection is improved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a fault detection method, apparatus, computer equipment, medium and product based on log data. Background Art

[0002] As microservice architecture systems become increasingly complex, the number of component services they contain has increased significantly, resulting in increasingly large and complex log data. These logs contain rich information that is essential for fault detection and fault classification.

[0003] Traditional log fault detection methods mainly include template matching and keyword retrieval. Template matching mainly matches the information in the log data with the preset log template to determine whether the log is abnormal, while keyword retrieval searches for specific keywords or phrases in the log data to determine whether the log contains abnormal information.

[0004] However, since logs come from multiple different components and services, there are often complex associations and dependencies between these services. Traditional log fault classification models can only process each log in isolation, and it is difficult to link the fault conditions in the logs with their source services, thus affecting the accuracy of fault detection. Summary of the invention

[0005] Based on this, it is necessary to provide a log data-based fault detection method, device, computer equipment, medium and product that can accurately detect log data faults in response to the above technical problems.

[0006] In a first aspect, the present application provides a fault detection method based on log data, comprising:

[0007] The interference content in the log data is converted and processed to obtain the target data; the interference content includes content that is irrelevant to fault detection;

[0008] Establish a neural network architecture based on relationship perception and train it with sample data;

[0009] The feature vector of the target data is extracted through the trained neural network architecture, and the fault detection result is obtained based on the feature vector; the fault detection result includes the fault type and fault location of the target fault.

[0010] In one embodiment, the step of converting the interference content in the log data to obtain the target data includes:

[0011] Determine the data type of the interfering content in the log data;

[0012] The label content corresponding to the data type is obtained, and the corresponding interference content in the log data is replaced by the label content to obtain the target data.

[0013] In one embodiment, the relationship-aware neural network architecture includes a feature extraction layer and a classification layer composed of a preset number of encoders; the output data of the previous layer of encoders in the feature extraction layer is used as the input data of the next layer of encoders, and the output data of the last layer of encoders in the feature extraction layer is used as the input data of the classification layer.

[0014] In one embodiment, the step of training the neural network architecture using sample data includes:

[0015] Determine the vocabulary association relationship in the sample data, and obtain the relationship vector according to the vocabulary association relationship;

[0016] The neural network architecture is trained through the relationship vector and the weight parameters of the encoder are adjusted.

[0017] In one embodiment, the step of extracting a feature vector of target data through a trained neural network architecture includes:

[0018] The target data is used as the input data of the first encoder in the feature extraction layer, and the input data is encoded by the encoder in turn;

[0019] The feature vector of the target data is obtained through the output of the last encoder in the feature extraction layer.

[0020] In one embodiment, the step of obtaining a fault detection result according to a feature vector includes:

[0021] The feature vector of the target data is predicted and processed through the classification layer to obtain different candidate faults and their corresponding fault probabilities;

[0022] The candidate faults corresponding to the fault probabilities that meet the preset conditions are taken as the target faults, and the fault location is obtained based on the target faults.

[0023] In a second aspect, the present application also provides a fault detection device based on log data, comprising:

[0024] A data processing module is used to convert and process the interference content in the log data to obtain target data; the interference content includes content that is irrelevant to fault detection;

[0025] The model building module is used to build a neural network architecture based on relationship perception and train the neural network architecture through sample data;

[0026] The fault detection module is used to extract the feature vector of the target data through the trained neural network architecture and obtain the fault detection result according to the feature vector; the fault detection result includes the fault type and fault location of the target fault.

[0027] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method steps of any one of the first aspects are implemented.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements any one of the method steps in the first aspect when the computer program is executed by a processor.

[0029] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements any one of the method steps in the first aspect when executed by a processor.

[0030] The above-mentioned fault detection method, device, computer equipment, medium and product based on log data can improve data quality by removing interference content irrelevant to fault detection, provide accurate data support for subsequent detection, and deeply explore the implicit relationship between specific content in the fault log and potential fault service source through the relationship-aware neural network architecture, and can accurately capture fault characteristics, thereby improving the accuracy of fault detection. In addition, by accurately detecting the logs output by each service, it is possible to timely determine whether the system has a fault, providing real-time guarantee for the stable operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 An application environment diagram of a fault detection method based on log data in an embodiment;

[0033] Figure 2 A flow chart of a fault detection method based on log data in one embodiment;

[0034] Figure 3 A schematic diagram of the structure of a neural network architecture in one embodiment;

[0035] Figure 4 is a flow chart of a fault detection method based on log data in another embodiment;

[0036] Figure 5 is a structural block diagram of a fault detection device based on log data in an embodiment;

[0037] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] The fault detection method based on log data provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 is used to convert and process the interference content in the log data, obtain the target data, establish a neural network architecture based on relationship perception, and train the neural network architecture through sample data, extract the feature vector of the target data through the trained neural network architecture, and obtain the fault detection result according to the feature vector; the fault detection result includes the fault type and fault location of the target fault. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0040] In an exemplary embodiment, Figure 2 As shown, a fault detection method based on log data is provided, and the method is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, including the following steps 202 to 206. Among them:

[0041] S202: converting the interference content in the log data to obtain target data; the interference content includes content irrelevant to fault detection.

[0042] Alternatively, log data usually contains various information during system operation, but some of the content may not be directly related to fault detection, that is, interference content. For example, in the server log, the user's IP address or other invalid content will be recorded. The purpose of converting these interference contents is to increase the proportion of valid content in the log data.

[0043] S204: Establish a neural network architecture based on relationship perception, and train the neural network architecture through sample data.

[0044] Optionally, a relation-aware neural network architecture is a model structure specifically designed to capture complex relationships between data. It can perceive the intrinsic connections between data elements. In the fault detection scenario, it can understand the associations between different log data items and how these associations change over time. The design of this architecture may involve the setting of multiple hidden layers, the determination of the connection mode between neurons, and the selection of activation functions, etc., to better adapt to the processing requirements of fault detection related data. The established neural network architecture is trained through a large amount of sample data, which includes historical data with known fault types and fault locations. The neural network will continuously adjust its own parameters (such as weights and biases) according to the input sample data, so that its output results (prediction of fault types and fault locations) are as close as possible to the actual fault conditions in the sample data. Through repeated iterative training, the neural network gradually learns to accurately identify different fault modes from the input data features.

[0045] S206: extracting a feature vector of the target data through the trained neural network architecture, and obtaining a fault detection result based on the feature vector; the fault detection result includes a fault type and a fault location of the target fault.

[0046] Optionally, the trained neural network architecture has a strong feature extraction capability. When the processed target data is input into the neural network, it can automatically capture the key features in the target data and express these features in the form of a mathematical vector, which is a feature vector. Based on the extracted feature vector, the neural network further analyzes and gives the fault detection results, including the fault type and fault location of the target fault.

[0047] In the above-mentioned fault detection method based on log data, by removing interference content irrelevant to fault detection, the data quality can be improved, and accurate data support can be provided for subsequent detection. Through the relationship-aware neural network architecture, the implicit relationship between specific content in the fault log and the potential fault service source can be deeply mined, and the fault characteristics can be accurately captured, thereby improving the accuracy of fault detection. In addition, by accurately detecting the logs output by each service, it is possible to timely determine whether the system has a fault, providing real-time guarantee for the stable operation of the system.

[0048] In an exemplary embodiment, the step of converting the interference content in the log data to obtain the target data includes: determining the data type of the interference content in the log data; obtaining the label content corresponding to the data type, and replacing the corresponding interference content in the log data with the label content to obtain the target data.

[0049] Optionally, when the model processes interference content in the log data, it may be difficult to accurately grasp the rules that are truly related to the fault due to their irregularity and irrelevance, thereby interfering with the effective tuning process of the model parameters. For example, the log data usually contains IP address information and punctuation marks that are not related to fault detection. The model may mistakenly regard the numerical change pattern in the IP address or the frequency of occurrence of punctuation marks as fault-related features, resulting in deviations in the parameter adjustment direction, ultimately affecting the accuracy and reliability of the model.

[0050] Specifically, for different types of interference content, a specific tag label will be set for unified replacement. For example, the IP addresses in the logs are uniformly replaced with the words "[IP]". While retaining the traces of the existence of such information, by replacing specific data with a unified label, the data presentation is simplified, so that when processing log data later, the model can focus more on the valid content that is truly related to the fault. In addition, the interference content can be directly removed from the log data to increase the proportion of valid content in the log data.

[0051] In this embodiment, by replacing the interference content with a unified label content, the model can more easily capture key features related to the fault when processing data, thereby improving the accuracy and efficiency of fault detection and classification.

[0052] In an exemplary embodiment, the relationship-aware neural network architecture includes a feature extraction layer and a classification layer composed of a preset number of encoders; the output data of the previous layer of encoders in the feature extraction layer is used as the input data of the next layer of encoders, and the output data of the last layer of encoders in the feature extraction layer is used as the input data of the classification layer.

[0053] Optionally, the feature extraction layer in the relation-aware neural network architecture is composed of a preset number of encoders, and when processing log data, the data will flow through these encoders in sequence. First, the original data set (i.e., the target data) will enter the first layer of encoders in the feature extraction layer, where the encoder will perform specific conversion operations on the input data according to its own calculation rules and parameter settings to extract part of the feature information in the data. Then, the output data of the first layer of encoders will be used as the input data of the second layer of encoders. The second layer of encoders also uses its own mechanism to further mine and refine the features of the data based on the features extracted in the first layer, and this process will be repeated continuously. In other words, each layer of encoders continuously and deeply extracts richer and more detailed feature information of the log data based on the output of the previous layer, until the output of the last layer of encoders, at which time the data obtained has undergone multiple layers of feature extraction and refinement, and contains more comprehensive and in-depth log feature information. The output data of the last encoder in the feature extraction layer is used as the input data of the classification layer. The classification layer will classify the log data based on the received data after deep feature extraction. For example, it can distinguish different types of fault logs, normal operation logs, etc., and finally obtain the classification results of the logs, and judge whether the service corresponding to a log has a fault and the specific type of fault.

[0054] For example, taking the Transformer architecture as an example, compared with the traditional Transformer architecture, the Transformer based on the relation-aware mechanism adds a parameter r when calculating the dot product and weighted sum to obtain the self-attention relationship vector between different words in the log. ij To represent two log words x i and x j Assuming there are R different relationships, it can be defined as the concatenation of R eigenvectors. For each relationship, if x i and x j If there is such a relationship between them, then the corresponding feature vector will be used as a parameter for learning the union vector. Each feature vector represents the log word x. i and the relationship between some known system architectures, thereby providing richer correlation information for log feature extraction. i and x j If there is no relationship, the corresponding feature vector will be replaced by a zero vector. For example, when calculating the relationship vector between word 1 and word 2 in the log content of service 1, word 2 happens to be the name of service 2, and service 2 is one of the items in the system service list, then the association between the corresponding log word 1 in service 1 and service 2 will be recorded, and this association will be written into the relationship vector of the Transformer as a parameter.

[0055] An example of a relation-aware Transformer architecture is shown below: Figure 3 As shown in the figure, the encoder includes an input embedding module, a positional encoding module, a sequential attention module, a relation-aware attention module, an addition and normalization module, and a feed forward network module. The input log text is feature extracted by stacking multiple layers of relation-aware Transformer encoders. After feature extraction, the last encoder is followed by a classification layer to complete the model construction.

[0056] Among them, Input Embedding is used to convert the preprocessed log text into a vector form that can be processed by the computer. This is the first step for neural networks to process natural language data. It captures semantic information by mapping words or characters to low-dimensional vector space. Positional Encoding is used to add position information to each position in the input sequence so that the model can understand the order of words. Sequential Attention is used to capture the sequential dependencies in the input sequence. When processing log text, it can help the model understand the order in which log events occur. Relation-AwareAttention is used to capture the implicit relationship between log words. By adding parameters to represent the implicit relationship between different words, it provides richer association information for log feature extraction. Add&Norm is used to perform residual connection (Add) and layer normalization (Norm) on the results processed by the attention mechanism. Residual connection can alleviate the problem of gradient disappearance, and layer normalization can stabilize the training process. Feed Forward is used to further perform nonlinear transformation on the data to enhance the expressiveness of the model.

[0057] In this embodiment, through a feature extraction layer composed of multiple encoders, after multi-layer processing and feature mining, deep implicit features such as the association relationship between different services can be mined, so that the feature vector finally extracted can more comprehensively reflect the essential characteristics of the log data, thereby improving the accuracy of fault detection and classification.

[0058] In an exemplary embodiment, training a neural network architecture with sample data includes: determining vocabulary associations in the sample data, obtaining relationship vectors based on the vocabulary associations; training the neural network architecture with the relationship vectors, and adjusting weight parameters of the encoder.

[0059] Optionally, the sample data is usually annotated log data, which contains examples of various fault types and normal operating states. At this stage, it is necessary to analyze the association between the words in the sample data. Based on the determined word association relationship, a relationship vector is constructed, and the relationship vector can represent the strength and type of association between words in a numerical way. The sample data containing the relationship vector is input into the neural network architecture for training. During the training process, the neural network adjusts its own parameters according to the input data and preset goals (such as correct fault type classification, etc.) to make the output result as close to the actual annotation result as possible. As an additional input information, the relationship vector can help the neural network better understand the word relationship in the log data, so as to detect and classify faults more accurately. In the neural network architecture, the encoder is the key part responsible for extracting features. By continuously adjusting the weight parameters of the encoder during training, the encoder can better capture the key features and word associations in the log data.

[0060] In this embodiment, by determining the vocabulary association relationship in the sample data and training the neural network architecture through the relationship vector, it is possible to better understand the intrinsic relationship in the log data, thereby more accurately detecting faults and further improving the performance of the entire neural network architecture.

[0061] In an exemplary embodiment, the step of extracting a feature vector of target data through a trained neural network architecture includes: using the target data as input data of the first layer encoder in the feature extraction layer, encoding the input data through the encoder in turn; and obtaining the feature vector of the target data through the output of the last layer encoder in the feature extraction layer.

[0062] Optionally, the processed target data is fed into the first encoder of the feature extraction layer in the relation-aware neural network architecture. The first encoder performs preliminary processing and feature extraction on the input target data according to its own parameters and algorithms. After that, the output of the first encoder is used as the input of the second encoder. In this process, each encoder further extracts features based on the previous encoder. After the target data passes through all encoders in the feature extraction layer in turn, the last encoder outputs a feature vector, which is used as the input of the classification layer, and the fault detection result is finally output.

[0063] In this embodiment, through the sequential processing of multiple layers of encoders, the features in the target data can be deeply mined, and more complex and hidden features can be captured, so that the final feature vector can more accurately reflect the fault-related information, thereby improving the accuracy of fault detection.

[0064] In an exemplary embodiment, the step of obtaining a fault detection result based on a feature vector includes: performing predictive processing on the feature vector of the target data through a classification layer to obtain different candidate faults and their corresponding fault probabilities; taking the candidate fault corresponding to the fault probability that meets a preset condition as a target fault, and obtaining the fault location based on the target fault.

[0065] Optionally, when the last encoder of the feature extraction layer outputs the feature vector of the target data, the feature vector will be sent to the classification layer, which is designed based on a specific algorithm and model structure, and its purpose is to analyze and process the input feature vector. In this process, the classification layer will predict the feature vector based on the pre-trained parameters and model logic, calculate the probability value of the fault, and finally obtain the target fault that has occurred, and locate the specific fault source service of the target fault.

[0066] In this embodiment, by predicting and processing the feature vector through the classification layer, the possibility of different fault types can be judged more accurately, and the misjudgment and missed judgment can be reduced. The fault location is obtained based on the target fault, which can help operation and maintenance personnel quickly find the specific location of the fault, thereby improving the availability and reliability of the system in a targeted manner.

[0067] In an exemplary embodiment, Figure 4 As shown, a fault detection method based on log data is provided, and the method comprises the following steps:

[0068] Determine the data type of the interference content in the log data; obtain the label content corresponding to the data type, and replace the corresponding interference content in the log data with the label content to obtain the target data. The interference content includes content that is irrelevant to fault detection.

[0069] A neural network architecture based on relationship perception is established, which includes a feature extraction layer and a classification layer composed of a preset number of encoders; the output data of the previous layer of encoders in the feature extraction layer is used as the input data of the next layer of encoders, and the output data of the last layer of encoders in the feature extraction layer is used as the input data of the classification layer.

[0070] Determine the vocabulary association in the sample data, and obtain the relationship vector based on the vocabulary association; train the neural network architecture through the relationship vector and adjust the weight parameters of the encoder.

[0071] The target data is used as the input data of the first encoder layer in the feature extraction layer, and the input data is encoded by the encoder in turn; the feature vector of the target data is obtained through the output of the last encoder layer in the feature extraction layer.

[0072] The feature vector of the target data is predicted and processed through the classification layer to obtain different candidate faults and their corresponding fault probabilities; the candidate fault corresponding to the fault probability that meets the preset conditions is taken as the target fault, and the fault location is obtained based on the target fault.

[0073] In this embodiment, by removing interference content that is not related to fault detection, data quality can be improved, providing accurate data support for subsequent detection, and through the relationship-aware neural network architecture, the implicit relationship between specific content in the fault log and the potential fault service source can be deeply mined, and the fault characteristics can be accurately captured, thereby improving the accuracy of fault detection. In addition, by accurately detecting the logs output by each service, it is possible to promptly determine whether the system has a fault, providing real-time guarantee for the stable operation of the system.

[0074] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0075] Based on the same inventive concept, the embodiment of the present application also provides a log data-based fault detection device for implementing the log data-based fault detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more log data-based fault detection device embodiments provided below can refer to the above limitations on the log data-based fault detection method, which will not be repeated here.

[0076] In an exemplary embodiment, Figure 5 As shown, a fault detection device based on log data is provided, comprising: a data processing module 10, a model building module 20 and a fault detection module 30, wherein:

[0077] The data processing module 10 is used to convert the interference content in the log data to obtain target data; the interference content includes content that is irrelevant to fault detection.

[0078] The model building module 20 is used to build a neural network architecture based on relationship perception and train the neural network architecture through sample data.

[0079] The fault detection module 30 is used to extract the feature vector of the target data through the trained neural network architecture, and obtain the fault detection result according to the feature vector; the fault detection result includes the fault type and fault location of the target fault.

[0080] In an exemplary embodiment, the data processing module 10 is also used to determine the data type of the interference content in the log data; obtain the label content corresponding to the data type, and replace the corresponding interference content in the log data with the label content to obtain the target data.

[0081] In an exemplary embodiment, the relationship-aware neural network architecture involved in the model building module 20 includes a feature extraction layer and a classification layer composed of a preset number of encoders; the output data of the previous layer of encoders in the feature extraction layer is used as the input data of the next layer of encoders, and the output data of the last layer of encoders in the feature extraction layer is used as the input data of the classification layer.

[0082] In an exemplary embodiment, the model building module 20 is also used to determine the vocabulary association relationship in the sample data, obtain the relationship vector according to the vocabulary association relationship, train the neural network architecture through the relationship vector, and adjust the weight parameters of the encoder.

[0083] In an exemplary embodiment, the fault detection module 30 is also used to use the target data as the input data of the first layer encoder in the feature extraction layer, and encode the input data through the encoder in turn; and obtain the feature vector of the target data through the output of the last layer encoder in the feature extraction layer.

[0084] In an exemplary embodiment, the fault detection module 30 is also used to perform predictive processing on the feature vector of the target data through the classification layer to obtain different candidate faults and their corresponding fault probabilities; the candidate fault corresponding to the fault probability that meets the preset conditions is taken as the target fault, and the fault location is obtained according to the target fault.

[0085] Each module in the above-mentioned fault detection device based on log data can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0086] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a fault detection method based on log data is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0087] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0088] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: converting interference content in log data to obtain target data; the interference content includes content irrelevant to fault detection; establishing a neural network architecture based on relationship perception, and training the neural network architecture with sample data; extracting a feature vector of the target data through the trained neural network architecture, and obtaining a fault detection result based on the feature vector; the fault detection result includes the fault type and fault location of the target fault.

[0089] In one embodiment, when a processor executes a computer program, the process of converting interference content in log data to obtain target data includes: determining a data type of the interference content in the log data; obtaining label content corresponding to the data type, and replacing the corresponding interference content in the log data with the label content to obtain the target data.

[0090] In one embodiment, the relationship-aware neural network architecture involved in the execution of a computer program by a processor includes a feature extraction layer and a classification layer composed of a preset number of encoders; the output data of the previous layer of encoders in the feature extraction layer is used as the input data of the next layer of encoders, and the output data of the last layer of encoders in the feature extraction layer is used as the input data of the classification layer.

[0091] In one embodiment, the processor executes a computer program involving training a neural network architecture through sample data, including: determining vocabulary associations in the sample data, obtaining relationship vectors based on the vocabulary associations; training the neural network architecture through the relationship vectors, and adjusting the weight parameters of the encoder.

[0092] In one embodiment, when a processor executes a computer program, the process of extracting a feature vector of target data through a trained neural network architecture includes: using the target data as input data of the first layer encoder in the feature extraction layer, encoding the input data through the encoder in turn; and obtaining the feature vector of the target data through the output of the last layer encoder in the feature extraction layer.

[0093] In one embodiment, the method of obtaining a fault detection result based on a feature vector when a processor executes a computer program includes: predicting the feature vector of the target data through a classification layer to obtain different candidate faults and their corresponding fault probabilities; taking the candidate fault corresponding to the fault probability that meets a preset condition as a target fault, and obtaining the fault location based on the target fault.

[0094] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: interference content in log data is converted and processed to obtain target data; the interference content includes content that is irrelevant to fault detection; a neural network architecture based on relationship perception is established, and the neural network architecture is trained through sample data; a feature vector of the target data is extracted through the trained neural network architecture, and a fault detection result is obtained based on the feature vector; the fault detection result includes the fault type and fault location of the target fault.

[0095] In one embodiment, when a computer program is executed by a processor, the conversion process of interference content in log data to obtain target data includes: determining the data type of the interference content in the log data; obtaining label content corresponding to the data type, and replacing the corresponding interference content in the log data with the label content to obtain the target data.

[0096] In one embodiment, the relationship-aware neural network architecture involved when the computer program is executed by a processor includes a feature extraction layer and a classification layer composed of a preset number of encoders; the output data of the previous layer of encoders in the feature extraction layer is used as the input data of the next layer of encoders, and the output data of the last layer of encoders in the feature extraction layer is used as the input data of the classification layer.

[0097] In one embodiment, the computer program executed by the processor involves training a neural network architecture through sample data, including: determining vocabulary associations in the sample data, obtaining relationship vectors based on the vocabulary associations; training the neural network architecture through the relationship vectors, and adjusting the weight parameters of the encoder.

[0098] In one embodiment, when a computer program is executed by a processor, the process of extracting a feature vector of target data through a trained neural network architecture includes: using the target data as input data of the first layer encoder in the feature extraction layer, encoding the input data through the encoder in turn; and obtaining the feature vector of the target data through the output of the last layer encoder in the feature extraction layer.

[0099] In one embodiment, the computer program executed by the processor involves obtaining fault detection results based on feature vectors, including: predicting the feature vectors of the target data through a classification layer to obtain different candidate faults and their corresponding fault probabilities; taking the candidate fault corresponding to the fault probability that meets the preset conditions as the target fault, and obtaining the fault location based on the target fault.

[0100] In one embodiment, a computer program product is provided, including a computer program, which implements the following steps when executed by a processor: converting interference content in log data to obtain target data; the interference content includes content that is irrelevant to fault detection; establishing a neural network architecture based on relationship perception, and training the neural network architecture through sample data; extracting a feature vector of the target data through the trained neural network architecture, and obtaining a fault detection result based on the feature vector; the fault detection result includes the fault type and fault location of the target fault.

[0101] In one embodiment, when a computer program is executed by a processor, the conversion process of interference content in log data to obtain target data includes: determining the data type of the interference content in the log data; obtaining label content corresponding to the data type, and replacing the corresponding interference content in the log data with the label content to obtain the target data.

[0102] In one embodiment, the relationship-aware neural network architecture involved when the computer program is executed by a processor includes a feature extraction layer and a classification layer composed of a preset number of encoders; the output data of the previous layer of encoders in the feature extraction layer is used as the input data of the next layer of encoders, and the output data of the last layer of encoders in the feature extraction layer is used as the input data of the classification layer.

[0103] In one embodiment, the computer program executed by the processor involves training a neural network architecture through sample data, including: determining vocabulary associations in the sample data, obtaining relationship vectors based on the vocabulary associations; training the neural network architecture through the relationship vectors, and adjusting the weight parameters of the encoder.

[0104] In one embodiment, when a computer program is executed by a processor, the process of extracting a feature vector of target data through a trained neural network architecture includes: using the target data as input data of the first layer encoder in the feature extraction layer, encoding the input data through the encoder in turn; and obtaining the feature vector of the target data through the output of the last layer encoder in the feature extraction layer.

[0105] In one embodiment, the computer program executed by the processor involves obtaining fault detection results based on feature vectors, including: predicting the feature vectors of the target data through a classification layer to obtain different candidate faults and their corresponding fault probabilities; taking the candidate fault corresponding to the fault probability that meets the preset conditions as the target fault, and obtaining the fault location based on the target fault.

[0106] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0107] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0108] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A fault detection method based on log data, characterized in that: The method comprises: Performing conversion processing on the interference content in the log data to obtain target data; the interference content includes content irrelevant to fault detection; Establishing a neural network architecture based on relationship perception and training the neural network architecture through sample data; The feature vector of the target data is extracted through the trained neural network architecture, and a fault detection result is obtained according to the feature vector; the fault detection result includes the fault type and fault location of the target fault.

2. The method according to claim 1, characterized in that The step of converting the interference content in the log data to obtain target data includes: Determine the data type of the interfering content in the log data; The tag content corresponding to the data type is obtained, and the corresponding interference content in the log data is replaced by the tag content to obtain the target data.

3. The method according to claim 1, characterized in that The relationship-aware neural network architecture includes a feature extraction layer and a classification layer composed of a preset number of encoders; the output data of the previous layer of encoders in the feature extraction layer is used as the input data of the next layer of encoders, and the output data of the last layer of encoders in the feature extraction layer is used as the input data of the classification layer.

4. The method according to claim 3, characterized in that The training of the neural network architecture by using sample data includes: Determine the vocabulary association relationship in the sample data, and obtain the relationship vector according to the vocabulary association relationship; The neural network architecture is trained through the relationship vector, and the weight parameters of the encoder are adjusted.

5. The method according to claim 3, characterized in that: The extracting the feature vector of the target data through the trained neural network architecture includes: Using the target data as input data of the first encoder layer in the feature extraction layer, and encoding the input data through the encoder in sequence; The feature vector of the target data is obtained through the output of the last encoder layer in the feature extraction layer.

6. The method according to claim 3, characterized in that The obtaining of the fault detection result according to the feature vector comprises: Performing prediction processing on the feature vector of the target data through the classification layer to obtain different candidate faults and their corresponding fault probabilities; The candidate fault corresponding to the fault probability satisfying the preset condition is taken as the target fault, and the fault location is obtained according to the target fault.

7. A fault detection device based on log data, characterized in that: The device comprises: A data processing module, used for converting interference content in log data to obtain target data; the interference content includes content irrelevant to fault detection; A model building module, used to build a neural network architecture based on relationship perception and train the neural network architecture through sample data; The fault detection module is used to extract the feature vector of the target data through the trained neural network architecture, and obtain the fault detection result according to the feature vector; the fault detection result includes the fault type and fault location of the target fault.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images