Data processing method, device and server of log

Abstract

This specification provides a data processing method, apparatus, and server for logs, belonging to the field of artificial intelligence technology. Before implementation, tensor decomposition is performed based on the directed state graph of the samples to obtain the corresponding tensor decomposition results. Based on the tensor decomposition results, the GRU network is modified to construct an initial model with fewer undetermined parameters and higher training potential. Based on the initial model, a well-performing preset log processing model can be quickly trained using only a small amount of sample logs with low training cost. During implementation, after acquiring the target logs of the target system, the preset log processing model is used to process the target logs to obtain the target processing results. Based on the target processing results, it is possible to intelligently and efficiently determine whether there are anomalies in the target system. If anomalies are determined to exist in the target system, the specific location and cause of the anomaly in the target system can be accurately determined, improving the processing accuracy and efficiency of log data processing.

Description

Technical Field

[0001] This specification belongs to the field of artificial intelligence technology, and in particular relates to log data processing methods, devices and servers. Background Technology

[0002] In internet scenarios, systems often generate a large number of system logs during operation. Typically, operations and maintenance personnel will obtain and analyze these system logs to determine whether there are any anomalies in the system.

[0003] Existing methods often require acquiring and using a large number of log samples to train a model capable of automatically analyzing and processing system logs. This model is then used to analyze the system logs and determine if any anomalies exist. However, these methods require acquiring and using a large number of sample logs, resulting in relatively high model training costs and relatively low training efficiency. Furthermore, errors are prone to occur when using these models to analyze system logs.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This specification provides a log data processing method, apparatus, and server that can quickly train a pre-set log processing model with good performance using only a small amount of sample logs and with low model training cost; and based on the above-mentioned pre-set log processing model, it can intelligently and efficiently determine whether there is anomaly in the target system; and if it is determined that there is anomaly in the target system, it can further accurately determine the specific location and cause of the anomaly in the target system.

[0006] This specification provides a method for processing log data, including:

[0007] Obtain the target logs of the target system;

[0008] The target log is processed using a pre-defined log processing model to obtain the target processing result; wherein the pre-defined log processing model is pre-trained based on the modified GRU network; the modified GRU network is constructed based on the tensor decomposition result of the sample directed state graph;

[0009] Based on the target processing results, determine whether there is an anomaly in the target system; and if it is determined that there is an anomaly in the target system, determine the location and cause of the anomaly based on the target processing results.

[0010] In one embodiment, before obtaining the target system's target logs, the method further includes:

[0011] Obtain the directed state graph of the sample;

[0012] Based on the directed state graph of the sample, the corresponding tensor decomposition result is obtained through tensor decomposition.

[0013] Based on the tensor decomposition results, the GRU network is modified to obtain the initial model;

[0014] Train the initial model to obtain a pre-defined log processing model that meets the requirements.

[0015] In one embodiment, obtaining the directed state graph of samples related to the sample logs includes:

[0016] Check for the existence of sample logs related to the sample system;

[0017] If no sample logs for the sample system are found, obtain the project code of the sample system;

[0018] The project code was parsed and organized to obtain a set of multiple finite automatic state machines;

[0019] Based on a set of multiple finite automatic state machines, generate corresponding sample directed state diagrams.

[0020] In one embodiment, after detecting the existence of sample logs for the sample system, the method further includes:

[0021] If it is determined that sample logs exist for the sample system, obtain the sample logs;

[0022] The sample log is traversed sentence by sentence to obtain and utilize the keywords in the sample log, as well as the co-occurrence statements between the keywords, to construct the initial state diagram of the sample.

[0023] Obtain and determine the direction between keywords based on the syntactic relations of keywords in co-occurring sentences;

[0024] Based on the direction between keywords, the initial state diagram of the sample is corrected into the corresponding directed state diagram of the sample.

[0025] In one embodiment, after obtaining the directed state graph of samples associated with the sample logs, the method further includes:

[0026] Community detection is performed on the directed state graph of the samples to construct an improved directed state graph of the samples composed of keyword communities.

[0027] In one embodiment, based on the directed state graph of the samples, tensor decomposition is performed to obtain the corresponding tensor decomposition result, including:

[0028] Based on the internal vocabulary, the sample directed graph is converted into the corresponding third-order tensor, initial state vector and final state vector;

[0029] Tensor decomposition is performed on the third-order tensor to obtain the internal word vector matrix, the first state transition matrix, and the second state transition matrix;

[0030] Based on the external vocabulary, an external word vector matrix that matches the internal word vector matrix is ​​obtained through spatial mapping;

[0031] The initial state vector, final state vector, internal word vector matrix, external word vector matrix, first state transition matrix, and second state transition matrix are determined as the tensor decomposition result.

[0032] In one embodiment, the GRU network is modified based on the tensor decomposition results to obtain an initial model, including:

[0033] The internal word vector matrix is ​​fused with the external word vector matrix to obtain the fused word vector matrix;

[0034] Based on the fused word vector matrix, initial state vector, and final state vector, modify the reset gate and update gate in the GRU network; and based on the first state transition matrix and the second state transition matrix, modify the output in the GRU network to obtain the initial model.

[0035] In one embodiment, training an initial model to obtain a pre-defined log processing model that meets the requirements includes:

[0036] Input the sample logs into the initial model to obtain the corresponding sample processing results;

[0037] Based on the sample logs and sample processing results, the weight matrix in the initial model is adjusted and determined to obtain a preset log processing model that meets the requirements.

[0038] In one embodiment, determining whether an anomaly exists in the target system based on the target processing result includes:

[0039] Based on the target processing results, determine the log category of the target log;

[0040] If the target log is determined to be of the specified log category, then an anomaly is identified in the target system.

[0041] In one embodiment, if the target log is of a specified log category, the target processing result further includes: a weighted finite automatic state machine corresponding to the target log, and / or a text summary result corresponding to the target log.

[0042] In one embodiment, if the target log is determined to be of a specified log category, the method further includes:

[0043] Based on the target processing results, an anomaly analysis report about the target system is generated; wherein, the anomaly analysis report includes at least the location and / or cause of the anomaly in the target system;

[0044] The abnormal prompts and the abnormal analysis report about the target system are sent to the maintenance terminal.

[0045] This specification also provides a log data processing apparatus, including:

[0046] The acquisition module is used to acquire the target logs of the target system;

[0047] The processing module is used to process the target log using a preset log processing model to obtain the target processing result; wherein, the preset log processing model is pre-trained based on the modified GRU network; the modified GRU network is constructed based on the tensor decomposition result of the sample directed state graph;

[0048] The determination module is used to determine whether there is an anomaly in the target system based on the target processing results; and if it is determined that there is an anomaly in the target system, it determines the location and cause of the anomaly based on the target processing results.

[0049] This manual also provides a method for processing log data, including:

[0050] Obtain the directed state graph of the sample;

[0051] Based on the directed state graph of the sample, the corresponding tensor decomposition result is obtained through tensor decomposition.

[0052] Based on the tensor decomposition results, the GRU network is modified to obtain the initial model;

[0053] An initial model is trained to obtain a preset log processing model that meets the requirements; wherein, the preset log processing model is used to process the target logs of the target system to determine whether there are any anomalies in the target system.

[0054] This specification also provides a server, including a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the relevant steps of the log data processing method.

[0055] This specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, perform the following steps: acquiring target logs of a target system; processing the target logs using a preset log processing model to obtain target processing results; wherein the preset log processing model is pre-trained based on a modified GRU network; the modified GRU network is constructed based on the tensor decomposition results of a sample directed state graph; determining whether an anomaly exists in the target system based on the target processing results; and, if an anomaly is determined to exist in the target system, determining the location and cause of the anomaly based on the target processing results.

[0056] Based on the log data processing method, apparatus, and server provided in this specification, in the field of artificial intelligence technology, before specific implementation, tensor decomposition is performed based on the directed state graph of the samples to obtain the corresponding tensor decomposition results. Based on the tensor decomposition results, the GRU network is modified to construct an initial model with fewer undetermined parameters and higher training potential. Then, based on the above initial model, only a small amount of sample logs are needed to quickly train a preset log processing model with good performance at a low model training cost. In specific implementation, after obtaining the target logs of the target system, the target logs are processed using the preset log processing model to obtain the target processing results. Based on these results, it is possible to intelligently and efficiently determine whether there are anomalies in the target system. Furthermore, if anomalies are determined to exist in the target system, the specific location and cause of the anomaly can be accurately determined, improving the processing accuracy and efficiency of log data processing. Attached Figure Description

[0057] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating a log data processing method provided in one embodiment of this specification;

[0059] Figure 2 This is a schematic diagram illustrating one embodiment of the log data processing method provided in the embodiments of this specification, applied in a scenario example.

[0060] Figure 3 This is a schematic diagram illustrating one embodiment of the log data processing method provided in the embodiments of this specification, applied in a scenario example.

[0061] Figure 4This is a schematic diagram illustrating one embodiment of the log data processing method provided in the embodiments of this specification, applied in a scenario example.

[0062] Figure 5 This is a schematic diagram illustrating one embodiment of the log data processing method provided in the embodiments of this specification, applied in a scenario example.

[0063] Figure 6 This is a schematic diagram of the structural composition of a server provided in one embodiment of this specification;

[0064] Figure 7 This is a schematic diagram of the structure of a log data processing device provided in one embodiment of this specification. Detailed Implementation

[0065] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0066] See Figure 1 As shown in the embodiments of this specification, a data processing method for logs is provided. In specific implementation, the method may include the following:

[0067] S101: Obtain the target log of the target system;

[0068] S102: Process the target log using a preset log processing model to obtain the target processing result; wherein, the preset log processing model is pre-trained based on the modified GRU network; the modified GRU network is constructed based on the tensor decomposition result of the sample directed state graph;

[0069] S103: Based on the target processing results, determine whether there is an anomaly in the target system; and if it is determined that there is an anomaly in the target system, determine the location and cause of the anomaly based on the target processing results.

[0070] Based on the above embodiments, the target logs of the target system can be quickly analyzed and processed by using a pre-set log processing model trained on the modified GRU network to obtain the corresponding target processing results. Then, based on the target processing results, it can be intelligently and efficiently determined whether there are any anomalies in the target system. If it is determined that there are anomalies in the target system, the specific location and cause of the anomaly in the target system can be accurately determined, thereby effectively improving the processing accuracy and efficiency of log data processing.

[0071] In some embodiments, the above-described log data processing method can be specifically applied to the monitoring server side. See also... Figure 2 As shown, the aforementioned monitoring server can be deployed on one side of the data center and associated with the target system (e.g., the target transaction system) to monitor the operation of the target system, automatically determine whether there are any anomalies in the target system, and promptly handle the anomalies in order to reduce the impact of the anomalies on the normal operation of the target system.

[0072] Specifically, the aforementioned monitoring server can be a backend server capable of data transmission and data processing. Specifically, the monitoring server can be, for example, an electronic device with data processing, storage, and network interaction capabilities. Alternatively, the monitoring server can be a software program running on the electronic device, providing support for data processing, storage, and network interaction. In this embodiment, the number of servers included in the monitoring server is not specifically limited. The monitoring server can be a single server, several servers, or a server cluster formed by several servers.

[0073] In addition, see Figure 2 As shown, the monitoring server can also be connected to a maintenance terminal. Specifically, the maintenance terminal can be a front-end deployed on the side of the data center for maintenance personnel, capable of data collection and transmission. Specifically, the maintenance terminal can be, for example, an electronic device such as a desktop computer, tablet computer, laptop computer, or smartphone. Alternatively, the maintenance terminal can also be a software application that can run on the aforementioned electronic device. For example, it could be an app running on a smartphone.

[0074] In practice, when the monitoring server detects an anomaly in the target system, it can also send an anomaly alert to the maintenance terminal held by the operations and maintenance personnel, so as to promptly notify the relevant operations and maintenance personnel.

[0075] The aforementioned preset log processing model is obtained by pre-training a modified GRU network as the initial model. Furthermore, the modified GRU network can be specifically constructed by pre-decomposing the directed state graph of the samples using tensors, based on the tensor decomposition results.

[0076] Specifically, the tensor decomposition mentioned above is also called CP decomposition. Tensors can usually be viewed as multidimensional arrays, and the order of a tensor can be understood as the index number that uniquely determines a data element within the tensor. Tensor decomposition refers to decomposing a higher-order tensor into approximate computational results of multiple lower-order tensors. For example, a third-order tensor (K×V×S) can be decomposed into approximate computational results of three second-order tensors K×T, V×T, and S×T, where T is a specified value.

[0077] The directed state graph described above specifically reflects the transformation relationships between semantic keywords of different system states based on logs in the sample system. This directed state graph is a type of directed node graph. The sample system described above can be the target system, or a system with a similar structure and operation to the target system.

[0078] Based on the tensor decomposition results, by modifying the GRU gated network in the RNN network, the resulting modified GRU network degenerates into a network purely controlled by a state machine. Building an initial model based on this modified GRU network effectively reduces the number of undetermined parameters in the model. This means that with a smaller amount of sample data and fewer training iterations, a model meeting accuracy requirements can be obtained. Furthermore, this initial model has a relatively high lower bound and greater training potential, allowing for the training of models with better performance and higher accuracy.

[0079] In some embodiments, before obtaining the target system's target logs, refer to Figure 3 As shown, in specific implementations, the method may also include the following:

[0080] S1: Obtain the directed state graph of the sample;

[0081] S2: Based on the directed state graph of the sample, the corresponding tensor decomposition result is obtained through tensor decomposition;

[0082] S3: Based on the tensor decomposition results, modify the GRU network to obtain the initial model;

[0083] S4: Train the initial model to obtain a pre-defined log processing model that meets the requirements.

[0084] Based on the above embodiments, the GRU network can be modified by tensor decomposition to obtain an initial model with fewer undetermined parameters and higher training potential. Then, a relatively good and accurate preset log processing model that meets the requirements can be efficiently trained by using a small amount of sample data and training the initial model a few times.

[0085] In some embodiments, see Figure 4 As shown, the above-described method for obtaining the directed state graph of samples related to sample logs can, in practice, include the following:

[0086] S1: Check if there is a sample log for the sample system;

[0087] S2: If it is determined that there are no sample logs for the sample system, obtain the project code of the sample system;

[0088] S3: Analyze and organize the project code to obtain a set of multiple finite automatic state machines;

[0089] S4: Generate the corresponding sample directed state diagram based on the set of multiple finite automatic state machines.

[0090] Based on the above embodiments, even in the absence or lack of sample logs from the sample system (e.g., the sample system is a new system that has just been used), the directed state graph of the samples used to build the initial model can still be successfully obtained.

[0091] In some embodiments, see Figure 5 As shown, after detecting the existence of sample logs related to the sample system, the method may further include the following in its specific implementation:

[0092] S1: If it is determined that sample logs for the sample system exist, obtain the sample logs;

[0093] S2: Traverse the sample log sentence by sentence, obtain and utilize the keywords in the sample log, as well as the co-occurrence statements between the keywords, to construct the initial state diagram of the sample;

[0094] S3: Obtain and determine the direction between keywords based on the syntactic relationship of keywords in co-occurring sentences;

[0095] S4: Based on the direction between keywords, correct the initial state diagram of the sample to the corresponding directed state diagram of the sample.

[0096] Based on the above embodiments, when sample logs of the sample system exist, the sample logs can be automatically and efficiently analyzed and processed to obtain the corresponding directed state graph of the samples.

[0097] In some embodiments, the initial state graph of the sample may specifically include multiple nodes corresponding to keywords, and edges corresponding to the co-occurrence relationships between keywords. The initial state graph of the sample is an undirected node graph. The co-occurrence relationship can specifically be represented by different keywords appearing together in the same statement in the log.

[0098] The initial state diagram of the above sample can be denoted as G. init , represented as: G init ={V init E init}, where V init E represents the set of nodes corresponding to the keyword. init This represents the set of edges that have a co-occurrence relationship.

[0099] The above-mentioned directed state graph can be understood as a type of directed node graph, which can be denoted as G. init_row Specifically, it can be represented as: G init_row ={V init E init_row}, where E init_row This represents the set of directed edges.

[0100] In some embodiments, the above-mentioned sentence-by-sentence traversal of the sample log to determine the keywords in the sample log and the co-occurring statements that have a co-occurrence relationship among the keywords may specifically include: using a keyword extraction and document summarization importance ranking algorithm to traverse the sample log sentence by sentence, obtain and utilize the keywords in the sample log and the co-occurring statements that have a co-occurrence relationship among the keywords.

[0101] Specifically, for example, the Text Rank algorithm in Snow NLP can be used to iterate through the sample logs sentence by sentence to identify the keywords in the sample logs and the co-occurring sentences that have a co-occurrence relationship between the keywords.

[0102] Specifically, Text Rank refers to a graph-based importance ranking algorithm for keyword extraction and document summarization. It's an improvement on the PageRank algorithm, which ranks web page importance. Its basic idea is to treat a document as a network of words, where links represent semantic relationships between words. Based on Text Rank, keywords or phrases can be extracted using co-occurrence information of words within a document, and key sentences can be extracted using extractive automatic summarization methods. In practice, this algorithm can be implemented using third-party Python packages such as Snow NLP or Jieba.

[0103] In some embodiments, the above-mentioned acquisition of the syntactic relations of keywords in co-occurring sentences may specifically include: processing sample logs using a syntactic analysis tree to obtain the syntactic relations of keywords in co-occurring sentences. Specifically, the syntactic relations may include one or more of the following: subject-object relation, subject-adverbial relation, subject-complement relation, subject-attributive relation, etc.

[0104] Specifically, for example, the syntactic analysis tree method in Snow NLP can be used to process sample logs to obtain the syntactic relationships of keywords in co-occurring statements.

[0105] In some embodiments, after obtaining the sample logs, the method may further include: selecting representative typical sample logs from the sample logs. Specifically, the aforementioned typical sample logs may refer to sample logs that can cover most system state transition relationships.

[0106] Specifically, the sample logs can be searched for keywords based on a pre-defined system status keyword list; and typical sample logs can be selected from the search results. The pre-defined system status keyword list contains keywords related to common system states. This pre-defined system status keyword list can be determined in advance through clustering of a large amount of historical data.

[0107] In some embodiments, if it is determined that there are sample logs about the sample system, the initial state diagram of the sample can also be constructed in the following manner: the sample logs are summarized by combining expert experience to obtain a set of multiple regular expressions; and the corresponding initial state diagram of the sample is constructed based on the set of multiple regular expressions.

[0108] In some embodiments, after obtaining the directed state graph of samples related to sample logs, the method may further include the following: performing community detection on the directed state graph of samples to construct an improved directed state graph of samples composed of keyword communities.

[0109] When conducting community discovery, you can first search for the semantic similarity between keywords; based on the semantic similarity between keywords, identify multiple nodes with community relationships from the multiple nodes corresponding to the keywords; then merge the multiple nodes with community relationships into one node as a keyword community.

[0110] Based on the above embodiments, community discovery can be used to find and merge nodes with community relationships in the sample directed state graph, resulting in an improved sample directed state graph with a relatively small number of nodes and relatively low complexity. Subsequently, the data processing can be completed more efficiently based on the improved sample directed state graph.

[0111] In some embodiments, the tensor decomposition result obtained by tensor decomposition based on the sample directed state graph can specifically include the following:

[0112] S1: Based on the internal vocabulary, convert the sample directed graph into a corresponding third-order tensor (e.g., it can be denoted as TensorG). L×|V|×|V| ), initial state vector (which can be denoted as ) and final state vector (which can be denoted as α) end );

[0113] S2: Perform tensor decomposition on the third-order tensor to obtain the internal word vector matrix (e.g., it can be denoted as E_In). R (|L|×R)), the first state transition matrix (which can be denoted as D1(|V)). init |×R)) and the second state transition matrix (which can be denoted as D2(|V)) init |×R));

[0114] S3: Based on the external vocabulary, through spatial mapping, obtain the external word vector matrix (which can be denoted as E_Out) that matches the internal word vector matrix. O );

[0115] S4: The initial state vector, final state vector, internal word vector matrix, external word vector matrix, first state transition matrix, and second state transition matrix are determined as the tensor decomposition result.

[0116] Specifically, tensor decomposition can be implemented using Python packages such as tensorly.

[0117] Based on the above embodiments, tensor decomposition results required for modifying the GRU network can be obtained by tensor decomposition based on the directed state graph of the samples.

[0118] In some embodiments, the above-mentioned method of obtaining an external word vector matrix that matches the internal word vector matrix through spatial mapping, based on an external vocabulary, can specifically include: determining the corresponding external word vectors according to the keywords and the external vocabulary; combining the external word vectors to obtain an initial external word vector matrix; obtaining and processing the initial external word vector matrix using a spatial transformation matrix to map the initial external word vector matrix to the vector space where the internal word vector matrix resides, thereby obtaining an external word vector matrix that matches the internal word vector matrix. This ensures that the dimension of the introduced external word vector matrix is ​​the same as the spatial dimension of the internal word vector matrix.

[0119] In some embodiments, the above-described tensor decomposition of the third-order tensor may specifically include: determining the rank of the third-order tensor; and performing tensor decomposition on the third-order tensor based on the rank of the third-order tensor.

[0120] In some embodiments, the GRU network is modified based on the tensor decomposition results to obtain an initial model. In specific implementations, this may include the following:

[0121] S1: Merge the internal word vector matrix with the external word vector matrix to obtain the merged word vector matrix;

[0122] S2: Based on the fused word vector matrix, initial state vector, and final state vector, modify the reset gate and update gate in the GRU network; and based on the first state transition matrix and the second state transition matrix, modify the output in the GRU network to obtain the initial model.

[0123] Based on the above embodiments, an external word vector matrix can be introduced and fused with the internal word vector matrix to obtain a relatively richer fused word vector matrix; then, the GRU network can be modified in a targeted manner according to the fused word vector matrix to obtain an initial model with relatively better performance.

[0124] In some embodiments, the above-mentioned training of the initial model to obtain a preset log processing model that meets the requirements may, in specific implementation, include:

[0125] S1: Input the sample logs into the initial model to obtain the corresponding sample processing results;

[0126] S2: Based on the sample logs and sample processing results, adjust and determine the weight matrix in the initial model to obtain a preset log processing model that meets the requirements.

[0127] Based on the above embodiments, only a small number of sample logs are needed to complete the model training of the initial model and obtain a preset log processing model that meets the requirements.

[0128] In some embodiments, determining whether the target system has an anomaly based on the target processing result may specifically include: determining the log category of the target log based on the target processing result; and determining that the target system has an anomaly if the log category of the target log is a specified log category.

[0129] Based on the above embodiments, the target processing results of the target log output can be processed using a preset log processing model, and the abnormality of the target system related to the target log can be determined efficiently and accurately.

[0130] In some embodiments, when the log category of the target log is a specified log category, the target processing result may further include: a weighted finite automatic state machine corresponding to the target log, and / or, a text summary result corresponding to the target log, etc.

[0131] Specifically, while processing the target log using a pre-defined log processing model and outputting its log category, the model also outputs a weighted finite automatic state machine (FEM) corresponding to the target log. Next, the pre-defined log processing model can adjust relevant thresholds to convert the relevant weight values ​​to 0 or 1, resulting in a transformed FEM; this transformed FEM can be understood as a state machine that can be normally interpreted by machines or maintenance personnel. Further, the pre-defined log processing model can extract key sentences containing state transition keywords according to the sentence order of the transformed FEM; then, it uses a relevant text summarization algorithm (e.g., the text summarization module in Snow NLP) to process these key sentences and output the corresponding text summarization results.

[0132] Based on the above embodiments, a preset log processing model can be used to process the target log, output the log category of the target log, and further process the target log to obtain and output a transformed automatic state machine suitable for normal interpretation by machines or maintenance personnel. Then, semantic analysis processing is performed based on the transformed automatic state machine to extract text summary results containing key information.

[0133] In some embodiments, when the log category of the target log is determined to be a specified log category, the method may further include the following:

[0134] S1: Based on the target processing results, generate an anomaly analysis report about the target system; wherein, the anomaly analysis report includes at least the location and / or cause of the anomaly in the target system;

[0135] S2: Send the anomaly alerts and anomaly analysis reports about the target system to the maintenance terminal.

[0136] Based on the above embodiments, when the monitoring server determines that an anomaly exists in the target system according to the target processing results, it can further analyze the location and / or cause of the anomaly in the target system based on the weighted finite automatic state machine corresponding to the target log in the target processing results, and / or the text digest results corresponding to the target log, thereby generating a corresponding anomaly analysis report. This anomaly analysis report, along with an anomaly alert, is then sent to the maintenance terminal to promptly notify the operations and maintenance personnel. Correspondingly, the operations and maintenance personnel can respond to the anomaly alert and, based on the anomaly analysis report, efficiently and specifically perform repair processing to eliminate the anomaly in the target system and ensure the healthy and stable operation of the target system.

[0137] In some embodiments, after generating an anomaly analysis report for the target system based on the target processing results, the method further includes: selecting, based on the anomaly analysis report, a preset repair solution matching the anomaly location and / or cause from a preset repair solution set as a target repair solution for the target system; and then automatically performing emergency repairs on the target system based on the target repair solution. The preset repair solution set contains multiple preset repair solutions, which are obtained by pre-organizing and clustering a large number of system repair records.

[0138] As can be seen from the above, the log data processing method provided in the embodiments of this specification, in the field of artificial intelligence technology, before specific implementation, involves tensor decomposition based on the directed state graph of the samples to obtain the corresponding tensor decomposition results; and based on the tensor decomposition results, the GRU network is modified to construct an initial model with fewer undetermined parameters and higher training potential; then, based on the above initial model, only a small amount of sample logs are needed to quickly train a preset log processing model with good performance at a low model training cost. In specific implementation, after obtaining the target logs of the target system, the target logs are processed using the preset log processing model to obtain the target processing results; based on the target processing results, it is possible to intelligently and efficiently determine whether there are anomalies in the target system; and if it is determined that there are anomalies in the target system, the specific location and cause of the anomaly in the target system can be further accurately determined, thereby improving the processing accuracy and efficiency of log data processing.

[0139] This specification also provides a server, including a processor and a memory for storing processor-executable instructions. Specifically, the processor can perform the following steps according to the instructions: acquiring target logs of a target system; processing the target logs using a preset log processing model to obtain target processing results; wherein the preset log processing model is pre-trained based on a modified GRU network; the modified GRU network is constructed based on the tensor decomposition results of a sample directed state graph; determining whether an anomaly exists in the target system based on the target processing results; and, if an anomaly is determined in the target system, determining the location and cause of the anomaly based on the target processing results.

[0140] To execute the above instructions more accurately, please refer to... Figure 6 As shown in the embodiments of this specification, another specific server is also provided, wherein the server includes a network communication port 601, a processor 602 and a memory 603, and the above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0141] Specifically, the network communication port 601 can be used to obtain the target logs of the target system.

[0142] The processor 602 can specifically be used to process target logs using a preset log processing model to obtain target processing results; wherein, the preset log processing model is pre-trained based on a modified GRU network; the modified GRU network is constructed based on the tensor decomposition results of the sample directed state graph; based on the target processing results, it is determined whether there is an anomaly in the target system; and if it is determined that there is an anomaly in the target system, based on the target processing results, it is determined that the location and cause of the anomaly in the target system are determined.

[0143] The memory 603 can be used to store the corresponding instruction program.

[0144] In this embodiment, the network communication port 601 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0145] In this embodiment, the processor 602 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0146] In this embodiment, the memory 603 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0147] This specification also provides a computer storage medium based on the above-described log data processing method. The computer storage medium stores computer program instructions, which, when executed, implement the following: acquiring target logs of a target system; processing the target logs using a preset log processing model to obtain target processing results; wherein the preset log processing model is pre-trained based on a modified GRU network; the modified GRU network is constructed based on the tensor decomposition results of a sample directed state graph; determining whether an anomaly exists in the target system based on the target processing results; and, if an anomaly is determined in the target system, determining the location and cause of the anomaly based on the target processing results.

[0148] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0149] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer storage medium can be explained in comparison with other implementation methods, and will not be repeated here.

[0150] This specification also provides a computer program product, including a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to execute and implement the following steps: acquiring target logs of a target system; processing the target logs using a preset log processing model to obtain target processing results; wherein the preset log processing model is pre-trained based on a modified GRU network; the modified GRU network is constructed based on the tensor decomposition results of a sample directed state graph; determining whether an anomaly exists in the target system based on the target processing results; and, if an anomaly is determined to exist in the target system, determining the location and cause of the anomaly based on the target processing results.

[0151] See Figure 7 As shown, at the software level, this specification also provides a log data processing apparatus, which may specifically include the following structural modules:

[0152] Module 701 can be used to obtain target logs from the target system.

[0153] The processing module 702 is specifically used to process the target log using a preset log processing model to obtain the target processing result; wherein, the preset log processing model is pre-trained based on the modified GRU network; the modified GRU network is constructed based on the tensor decomposition result of the sample directed state graph;

[0154] The determination module 703 can be used to determine whether there is an anomaly in the target system based on the target processing result; and if it is determined that there is an anomaly in the target system, to determine the location and cause of the anomaly based on the target processing result.

[0155] In some embodiments, the device may further include a training module. Specifically, the training module may train a preset log processing model in the following manner: acquiring a sample directed state graph; obtaining the corresponding tensor decomposition result based on the sample directed state graph through tensor decomposition; modifying the GRU network according to the tensor decomposition result to obtain an initial model; and training the initial model to obtain a preset log processing model that meets the requirements.

[0156] In some embodiments, when the training module is specifically implemented, the directed state graph of the sample log can be obtained in the following manner: detect whether there is a sample log about the sample system; if it is determined that there is no sample log about the sample system, obtain the project code of the sample system; parse and organize the project code to obtain a set of multiple finite automatic state machines; generate the corresponding directed state graph of the sample based on the set of multiple finite automatic state machines.

[0157] In some embodiments, after detecting whether there are sample logs about the sample system, the training module described above can also be used to obtain sample logs if it is determined that there are sample logs about the sample system; traverse the sample logs sentence by sentence, obtain and utilize the keywords in the sample logs, as well as the co-occurrence statements between the keywords, to construct an initial sample state graph; obtain and determine the direction between the keywords based on the syntactic relationship between the keywords in the co-occurrence statements; and correct the initial sample state graph into a corresponding directed sample state graph based on the direction between the keywords.

[0158] In some embodiments, after obtaining the directed state graph of samples related to sample logs, the above-mentioned training module can also be used to perform community detection on the directed state graph of samples in order to construct an improved directed state graph of samples composed of keyword communities.

[0159] In some embodiments, when the above training module is specifically implemented, it can obtain the corresponding tensor decomposition result based on the directed state graph of the samples through tensor decomposition in the following manner: based on the internal vocabulary, the directed graph of the samples is converted into a corresponding third-order tensor, initial state vector, and final state vector; tensor decomposition is performed on the third-order tensor to obtain an internal word vector matrix, a first state transition matrix, and a second state transition matrix; based on the external vocabulary, an external word vector matrix matching the internal word vector matrix is ​​obtained through spatial mapping; the initial state vector, final state vector, internal word vector matrix, external word vector matrix, first state transition matrix, and second state transition matrix are determined as the tensor decomposition result.

[0160] In some embodiments, when the above training module is specifically implemented, the GRU network can be modified according to the tensor decomposition results to obtain the initial model in the following manner: the internal word vector matrix and the external word vector matrix are fused to obtain the fused word vector matrix; the reset gate and update gate in the GRU network are modified according to the fused word vector matrix, the initial state vector and the final state vector; and the output in the GRU network is modified according to the first state transition matrix and the second state transition matrix to obtain the initial model.

[0161] In some embodiments, when the above training module is specifically implemented, the initial model can be trained in the following manner to obtain a preset log processing model that meets the requirements: input sample logs into the initial model to obtain the corresponding sample processing results; adjust and determine the weight matrix in the initial model according to the sample logs and sample processing results to obtain a preset log processing model that meets the requirements.

[0162] In some embodiments, when the determination module 703 is specifically implemented, it can determine whether there is an anomaly in the target system based on the target processing result in the following manner: determine the log category of the target log based on the target processing result; if the log category of the target log is determined to be the specified log category, it is determined that there is an anomaly in the target system.

[0163] In some embodiments, when the log category of the target log is a specified log category, the target processing result further includes: a weighted finite automatic state machine corresponding to the target log, and / or, a text summary result corresponding to the target log, etc.

[0164] In some embodiments, when the log category of the target log is determined to be a specified log category, the device can also be used to generate an anomaly analysis report about the target system based on the target processing result; wherein, the anomaly analysis report at least includes the location and / or cause of the anomaly in the target system; and the anomaly prompt about the target system and the anomaly analysis report are sent to the maintenance terminal.

[0165] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0166] As can be seen from the above, the log data processing device provided in the embodiments of this specification, before specific implementation, obtains the corresponding tensor decomposition results by performing tensor decomposition based on the directed state graph of the samples; and based on the tensor decomposition results, modifies the GRU network to construct an initial model with fewer undetermined parameters and higher training potential; then, based on the above initial model, only a small amount of sample logs are needed to quickly train a preset log processing model with good performance at a low model training cost. In specific implementation, after obtaining the target logs of the target system, the target logs are processed using the preset log processing model to obtain the target processing results; based on the target processing results, it is possible to intelligently and efficiently determine whether there are anomalies in the target system; and if it is determined that there are anomalies in the target system, the specific location and cause of the anomaly in the target system can be further accurately determined, thereby improving the processing accuracy and efficiency of log data processing.

[0167] In a specific scenario example, the log data processing methods provided in this specification can be applied to train a preset log processing model capable of automatically parsing and processing system logs at a low training cost. In practice, this may include the following:

[0168] S1: Transform human experience into a set of operable regular expressions or a set of finite automatic state machines.

[0169] S2: Combine the vocabulary to summarize the set of regular expressions (or the set of finite automatic state machines) into a state transition tensor and perform CP decomposition of the tensor, thereby serving as the initialization (e.g., initial model) of the RNN network and training the model.

[0170] S3: Obtain log samples (e.g., target logs), analyze the log samples using the model, and output results (e.g., target processing results). The output results may take the form of, but are not limited to: a weighted finite automaton corresponding to the sample (e.g., a weighted finite automaton corresponding to the target log), a sample summary as a conclusion (e.g., a text summary result corresponding to the target log), and the category of sample anomalies (e.g., the log category of the target log).

[0171] When executing S1, considering that log analysis can generally be summarized as searching among a finite number of cases, log analysis experience can be summarized as a set / combination of several regular expressions. Alternatively, in the special case of zero samples (or very few samples), the judgment process of the project code can be summarized to form a set / combination of several finite automatic state machines. Generally speaking, there are no uniform and explicit requirements for the method of generating regular expressions. This specification only addresses the general case, i.e., the case with a small number of typical log samples, and will use the following steps for automated processing. The specific implementation may include the following steps.

[0172] S1-1: When iterating through each log sample sentence by sentence (these samples are few but should be typical and have high discriminative power), the Text Rank method in Snow NLP is used to extract keywords from the log samples. Sentences with co-occurrence relationships between keywords are treated as edges connecting keywords, resulting in an initial set of keywords V. init The set of co-occurring edges E init The initial state diagram (e.g., sample initial state diagram): G init ={V init E init}

[0173] S1-2: Due to the co-occurrence edge set E in S1-1 init Since the edges are undirected, the main purpose of S1-2 is to transform them into directed edges. Specifically, the parse tree method in Snow NLP can be used to determine the direction between keywords based on the syntactic relationships between keywords in co-occurring sentences. At this point, the initial state graph G... init It was transformed into a directed state transition graph (e.g., a sample directed state graph): G init_row ={V init E init_row}

[0174] S1-3 (optional): For directed graph G init_row Community detection is performed to reduce the size of the graph by further dividing the directed graph into directed graphs composed of keyword communities (abstract state nodes) to reduce the computational complexity of the following steps.

[0175] When executing S2, it is necessary to consider the directed graph G. init_row It is obvious that V can be used init Each state node v in the equation is represented as a |V init |×|V init A matrix, where any element v i,j , indicating from state v i to state v j Does a possible state transition path exist? If so, denote it as 1; otherwise, denote it as 0. Continue this derivation to the entire vocabulary (e.g., the internal vocabulary). If the vocabulary length is |L|, then the directed state transition graph can be represented as a third-order tensor G. L×|V|×|V| And two state vectors that depend on the specific situation: the initial state vector. and final state vector α end Among them, a certain word L in the vocabulary list i Unique Corresponding V init One of them |V init |×|V init| matrix. Specifically, a path in a directed graph can be represented as:

[0176] Also considering the third-order tensor G L×|V|×|V| It still has too many parameters, so the third-order tensor G can be used. L×|V|×|V| Decomposed into three matrices: E_In R (|L|×R) (e.g., internal word vector matrix), D1(|V) init |×R)(e.g., the first state transition matrix) and D2(|V init |×R)(e.g., the second state transition matrix). Where R is a hyperparameter, typically a value that can be specified in the CP decomposition, and is less than the rank of the original third-order tensor. Where E_In R This can be viewed as the internal word vector matrix after tensor decomposition. When an external word vector matrix E_Out exists... O At this time, a transformation matrix G(O×R) can be used to transform the external word vector u∈E_Out. o By mapping from O-dimensional space to R-dimensional space, corresponding external word vectors can be introduced.

[0177] Furthermore, considering the formula for recurrent neural networks, the GRU network can be modified according to the following formula:

[0178]

[0179] z t =βw t +(1-β)u t G

[0180] f t =sigmoid(W f z t +U f h t-1 +b f )

[0181] r t =sigmoid(W r z t +U r h t-1 +b r )

[0182]

[0183]

[0184]

[0185]

[0186] Where h0 represents the model input at time 0, Let z represent the initial state vector. t w represents the fused word vectors. t Represents the internal word vector, u t Let G represent the external word vectors, G represent the transformation matrix from external word vectors to internal word vectors, β represent the hyperparameters, and f represent the vectors. t This represents the reset gate output at time t, r t b represents the update gate output at time t. f b r These represent the corresponding gate parameters, W. f U f W r , and U r h represents the weight matrix to be learned. t-1 h represents the model output at time t-1. t-1 Let t represent the model output at time t, a represent intermediate data, D1 represent the first state transition matrix, and D2 represent the second state transition matrix.

[0187] Based on the above formula, after modifying the GRU network accordingly, the internal and external word vectors obtained from tensor decomposition can be fused. Then, the outputs of the GRU reset and update gates are updated. Next, the current memory content is updated using the decomposed state transition matrices D1 and D2. Finally, the final memory output at time t is calculated. Therefore, through the above modifications, the number of parameters that need to be learned is significantly less than that of the original GRU gated network, thus allowing for learning with fewer samples. Furthermore, due to the tensor decomposition process, when b... f and b r As the value approaches 1, the modified GRU degenerates into a network controlled purely by a state machine, thus resulting in a higher lower bound and greater training potential.

[0188] When executing S3, even though the model has been trained, it can still be represented by the three matrices E_In after training. R D1 and D2 reconstruct the third-order tensor G' L×|V|×|V| However, the element values ​​in the original tensor are no longer 0 / 1, but rather a weighted finite automaton. By setting a threshold, the weights are transformed back into 0 / 1 values, resulting in a human-understandable automaton. Therefore, when the model processes log sample categories, it can not only obtain the sample category, but also, based on the transformed automaton, output sentences containing state transition keywords according to the sentence order in the text. Combined with Snow NLP's text summarization module, an analysis report can be generated, helping operations and maintenance personnel quickly understand the cause of the problem and thus quickly resolve system anomalies.

[0189] Through the above scenario examples, the log data processing method provided in this manual is verified. First, it extracts keywords automatically, allowing humans to add more robust formal experiences in a more flexible way. Second, by modifying the recurrent neural network based on tensor decomposition, the number of training parameters is greatly reduced, making the model more suitable for few-shot or zero-shot learning scenarios. It also allows the introduction of external word vectors to enrich the model's perceptual capabilities and improve its upper limit. Furthermore, it makes the model's output more diverse and interpretable, enabling maintenance personnel to efficiently and accurately determine the cause and location of system anomalies, and thus resolve system anomalies in a timely and effective manner. This effectively solves the following problems existing in current methods: high labor costs and inability to demonstrate the logic of error outbreaks; large number of parameters and computational burden when using recurrent neural network-based models; and inability to handle few-shot or zero-shot scenarios.

[0190] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0191] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0192] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0193] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0194] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0195] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A log data processing method, characterized in that, include: Obtain the target logs of the target system; The target log is processed using a pre-defined log processing model to obtain the target processing result; wherein the pre-defined log processing model is pre-trained based on the modified GRU network; the modified GRU network is constructed based on the tensor decomposition result of the sample directed state graph; Based on the target processing results, determine whether there is an anomaly in the target system; and if it is determined that there is an anomaly in the target system, determine the location and cause of the anomaly based on the target processing results. The training process of the preset log processing model includes: Obtain a directed state graph of the sample; wherein, the directed state graph of the sample is used to reflect the transformation relationship between semantic keywords of different system states of the sample system based on logs; Based on the internal vocabulary, the directed state graph of the sample is converted into the corresponding third-order tensor, initial state vector and final state vector; Tensor decomposition is performed on the third-order tensor to obtain the internal word vector matrix, the first state transition matrix, and the second state transition matrix; Based on the external vocabulary, an external word vector matrix that matches the internal word vector matrix is ​​obtained through spatial mapping; The initial state vector, final state vector, internal word vector matrix, external word vector matrix, first state transition matrix, and second state transition matrix are determined as the tensor decomposition result; Based on the tensor decomposition results, the GRU network is modified to obtain the initial model; Train the initial model to obtain a pre-defined log processing model that meets the requirements.

2. The method according to claim 1, characterized in that, Obtain the directed state graph of the samples related to the sample logs, including: Check for the existence of sample logs related to the sample system; If no sample logs for the sample system are found, obtain the project code of the sample system; The project code was parsed and organized to obtain a set of multiple finite automatic state machines; Based on a set of multiple finite automatic state machines, generate corresponding sample directed state diagrams.

3. The method according to claim 2, characterized in that, After detecting the existence of sample logs for the sample system, the method further includes: If it is determined that sample logs exist for the sample system, obtain the sample logs; The sample log is traversed sentence by sentence to obtain and utilize the keywords in the sample log, as well as the co-occurrence statements between the keywords, to construct the initial state diagram of the sample. Obtain and determine the direction between keywords based on the syntactic relations of keywords in co-occurring sentences; Based on the direction between keywords, the initial state diagram of the sample is corrected into the corresponding directed state diagram of the sample.

4. The method according to claim 1, characterized in that, After obtaining the directed state graph of samples related to the sample logs, the method further includes: Community detection is performed on the directed state graph of the samples to construct an improved directed state graph of the samples composed of keyword communities.

5. The method according to claim 1, characterized in that, Based on the tensor decomposition results, the GRU network is modified to obtain the initial model, including: The internal word vector matrix is ​​fused with the external word vector matrix to obtain the fused word vector matrix; Based on the fused word vector matrix, initial state vector, and final state vector, modify the reset gate and update gate in the GRU network; and based on the first state transition matrix and the second state transition matrix, modify the output in the GRU network to obtain the initial model.

6. The method according to claim 1, characterized in that, Train the initial model to obtain a pre-defined log processing model that meets the requirements, including: Input the sample logs into the initial model to obtain the corresponding sample processing results; Based on the sample logs and sample processing results, the weight matrix in the initial model is adjusted and determined to obtain a preset log processing model that meets the requirements.

7. The method according to claim 1, characterized in that, Based on the target processing results, determine whether there are any anomalies in the target system, including: Based on the target processing results, determine the log category of the target log; If the target log is determined to be of the specified log category, then an anomaly is identified in the target system.

8. The method according to claim 1, characterized in that, If the target log is of a specified log category, the target processing result further includes: a weighted finite automatic state machine corresponding to the target log, and / or a text summary result corresponding to the target log.

9. The method according to claim 8, characterized in that, If the target log is determined to be of a specified log category, the method further includes: Based on the target processing results, an anomaly analysis report about the target system is generated; wherein, the anomaly analysis report includes at least the location and / or cause of the anomaly in the target system; The abnormal prompts and the abnormal analysis report about the target system are sent to the maintenance terminal.

10. A log data processing apparatus, characterized in that, include: The acquisition module is used to acquire the target logs of the target system; The processing module is used to process the target log using a preset log processing model to obtain the target processing result; wherein, the preset log processing model is pre-trained based on the modified GRU network; the modified GRU network is constructed based on the tensor decomposition result of the sample directed state graph; The determination module is used to determine whether there is an anomaly in the target system based on the target processing results; and if it is determined that there is an anomaly in the target system, it determines the location and cause of the anomaly based on the target processing results. The training process of the preset log processing model includes: Obtain a directed state graph of the sample; wherein, the directed state graph of the sample is used to reflect the transformation relationship between semantic keywords of different system states of the sample system based on logs; Based on the internal vocabulary, the directed state graph of the sample is converted into the corresponding third-order tensor, initial state vector and final state vector; Tensor decomposition is performed on the third-order tensor to obtain the internal word vector matrix, the first state transition matrix, and the second state transition matrix; Based on the external vocabulary, an external word vector matrix that matches the internal word vector matrix is ​​obtained through spatial mapping; The initial state vector, final state vector, internal word vector matrix, external word vector matrix, first state transition matrix, and second state transition matrix are determined as the tensor decomposition result; Based on the tensor decomposition results, the GRU network is modified to obtain the initial model; Train the initial model to obtain a pre-defined log processing model that meets the requirements.

11. A log data processing method, characterized in that, include: Obtain a directed state graph of the sample; wherein, the directed state graph of the sample is used to reflect the transformation relationship between semantic keywords of different system states of the sample system based on logs; Based on the internal vocabulary, the directed state graph of the sample is converted into the corresponding third-order tensor, initial state vector and final state vector; Tensor decomposition is performed on the third-order tensor to obtain the internal word vector matrix, the first state transition matrix, and the second state transition matrix; Based on the external vocabulary, an external word vector matrix that matches the internal word vector matrix is ​​obtained through spatial mapping; The initial state vector, final state vector, internal word vector matrix, external word vector matrix, first state transition matrix, and second state transition matrix are determined as the tensor decomposition result; Based on the tensor decomposition results, the GRU network is modified to obtain the initial model; An initial model is trained to obtain a preset log processing model that meets the requirements; wherein, the preset log processing model is used to process the target logs of the target system to determine whether there are any anomalies in the target system.

12. A server, characterized in that, It includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method according to any one of claims 1 to 9, or 11.

13. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9 or 11.

Images