Network operation and maintenance anomaly detection method and system based on multi-modal information dynamic extraction

Abstract

The application provides a network operation and maintenance anomaly detection method and system based on multi-modal information dynamic extraction, and relates to the technical field of network operation and maintenance. By constructing a network operation and maintenance knowledge graph, storing multi-modal data of network devices, application programs and other related information, and the relationship of the multi-modal data, obtaining multi-modal information of network devices, application programs and other related information, and preprocessing the obtained multi-modal information, learning the representation features of the multi-modal information, constructing a multi-modal information anomaly detection model, using an extraction algorithm from coarse granularity to fine granularity to dynamically extract the network operation and maintenance knowledge graph, obtaining the first M knowledge triplets from the network operation and maintenance knowledge graph, and learning the global representation features of the triplets, the representation features of the multi-modal information and the global representation features of the triplets are sequentially combined and input into the multi-modal information anomaly detection model for anomaly detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network operation and maintenance, in particular to a network operation and maintenance anomaly detection method and system based on multi-modal information dynamic extraction. BACKGROUND

[0002] With the rapid development of information technology, the scale and structure of the network are constantly expanding, and the network has been integrated into various fields of production and life in various forms, becoming an important channel for information exchange and transmission. With the widespread application of industrial internet, it is crucial to ensure its service quality and network performance. However, the industrial internet network is huge in scale, with a large number of devices, complex network structure, and a variety of faults, and devices are interconnected, resulting in faults that are not independent. Traditional operation and maintenance methods are difficult to quickly and accurately locate fault points, affecting production efficiency and benefits. Therefore, improving network performance, ensuring service quality and security, and quickly and accurately locating fault points are difficult problems to be solved in network operation and maintenance.

[0003] In the field of network operation and maintenance, anomaly detection is a key technology. In network operation and maintenance anomaly detection, multi-modal information can be used as auxiliary materials to improve detection accuracy and efficiency. Common multi-modal information includes real-time information, security event information, log information, and user behavior information. Real-time information refers to the running state information of network devices and services, including CPU, memory, network bandwidth, and disk space. Security event information refers to the record and analysis information of network security events, including intrusion, attack, virus detection and response information. Log information refers to the log record information of network devices and services, including system logs, application logs, and security logs. User behavior information refers to network user behavior information, including access records and operation records. Multi-modal information can provide rich auxiliary materials in network operation and maintenance anomaly detection to improve detection accuracy and efficiency. Through the fusion of multi-modal information, network status can be comprehensively monitored and managed to ensure normal operation of the network. Existing anomaly detection methods are mainly based on single modal information, such as anomaly detection methods based on single data sources such as logs. These methods can monitor and analyze network status, but often only analyze through a single dimension of data source, making it difficult to fully reflect the overall situation of network status, and lacking multi-dimensional information analysis capability, making it difficult to comprehensively and deeply analyze network status from multiple angles. SUMMARY

[0004] The present application provides a network operation and maintenance anomaly detection method and system based on multi-modal information dynamic extraction to solve the problem of analyzing only through a single dimension of data, lacking multi-dimensional information analysis capability, and being difficult to fully reflect the overall situation of network status, and being difficult to comprehensively and deeply analyze network status from multiple angles.

[0005] In a first aspect, the present application provides a network operation and maintenance anomaly detection method based on dynamic extraction of multi-modal information, comprising:

[0006] Constructing a network operation and maintenance knowledge graph, storing multi-modal data of network devices, application programs and other related information, and the relationship of the multi-modal data;

[0007] Obtaining multi-modal information of network devices, application programs and other related information, and preprocessing the obtained multi-modal information;

[0008] Learning the representation features of the multi-modal information;

[0009] Constructing a multi-modal information anomaly detection model;

[0010] Using extraction algorithms from coarse granularity to fine granularity to dynamically extract the network operation and maintenance knowledge graph, obtaining the first M knowledge triples from the network operation and maintenance knowledge graph, and learning the global representation features of the triples;

[0011] Combining the representation features of the multi-modal information and the global representation features of the triples in turn, and inputting them into the multi-modal information anomaly detection model for anomaly detection.

[0012] Based on the first aspect, learning the representation features of the multi-modal information comprises:

[0013] Inputting the multi-modal information into the BERT model to obtain a fixed-length vector representation;

[0014] Inputting the fixed-length vector representation into the multi-modal Transformer model to fuse different types of multi-modal information, and obtaining the representation features of the multi-modal information.

[0015] Based on the first aspect, the representation features q w of the multi-modal information are defined as follows:

[0016] q w =Transformer(f e (q)),

[0017] In the formula: f e represents a model BERT for converting a string into a word vector; l q represents the maximum length of the multi-modal information.

[0018] Based on the first aspect, the multi-modal information anomaly detection model comprises:

[0019] input the multi-modal information representation feature into the multi-modal information anomaly detection model to obtain a prediction result of the multi-modal information determine whether the multi-modal information is abnormal based on the prediction result

[0020] The prediction result includes an abnormal prediction label and a normal prediction label. The abnormal prediction label and the normal prediction label are defined as follows:

[0021]

[0022] In the formula, θ MLP is a multilayer perceptron that generates a final abnormal prediction label, q w is a multi-modal information representation feature.

[0023] The multi-modal information anomaly detection model is trained using a loss function, which is defined as follows:

[0024]

[0025] In the formula, is a binary cross-entropy loss function used for training the multi-modal information anomaly detection model, is a prediction result, and y is a true result.

[0026] Based on the first aspect, the multi-modal information anomaly detection model is used to dynamically extract the network operation and maintenance knowledge graph from the network operation and maintenance knowledge graph to obtain the first M knowledge triples, and learn the global representation feature of the triples, including:

[0027] In the coarse-grained stage, the multi-modal information anomaly detection model is used to calculate the information gain of the subgraph in the network operation and maintenance knowledge graph for multi-modal information anomaly detection, and the first K subgraphs are selected from the subgraphs of the network operation and maintenance knowledge graph.

[0028] In the fine-grained stage, the multi-modal information anomaly detection model is used to calculate the information gain of the triples in the first K subgraphs for multi-modal information anomaly detection, and the first M triples are selected from the first K subgraphs.

[0029] Based on the first aspect, in the coarse-grained stage, the multi-modal information anomaly detection model is used to calculate the information gain of the subgraph in the network operation and maintenance knowledge graph for multi-modal information anomaly detection, and the first K subgraphs are selected from the subgraphs of the network operation and maintenance knowledge graph, including:

[0030] Obtain the representation feature of each subgraph in the network operation and maintenance knowledge graph.

[0031] The representation feature of each subgraph is averaged and aggregated to obtain the global representation feature of each subgraph.

[0032] inputting the global representation feature of each subgraph into the multi-modal information anomaly detection model, and calculating information gain of multi-modal information anomaly detection of each subgraph;

[0033] selecting the top K subgraphs in the subgraphs of the network operation knowledge graph based on the information gain of multi-modal information anomaly detection of each subgraph.

[0034] According to the first aspect, the information gain g(q, s i ) of multi-modal information anomaly detection of each subgraph is defined as follows:

[0035]

[0036] In the formula: q g is a global representation feature of multi-modal information, is a global representation feature of subgraph s i , AP is an average pooling process, s i is a representation feature of subgraph, H(q g ) represents the entropy of multi-modal information q, is the conditional entropy of multi-modal information q anomaly detection under the condition of given subgraph s i .

[0037] According to the first aspect, in the fine-grained stage, the information gain of multi-modal information anomaly detection of the top M triplets in the top K subgraphs is calculated by using the multi-modal information anomaly detection model, and the top M triplets are selected in the top K subgraphs, which includes:

[0038] obtaining the representation feature of each triplet in the top K subgraphs;

[0039] performing average pooling processing on the representation feature of each triplet to obtain the global representation feature of each triplet;

[0040] inputting the global representation feature of each triplet into the multi-modal information anomaly detection model, and calculating the information gain of multi-modal information anomaly detection of each triplet;

[0041] selecting the top M triplets in the top K subgraphs based on the information gain of multi-modal information anomaly detection of each triplet.

[0042] According to the first aspect, the representation feature of the multi-modal information and the global representation feature of the triplet are sequentially combined and input into the multi-modal information anomaly detection model for anomaly detection, which includes:

[0043] dividing the multi-modal information into N sub-multi-modal information blocks;

[0044] The N sub-multimodal information blocks are sequentially combined with the global representation features of the triplets and input into the multimodal information anomaly detection model for anomaly detection.

[0045] In a second aspect, the embodiments of the present application provide a network operation and maintenance anomaly detection system based on dynamic extraction of multimodal information, which comprises:

[0046] A first construction module is configured to construct a network operation and maintenance knowledge graph, store multimodal data of network devices, application programs and other related information, and relationships of the multimodal data;

[0047] A obtaining module is configured to obtain multimodal information of network devices, application programs and other related information, and pre-process the obtained multimodal information;

[0048] A learning module is configured to learn representation features of the multimodal information;

[0049] A second construction module is configured to construct a multimodal information anomaly detection model;

[0050] An extraction module is configured to use extraction algorithms from coarse granularity to fine granularity to dynamically extract the network operation and maintenance knowledge graph, obtain the first M knowledge triplets from the network operation and maintenance knowledge graph, and learn global representation features of the triplets;

[0051] A detection module is configured to sequentially combine the representation features of the multimodal information and the global representation features of the triplets, and input them into the multimodal information anomaly detection model for anomaly detection.

[0052] The present application has the following advantages:

[0053] The embodiments of the present application provide a network operation and maintenance anomaly detection method and system based on dynamic extraction of multimodal information. By constructing a network operation and maintenance knowledge graph, storing multimodal data of network devices, application programs and other related information, and relationships of the multimodal data, obtaining multimodal information of network devices, application programs and other related information, and pre-processing the obtained multimodal information, learning representation features of the multimodal information, constructing a multimodal information anomaly detection model, using extraction algorithms from coarse granularity to fine granularity to dynamically extract the network operation and maintenance knowledge graph, obtaining the first M knowledge triplets from the network operation and maintenance knowledge graph, and learning global representation features of the triplets, the representation features of the multimodal information and the global representation features of the triplets are sequentially combined and input into the multimodal information anomaly detection model for anomaly detection.

[0054] The present application can make full use of various types of information for anomaly detection, including logs, performance indicators, configuration files, alarm information, user behavior data and various modalities; using multi-modal information for anomaly detection can reflect the network state from multiple angles, and comprehensively understand the network state, thereby improving the accuracy and efficiency of anomaly detection;

[0055] The present application is based on a dynamic extraction algorithm, which can give multi-modal information blocks of anomalies, so that operation and maintenance personnel can quickly locate and solve problems, and avoid losses caused by abnormal network operation. At the same time, the present application also combines multi-modal network operation knowledge graph, which can give interpretable knowledge, thereby better helping operation and maintenance personnel to understand and analyze the causes and roots of abnormal problems;

[0056] The present application can timely capture abnormal conditions in network operation, avoid missed detection problems caused by data lag. In addition, the present application also has high efficiency and scalability, which can adapt to network operation environments of different scales and complexities, and can continuously improve its anomaly detection ability and accuracy through continuous iteration and updating;

[0057] The network operation and maintenance anomaly detection method and system based on multi-modal information dynamic extraction provided by the present application have multi-modal information multi-dimensional analysis, abnormal point positioning, abnormal explainability, dynamic capture, high efficiency and scalability, and can effectively solve abnormal problems in network operation and maintenance. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A flowchart of a network operation and maintenance anomaly detection method based on multi-modal information dynamic extraction according to an embodiment of the present application;

[0059] Figure 2 A schematic diagram of a network operation and maintenance anomaly detection method based on multi-modal information dynamic extraction according to an embodiment of the present application;

[0060] Figure 3 An architecture diagram of a network operation and maintenance anomaly detection system based on multi-modal information dynamic extraction according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0062] At present, only single-dimensional data sources are analyzed, it is difficult to comprehensively reflect the overall situation of the network state, and there is also a lack of multi-dimensional information analysis capability, it is difficult to comprehensively and deeply analyze the network state from multiple angles. Based on this, the application provides a network operation and maintenance anomaly detection method and system based on multi-modal information dynamic extraction, analyzes multi-modal information and network operation and maintenance knowledge graph, comprehensively and deeply analyzes the network state from multiple angles, and accurately detects anomalies by using the advantages of multi-modal information. In addition, the network operation and maintenance anomaly detection method based on multi-modal information dynamic extraction not only accurately gives whether an anomaly occurs in the network operation and maintenance process, but also constructs a network operation and maintenance knowledge graph, and provides explainable knowledge of the anomaly based on dynamic extraction of multi-modal information and the knowledge graph.

[0063] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0064] In a first aspect, the embodiments of the present application provide a network operation and maintenance anomaly detection method based on multi-modal information dynamic extraction. Please refer to Figure 1 , Figure 1 A network operation and maintenance anomaly detection method based on multi-modal information dynamic extraction is shown, which includes the following steps:

[0065] Step S110: Construct a network operation and maintenance knowledge graph, store multi-modal data of network devices, application programs and other related information, and the relationship of the multi-modal data;

[0066] Among them, the above-mentioned construction of network operation and maintenance knowledge graph, first needs to determine the entity (such as device, application program, configuration file, log, performance index, alarm) and relationship (topological relationship, reference relationship, time relationship) type of knowledge graph; In the construction process, multi-modal data needs to be obtained from various sources (such as network devices, systems, logs, etc.), and data cleaning, structured processing and feature extraction technologies are used to convert the above multi-modal data into nodes and edges of the knowledge graph. Finally, the nodes and edges of the above knowledge graph are stored in the knowledge graph database Neo4j to establish the relationship and attribute between entities.

[0067] With the continuous updating and evolution of network operation and maintenance data, the knowledge graph also needs to be regularly maintained and updated, including adding new entities and relationship types, updating the attributes of nodes and edges, etc., to ensure the accuracy and reliability of the data in the network operation and maintenance knowledge graph.

[0068] Step S120: Obtain network devices, application programs and other related multi-modal information, and pre-process the obtained multi-modal information;

[0069] The multi-modal information mainly includes real-time information, security event information, log information, user behavior information, etc. The real-time information refers to the running state information of network devices and services, including CPU, memory, network bandwidth, disk space, etc. The security event information refers to the record and analysis information of network security events, including intrusion, attack, virus, etc. The log information refers to the log record information of network devices and services, including system log, application log, security log, etc. The user behavior information refers to the behavior information of network users, including access record, operation record, etc. The multi-modal information can provide rich auxiliary materials in the abnormality detection process of network operation and maintenance, so as to improve the accuracy and efficiency of abnormality detection.

[0070] The preprocessing of the obtained multi-modal information refers to cleaning, structuring and feature extraction, etc. of the obtained multi-modal information, aiming at arranging the chaotic multi-modal information into ordered structured information, so as to facilitate the subsequent dynamic extraction of the network operation and maintenance knowledge graph.

[0071] Step S130: learning the representation feature of the multi-modal information;

[0072] In the embodiment of the application, the Transformer model is used to learn the z-dimensional representation feature q of the multi-modal information w , which is defined as follows:

[0073] q w =Transformer(f e (q)),

[0074] In the formula, f e represents the BERT model for converting a string into a word vector; l q represents the maximum length of the multi-modal information; the multi-modal information is input into the BERT model, and the fixed-length vector representation of the multi-modal information is obtained through the BERT model output, and then the vector representation of the multi-modal information is input into the multi-modal Transformer model, aiming at fusing different types of multi-modal information; the multi-modal Transformer model adds a special mark, such as [CLS] and [SEP], at the beginning and end of each vector sequence, aiming at helping the multi-modal Transformer model to segment the sequence input into the model.

[0075] In the embodiments of the present application, a multi-modal Transformer model is preferably used for multi-modal information representation feature learning, because the multi-modal Transformer model adopts an architecture similar to that of a single-modal Transformer, including multi-head self-attention mechanism and feed-forward network; therefore, the multi-modal Transformer model can fuse different types of information.

[0076] For example, an independent query, key and value are allocated for each input type in each attention head, and then the outputs of each attention head are weighted and summed to obtain a comprehensive feature representation q w . Specifically, for the i-th input type, the attention weight of the i-th input type is calculated i，j , and then a weighted sum vector z j is obtained:

[0077]

[0078] where N represents the number of input types, h i，j represents the feature vector of the i-th input type at the j-th time step, and i，j is the attention weight of the i-th input type at the j-th time step.

[0079] where the attention weight of the i-th input type at the j-th time step is i，j , which is defined as follows:

[0080]

[0081] where q i，j is the query of the i-th input type at the j-th time step, k i，j is the key of the i-th input type at the j-th time step, and q i，j and k i，j can be obtained through linear transformation, and T is the matrix transpose.

[0082] Then, the weighted sum vector z j is taken as the feature vector of the j-th time step of the output sequence and input to the feed-forward network for processing; finally, the feature vectors of all time steps are spliced to obtain the z-dimensional representation feature of the multi-modal information

[0083] Step S140: constructing a multi-modal information anomaly detection model;

[0084] wherein the construction of the multi-modal information anomaly detection model comprises:

[0085] input the multi-modal information representation feature into the multi-modal information anomaly detection model to obtain a prediction result of the multi-modal information determine whether the multi-modal information is abnormal based on the prediction result

[0086] The prediction result includes a final abnormal prediction label. The final abnormal prediction label is defined as follows:

[0087]

[0088] In the formula, θ MLP is a multilayer perceptron that generates a final abnormal prediction label, q w is a multi-modal information representation feature.

[0089] The multi-modal information anomaly detection model is trained using a loss function, which is defined as follows:

[0090]

[0091] In the formula, is a binary cross-entropy loss function used for training the multi-modal information anomaly detection model, is a prediction result, and y is a true result.

[0092] input the multi-modal information representation feature into the multi-modal information anomaly detection model to obtain a prediction result of the multi-modal information use a binary cross-entropy loss function train the multi-modal information anomaly detection model, that is, use a binary cross-entropy loss function the prediction result output by the multi-modal information anomaly detection model and the true result y, so that the prediction result output by the multi-modal information anomaly detection model approaches the true result, and the prediction result output by the multi-modal information anomaly detection model is optimized. By using optimization, a standard gradient-based optimizer (such as SGD or Adam) can be used to optimize the multi-modal information anomaly detection model end-to-end.

[0093] Step S150: using an extraction algorithm from coarse granularity to fine granularity, dynamically extracting the network operation and maintenance knowledge graph, obtaining the first M knowledge triples from the network operation and maintenance knowledge graph, and learning the global representation feature of the triples;

[0094] The extraction algorithm from coarse granularity to fine granularity is used to dynamically extract the network operation and maintenance knowledge graph, obtain the first M knowledge triples from the network operation and maintenance knowledge graph, and learn the global representation feature of the triples, including:

[0095] In the coarse-grained stage, the information gain of the subgraphs in the network operation and maintenance knowledge graph for multimodal information anomaly detection is calculated using the multimodal information anomaly detection model, and the top K subgraphs are selected from the subgraphs in the network operation and maintenance knowledge graph.

[0096] In the fine-grained stage, the information gain of the triples in the first K subgraphs for multimodal information anomaly detection is calculated using the multimodal information anomaly detection model, and the first M triples are selected from the first K subgraphs.

[0097] In the coarse-grained stage, the information gain of subgraphs in the network operation and maintenance knowledge graph for multimodal information anomaly detection is calculated using the multimodal information anomaly detection model, and the top K subgraphs are selected from the subgraphs of the network operation and maintenance knowledge graph, including the following steps:

[0098] Step S210: Obtain the representation features of each subgraph in the network operation and maintenance knowledge graph;

[0099] Step S220: Perform average convergence processing on the representation features of each subgraph to obtain the global representation features of each subgraph;

[0100] Step S230: Input the global representation features of each subgraph into the multimodal information anomaly detection model, and calculate the information gain of each subgraph for multimodal information anomaly detection;

[0101] Step S240: Based on the information gain of each subgraph for multimodal information anomaly detection, select the top K subgraphs in the network operation and maintenance knowledge graph.

[0102] In step S230, the information gain g(q, s) of each subgraph for multimodal information anomaly detection is... i ), defined as follows:

[0103]

[0104] In the formula: q g For global representation features of multimodal information, For subgraph s i Global representation features, AP is for average convergence processing, s i For the representational features of the subgraph, H(q) g ) represents the entropy of multimodal information q. Given a subgraph s i The conditional entropy of multimodal information q anomaly detection under the condition of H(q) g The q represents the entropy of the multimodal information, i.e., the uncertainty of the problem. Conditional entropy of multimodal information q abnormality detection given a subgraph, that is, uncertainty of multimodal information abnormality given a subgraph; the greater the information gain g(q, s i , the greater the degree of reducing uncertainty of abnormality.

[0105] Wherein, the global representation feature of each subgraph is defined as follows:

[0106]

[0107] In the formula, s i is the representation feature of each subgraph, and AP is average pooling; in the embodiment of the application, in order to match the input dimension of the subsequent module, preferably, the average pooling (AP) is performed on s i using a kernel with a size of 2 to obtain the global representation feature of the subgraph. Exemplarily, the calculation process of H(q g ) is as follows:

[0108] H(q g ) = E [-log(p(q g ))],

[0109]

[0110] In the formula, E represents the expectation operator; sigma is a sigmoid function, which is used to map the output result to the interval [0, 1]; W, and all are parameter matrices to be learned.

[0111] Wherein, is defined as follows:

[0112]

[0113]

[0114] In the formula, || is a concatenation operator; and AP is an average pooling operation with a size of 2.

[0115] Wherein, in the fine-grained stage, the information gain of the multimodal information abnormality detection of the triplets in the first K subgraphs is calculated by using the multimodal information abnormality detection model, and the first M triplets are selected from the first K subgraphs, comprising:

[0116] Obtaining the representation feature of each triplet in the first K subgraphs;

[0117] averaging and converging the representation features of each triple to obtain global representation features of each triple;

[0118] inputting the global representation features of each triple into the multi-modal information anomaly detection model to calculate information gain of multi-modal information anomaly detection of each triple;

[0119] selecting the first M triples from the first K subgraphs based on the information gain of multi-modal information anomaly detection of each triple.

[0120] In the embodiment of the application, the first K subgraphs selected have multiple triples; in the fine-grained stage, an algorithm similar to the above coarse-grained stage is used to calculate the information gain of multi-modal information anomaly detection of triples in the knowledge graph by using the network operation and maintenance anomaly detection model selecting the first M triples from all triples; specifically, in the fine-grained stage, first obtain the feature t i of each triple, calculate the information gain of multi-modal information q anomaly detection of the triple by using the network operation and maintenance anomaly detection model When the information gain of multi-modal information anomaly detection of the triple is greater than or equal to a set threshold τ, the triple t i is considered as an interpretable knowledge for anomaly detection.

[0121] Step S160: sequentially combine the representation features of the multi-modal information and the global representation features of the triples, and input them into the multi-modal information anomaly detection model for anomaly detection.

[0122] sequentially combining the representation features of the multi-modal information and the global representation features of the triples, and inputting them into the multi-modal information anomaly detection model for anomaly detection, comprising:

[0123] divide the multi-modal information into N sub-multi-modal information blocks;

[0124] sequentially combine the N sub-multi-modal information blocks and the global representation features of the triples, and input them into the multi-modal information anomaly detection model for anomaly detection.

[0125] In steps S110-S150, the prediction result of whether the multi-modal information is abnormal has been obtained. Therefore, the purpose of step S160 is to accurately locate the abnormal point where the anomaly occurs. In order to achieve this purpose, in step S160, the multi-modal information q is divided into several blocks q i , and the interpretable knowledge t obtained in step 150 is combined as a combination, input into the network operation and maintenance anomaly detection model, and anomaly detection is performed again to obtain the abnormal point where the anomaly occurs.

[0126] Specifically, in the embodiments of the present application, first, the representation features of each multi-modal information block q i , and the global representation features of the explainable knowledge t are obtained Then, according to the method in step S130, the representation features of the multi-modal information and the explainable knowledge are combined to obtain the multi-modal information representation feature for anomaly detection; these representation results are input into a multi-layer perception with binary cross-entropy loss to obtain the probability of whether each multi-modal information block is abnormal, and when the probability is greater than or equal to a prediction threshold P, it is considered that the multi-modal information block will cause an anomaly; in combination with the prediction result of whether the multi-modal information is abnormal that has been determined in the previous step, the specific abnormal points in the multi-modal information can be accurately located.

[0127] Based on the same inventive concept as described above, in the embodiments of the present application, the present application also provides a schematic diagram of a network operation and maintenance anomaly detection method based on dynamic extraction of multi-modal information; please refer to Figure 2 , Figure 2 The schematic diagram is a schematic diagram of a network operation and maintenance anomaly detection method based on dynamic extraction of multi-modal information according to an embodiment of the present application, which shows the specific implementation process of the network operation and maintenance anomaly detection method based on dynamic extraction of multi-modal information described above. Exemplarily, the network operation and maintenance anomaly detection method based on dynamic extraction of multi-modal information mainly consists of three main modules, namely a multi-modal information representation module, an explainable knowledge extraction module, and an anomaly detection module; the present application first learns the representation features of the multi-modal information and the network operation and maintenance knowledge graph through the multi-modal information representation module, then extracts the subgraph related to the multi-modal information in the network operation and maintenance knowledge graph in the coarse-grained stage and selects the knowledge triple related to the multi-modal information in the extracted subgraph as the explainable knowledge for this anomaly detection in the fine-grained stage, and then combines the multi-modal information block for the final anomaly detection in the anomaly detection module. The anomaly detection module will obtain the probability of the occurrence of the anomaly of the multi-modal information block, and if the probability is greater than or equal to a prediction threshold P, it is considered that the multi-modal information block will cause an anomaly, and the network operation and maintenance anomaly detection method based on dynamic extraction of multi-modal information described above will finally output the multi-modal information block and the explainable knowledge that appear abnormal.

[0128] In the implementation process, the network operation and maintenance knowledge graph is constructed to store multi-modal data of network devices, application programs and other related information, and relationships of the multi-modal data; multi-modal information of network devices, application programs and other related information is acquired, and the acquired multi-modal information is preprocessed; representation features of the multi-modal information are learned; a multi-modal information anomaly detection model is constructed; a coarse-to-fine extraction algorithm is used to dynamically extract the network operation and maintenance knowledge graph, to obtain the first M knowledge triplets from the network operation and maintenance knowledge graph, and to learn global representation features of the triplets; and the representation features of the multi-modal information and the global representation features of the triplets are sequentially combined and input into the multi-modal information anomaly detection model for anomaly detection. Based on this, the network operation and maintenance anomaly detection method based on dynamic extraction of multi-modal information is provided, multi-modal information and network operation and maintenance knowledge graph are analyzed, network states are comprehensively and deeply analyzed from multiple angles, accurate anomaly detection is performed by using the advantages of multi-modal information, and in addition, the method not only accurately indicates whether an anomaly occurs, but also constructs a network operation and maintenance knowledge graph, and provides explainable knowledge of the anomaly occurrence based on dynamic extraction of multi-modal information and the knowledge graph.

[0129] Based on the same inventive concept, the application also provides an architecture diagram of a network operation and maintenance anomaly detection system based on dynamic extraction of multi-modal information, please refer to Figure 3 , Figure 3 The architecture diagram of the network operation and maintenance anomaly detection system based on dynamic extraction of multi-modal information provided by the embodiment of the application, the system comprises:

[0130] The first construction module 110 is configured to construct a network operation and maintenance knowledge graph, store multi-modal data of network devices, application programs and other related information, and relationships of the multi-modal data.

[0131] The acquisition module 120 is configured to acquire multi-modal information of network devices, application programs and other related information, and pre-process the acquired multi-modal information.

[0132] The learning module 130 is configured to learn representation features of the multi-modal information.

[0133] The second construction module 140 is configured to construct a multi-modal information anomaly detection model.

[0134] The extraction module 150 is configured to use a coarse-to-fine extraction algorithm to dynamically extract the network operation and maintenance knowledge graph, to obtain the first M knowledge triplets from the network operation and maintenance knowledge graph, and to learn global representation features of the triplets.

[0135] The detection module 160 is configured to sequentially combine the representation features of the multi-modal information and the global representation features of the triplets, and input into the multi-modal information anomaly detection model for anomaly detection.

[0136] In the implementation process, the network operation and maintenance knowledge graph is constructed by the first construction module 110, multi-modal data of network devices, application programs and other related information, and the relationship of the multi-modal data are stored, the multi-modal information of network devices, application programs and other related information is collected by the acquisition module 120, and the collected multi-modal information is preprocessed, the representation features of the multi-modal information are learned by the learning module 130, the multi-modal information anomaly detection model is constructed by the second construction module 140, the network operation and maintenance knowledge graph is dynamically extracted by the extraction module 150 from coarse granularity to fine granularity, the first M knowledge triplets are obtained from the network operation and maintenance knowledge graph, and the global representation features of the triplets are learned, and the detection module 160 sequentially combines the representation features of the multi-modal information and the global representation features of the triplets, and inputs into the multi-modal information anomaly detection model for anomaly detection. Based on this, the present application provides a network operation and maintenance anomaly detection method based on multi-modal information dynamic extraction, which analyzes multi-modal information and network operation and maintenance knowledge graph, comprehensively and deeply analyzes the network state from multiple angles, accurately detects the anomaly by using the advantages of multi-modal information, and in addition, the method not only accurately gives whether an anomaly occurs, but also constructs a network operation and maintenance knowledge graph, and provides explainable knowledge of the anomaly based on dynamic extraction of multi-modal information and the knowledge graph.

[0137] In the embodiments of the present application, it should be understood that the disclosed system and method can also be implemented in other ways. The device embodiments described above are only schematic. For example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the devices, methods and computer program products according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from those described in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0138] In addition, each function module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0139] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0140] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, but can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the involved claims.

Claims

1. A network operation and maintenance anomaly detection method based on dynamic extraction of multimodal information, characterized in that, The method includes: Construct a network operation and maintenance knowledge graph to store multimodal data of network devices, applications and other related information, as well as the relationships between the multimodal data; Acquire network devices, applications, and other relevant multimodal information, and preprocess the acquired multimodal information; Learning the representation features of the multimodal information includes: inputting the multimodal information into a BERT model to obtain a fixed-length vector representation; inputting the fixed-length vector representation into a multimodal Transformer model to fuse different types of multimodal information to obtain the representation features of the multimodal information; Construct a multimodal information anomaly detection model; An extraction algorithm from coarse-grained to fine-grained is used to dynamically extract the network operation and maintenance knowledge graph, obtain the first M knowledge triples from the network operation and maintenance knowledge graph, and learn the global representation features of the triples. The representation features of the multimodal information and the global representation features of the triples are combined sequentially and input into the multimodal information anomaly detection model for anomaly detection; The construction of the multimodal information anomaly detection model includes: The multimodal information representation features are input into the multimodal information anomaly detection model to obtain the prediction results of the multimodal information. Based on the prediction results, determine whether anomalies have occurred in the multimodal information; Wherein, the prediction result The definition is as follows: ， In the formula It is a multilayer perceptron that generates the final anomaly prediction label. Features representing multimodal information; The multimodal information anomaly detection model is trained using a loss function, which is defined as follows: ， In the formula, It is the binary cross-entropy loss function used for training the multimodal information anomaly detection model. For the predicted results, This represents the actual result.

2. The network operation and maintenance anomaly detection method according to claim 1, characterized in that, The representation features of the multimodal information The definition is as follows: ， In the formula: , This refers to BERT, a model that converts strings into word vectors. This indicates the maximum length of the multimodal information.

3. The network operation and maintenance anomaly detection method according to claim 1, characterized in that, The method employs a coarse-grained to fine-grained extraction algorithm to dynamically extract from the network operation and maintenance knowledge graph, obtaining the first M knowledge triples from the knowledge graph, and learning the global representation features of the triples, including: In the coarse-grained stage, the information gain of the subgraphs in the network operation and maintenance knowledge graph for multimodal information anomaly detection is calculated using the multimodal information anomaly detection model, and the top K subgraphs are selected from the subgraphs in the network operation and maintenance knowledge graph. In the fine-grained stage, the information gain of the triples in the first K subgraphs for multimodal information anomaly detection is calculated using the multimodal information anomaly detection model, and the first M triples are selected from the first K subgraphs.

4. The network operation and maintenance anomaly detection method according to claim 3, characterized in that, In the coarse-grained stage, the multimodal information anomaly detection model is used to calculate the information gain of subgraphs in the network operation and maintenance knowledge graph for multimodal information anomaly detection. The top K subgraphs in the network operation and maintenance knowledge graph are selected, including: Obtain the representation features of each subgraph in the network operation and maintenance knowledge graph; The representation features of each subgraph are averaged and converged to obtain the global representation features of each subgraph; The global representation features of each subgraph are input into the multimodal information anomaly detection model, and the information gain of each subgraph for multimodal information anomaly detection is calculated. Based on the information gain of each subgraph for multimodal information anomaly detection, the top K subgraphs are selected from the subgraphs of the network operation and maintenance knowledge graph.

5. The network operation and maintenance anomaly detection method according to claim 4, characterized in that, The information gain of each subgraph for multimodal information anomaly detection The definition is as follows: ， In the formula: For global representation features of multimodal information, For subgraph Global representation features, AP stands for average convergence processing. The representational characteristics of a subgraph The entropy represents the multimodal information q. Given a subgraph Conditional entropy for multimodal information q anomaly detection under certain conditions.

6. The network operation and maintenance anomaly detection method according to claim 3, characterized in that, In the fine-grained stage, the multimodal information anomaly detection model is used to calculate the information gain of triples in the first K subgraphs for multimodal information anomaly detection. The first M triples are selected from the first K subgraphs, including: Obtain the representation features of each triple in the first K subgraphs; The representation features of each triplet are averaged and converged to obtain the global representation features of each triplet. The global representation features of each triple are input into the multimodal information anomaly detection model, and the information gain of each triple for multimodal information anomaly detection is calculated. Based on the information gain of each triplet for multimodal information anomaly detection, the first M triplets are selected from the first K subgraphs.

7. The network operation and maintenance anomaly detection method according to claim 1, characterized in that, The representation features of the multimodal information and the global representation features of the triples are sequentially combined and input into the multimodal information anomaly detection model for anomaly detection, including: The multimodal information is divided into N sub-multimodal information blocks; The N sub-multimodal information blocks are sequentially combined with the global representation features of the triples and input into the multimodal information anomaly detection model for anomaly detection.

8. A network operation and maintenance anomaly detection system based on dynamic extraction of multimodal information, characterized in that, The system includes: The first construction module is used to construct a network operation and maintenance knowledge graph, store multimodal data of network devices, applications and other related information, as well as the relationships between the multimodal data; The acquisition module is used to acquire network devices, applications and other related multimodal information, and to preprocess the acquired multimodal information; The learning module is used to learn the representation features of the multimodal information; specifically, it is used to input the multimodal information into the BERT model to obtain a fixed-length vector representation; and input the fixed-length vector representation into the multimodal Transformer model to fuse different types of multimodal information to obtain the representation features of the multimodal information. The second building module is used to build a multimodal information anomaly detection model; The extraction module is used to dynamically extract the network operation and maintenance knowledge graph using an extraction algorithm from coarse-grained to fine-grained, obtain the first M knowledge triples from the network operation and maintenance knowledge graph, and learn the global representation features of the triples. The detection module is used to sequentially combine the representation features of the multimodal information and the global representation features of the triples, and input them into the multimodal information anomaly detection model for anomaly detection; The second building module is specifically used for: The multimodal information representation features are input into the multimodal information anomaly detection model to obtain the prediction results of the multimodal information. Based on the prediction results, determine whether anomalies have occurred in the multimodal information; Wherein, the prediction result The definition is as follows: ， In the formula It is a multilayer perceptron that generates the final anomaly prediction label. Features representing multimodal information; The multimodal information anomaly detection model is trained using a loss function, which is defined as follows: ， In the formula, It is the binary cross-entropy loss function used for training the multimodal information anomaly detection model. For the predicted results, This represents the actual result.

Images