Network element association analysis method based on AMEE + algorithm

Abstract

The invention discloses a network element association analysis method based on an AMEE + algorithm. The method comprises the following steps: acquiring network element data; performing entity and relation extraction on the network element data, and constructing a network element knowledge graph; the triple data forming the network element knowledge graph are input into a rule learning model, an initial rule set is obtained, and the rule learning model is constructed based on an AMIE + algorithm; optimizing the initial rule set to obtain an optimal rule set; and reasoning the optimal rule set to obtain a reasoning result. By means of the method, management, operation and maintenance of a modern network can be better supported, and the overall performance and stability of the network are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of data mining and network management, and in particular relates to a network element association analysis method based on AMIE+ algorithm. Background Art

[0002] With the rapid development of information and communication technology, the scale and complexity of networks are increasing, and the various network elements (such as switches, routers, base stations, servers, etc.) in the network are becoming more complex and diverse. These network elements play a vital role in modern networks to ensure the normal operation and stability of the network. However, with the increase in the number and types of network elements, the relationship between network elements has become extremely complex, which has brought new challenges to network operation and maintenance, fault management, performance optimization, etc.

[0003] In existing network management and operation systems, the analysis of association relationships between network elements usually relies on static rules or experience-based models. For example, traditional network management systems may perform association analysis based on the topological structure and historical fault data between network elements, and then perform fault location or performance optimization. This method has the following shortcomings:

[0004] Limited to fixed rules: Traditional methods usually rely on predefined rules or models and cannot dynamically adapt to changes in the network environment. In modern networks, the associations between network elements are dynamic and complex, and fixed rules cannot fully cover these associations, resulting in inaccurate or incomplete association analysis results.

[0005] Difficulty in processing incomplete data: Network data is often incomplete and may contain missing data, errors, or anomalies. Traditional association analysis methods usually assume that the data is complete, so they are less effective when faced with incomplete data and are difficult to provide accurate analysis results.

[0006] High computational complexity: As the network scale increases, the number of network elements continues to increase, and the correlation between network elements grows exponentially. Traditional analysis methods are inefficient when processing large-scale data, and it is difficult to complete correlation analysis within a reasonable time.

[0007] Difficult to conduct in-depth mining: Most existing methods only analyze surface associations and are unable to mine deep implicit relationships between network elements. For example, some network elements may have non-explicit but critical associations, and traditional methods are difficult to identify these hidden patterns.

[0008] Association rule mining technology has been widely used in the field of data mining, especially in shopping basket analysis, market behavior prediction and other aspects. The basic idea of ​​association rule mining is to discover the potential association between data items by analyzing a large amount of data and present it in the form of rules. This technology has unique advantages in processing large-scale data and mining implicit patterns.

[0009] However, traditional association rule mining technology usually requires complete data and cannot handle the problem of missing data. Therefore, when applying association rule mining technology to network element association analysis in network management, it faces the challenges of the above-mentioned data incompleteness and other issues. Summary of the invention

[0010] In order to solve the above technical problems, the present invention proposes a network element association analysis method based on the AMIE+ algorithm, which can identify and mine the potential association relationships between network elements in the network by performing association analysis on them.

[0011] The present invention provides a network element association analysis method based on the AMIE+ algorithm, comprising:

[0012] Obtain network data;

[0013] Extracting entities and relationships from the network element data and constructing a network element knowledge graph;

[0014] Inputting the triplet data in the network element knowledge graph into a rule learning model to obtain an initial rule set, wherein the rule learning model is constructed based on the AMIE+ algorithm;

[0015] Optimizing the initial rule set to obtain an optimal rule set;

[0016] The optimal rule set is used to perform reasoning and obtain reasoning results.

[0017] Optionally, the network data includes: structured data, semi-structured data and unstructured data.

[0018] Optionally, before entity extraction of the network element data is performed, the method further includes: classifying and cleaning the network element data.

[0019] Optionally, extracting entities and relationships from the network data includes: extracting entities and relationships using a named entity recognition model, wherein the named entity recognition model includes: an R2RML mapping language layer, an LSTMCRF layer, and a PCNN segmented convolutional neural network layer;

[0020] The R2RML mapping language layer is used to map the structured data into RDF triples;

[0021] The LSTMCRF layer is used to extract entities from the unstructured data and semi-structured data;

[0022] The PCNN segmented convolutional neural network layer is used to extract the relationship between devices from the text.

[0023] Optionally, the input data of the rule learning model includes: an RDF data set, a header coverage threshold, a maximum rule length, and a minimum confidence threshold.

[0024] Optionally, inputting the triplet data in the network element knowledge graph into a rule learning model to obtain an initial rule set includes:

[0025] Generate several new rules based on the input data;

[0026] The new rules are tested to determine whether the confidence of the new rules meets a preset condition, and the initial rule set is obtained.

[0027] Optionally, optimizing the initial rule set to obtain the best rule set includes:

[0028] Weights are assigned to the initial rule set, and rule priorities are divided according to the weights to obtain the optimal rule set.

[0029] Optionally, reasoning the optimal rule set to obtain a reasoning result includes:

[0030] Inputting the optimal rule set into the PLS inference model to obtain a first inference result, wherein the first inference result is an optimal weighted rule set;

[0031] The first reasoning result is input into a multi-level reasoning model to obtain a reasoning result.

[0032] Optionally, inputting the first reasoning result into a multi-level reasoning model to obtain the reasoning result includes:

[0033] The optimal rule set is input into the multi-level PLS inference model. First, the input rules are divided into multiple sub-models. Each sub-model uses a rule subset corresponding to its relationship. The sub-models are run in sequence, and the inference results are gradually transferred until all sub-models complete the inference.

[0034] The reasoning results are integrated to obtain the reasoning results.

[0035] Compared with the prior art, the present invention has the following advantages and technical effects:

[0036] 1. Obtain network element data through various means (such as accessing existing network systems and using web crawlers), and classify and cleanse these data to ensure the integrity and accuracy of the data and remove redundant and invalid information. This provides a high-quality data foundation for subsequent knowledge extraction and rule learning, and improves the reliability of the model.

[0037] 2. Using advanced LSTM-CRF and PCNN technologies to extract network element entities, attributes and relationships in structured, semi-structured and unstructured data, it can accurately identify and extract complex relationships between network devices and automatically build a knowledge graph. By storing this knowledge in Jena, an RDF dataset for reasoning is effectively constructed, which improves the system's ability to process network element data.

[0038] 3. Using the AMIE+ algorithm for rule learning, combined with closed rule detection and pruning strategies, it can efficiently mine potential rules in RDF data sets, and the generated rules have high confidence and PCA confidence. By assigning weights to the rules through the rule optimization model, high-quality rules are further screened out, avoiding the influence of low-quality rules, thereby ensuring the accuracy and reliability of the reasoning results.

[0039] 4. The PSL reasoning model is introduced to enable reasoning based on optimized rules. Through MAP reasoning and lazy reasoning technology, the reasoning process can be carried out efficiently, reducing unnecessary calculations and saving memory. In addition, through MLE optimization weight learning, the accuracy of rule reasoning is improved, making the reasoning results more credible.

[0040] 5. The multi-level reasoning model effectively solves the problem of potential multiple relationship impacts between different rules by processing the reasoning results of the rules layer by layer. This method can reduce the uncertainty in the reasoning process, reasoning layer by layer, ensuring the consistency and accuracy of the reasoning results, while avoiding reasoning errors caused by rule conflicts.

[0041] 6. The multi-level reasoning model enables the system to process complex relational networks in RDF data, especially interactive reasoning of different relational rules, which significantly improves the flexibility and scalability of reasoning. By layered reasoning and layer-by-layer transmission of reasoning results, the model's ability to process complex relationships and associated information is effectively enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 It is a flow chart of a network element association analysis method based on the AMIE+ algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] The professional terms used in the embodiments are introduced as follows:

[0047] As an association rule mining technology for incomplete data, the AMIE+ algorithm (Association Rule Mining under Incomplete Evidence) can effectively cope with the challenge of missing data. The algorithm has demonstrated its unique advantages in the field of knowledge graph mining and can mine reliable association rules under incomplete data conditions. Specifically, the AMIE+ algorithm analyzes the evidence in the data and infers the potential correlation between the data, so that it can draw more accurate conclusions even if some data is missing.

[0048] Introducing the AMIE+ algorithm into network element correlation analysis can effectively solve the shortcomings of traditional correlation analysis methods, especially in the following aspects:

[0049] Dynamic mining of association rules: The AMIE+ algorithm does not rely on predefined rules, but automatically generates association rules by mining data. This enables it to dynamically adapt to changes in the network environment and update the association analysis results in real time.

[0050] Handling incomplete data: The core advantage of the AMIE+ algorithm is its ability to handle incomplete data, which is particularly critical in network operation and maintenance. Through this algorithm, even when network data is partially missing or incomplete, reliable association rules can still be mined to provide support for fault detection and performance optimization.

[0051] Efficient processing of large-scale data: The AMIE+ algorithm has high computational efficiency and can perform efficient association rule mining in large-scale data environments. This is an important feature for the management of modern large-scale networks.

[0052] The present invention proposes a network element association analysis method based on AMIE+ algorithm, such as Figure 1 As shown, the specific steps include:

[0053] Obtain network data;

[0054] Extract entities and relationships from network element data and build a network element knowledge graph;

[0055] Input the triplet data constituting the network element knowledge graph into the rule learning model to obtain an initial rule set, wherein the rule learning model is constructed based on the AMIE+ algorithm;

[0056] Optimize the initial rule set to obtain the best rule set;

[0057] Use the best rule set to perform reasoning and obtain reasoning results.

[0058] Specifically, the rule learning model is used to mine rules in RDF data, and its output is a series of rules including head coverage, confidence, and PCA confidence. The rule optimization model can assign weights to the rules and then screen high-quality rules. The role of the reasoning module is to apply PSL to the rules for reasoning. In order to distinguish the multiple effects of the reasoning results of the rules corresponding to one relationship as the condition of another relationship on the reasoning of other relationships, a multi-level method is introduced into the model to obtain a multi-level reasoning model for automatic rule learning.

[0059] Furthermore, network data includes: structured data, semi-structured data and unstructured data.

[0060] Specifically, it mainly includes structured data, semi-structured data and unstructured data. Structured data refers to data with a clear pattern and format. The attribute information of network elements, such as device ID, device type, device status, etc., is usually stored in the form of a table, with each network element corresponding to a row and columns including various attributes of the network element. Semi-structured data is between structured data and unstructured data. Although there is no strict table format, data can be organized by tags or labels to store XML or JSON format data of network element devices. The data may contain the attributes and relationships of network elements, but there is no fixed table structure. Unstructured data refers to data without a predefined data model and format, which cannot be directly stored in a traditional relational database. The log files of network element devices may contain a large number of free text records, such as device fault descriptions, alarm information, etc. The network element data is obtained in the following ways: Access the network element data stored in the existing network system (such as network device configuration, performance data, etc.). Use a web crawler to crawl data from a public network management platform or network device documents.

[0061] Furthermore, before entity extraction is performed on the network element data, the method further includes: classifying and cleaning the network element data.

[0062] Specifically, access existing network data: Investigate existing network management systems (such as network management system NMS and fault management system FMS) to understand the data source structure. Access structured and semi-structured network data in these systems to collect information such as network equipment, topology, configuration, and performance.

[0063] Use a web crawler to obtain network element data: Get the URL of the target network element data (such as the document page of the public API or device management platform). Submit an HTTP request to the corresponding URL. Parse the HTTP response and extract information such as parameter configuration and performance description from the device documentation. Store the parsing results, and classify and clean the data to remove redundant and invalid information.

[0064] Further, extracting entities and relations from network data includes: extracting entities and relations using a named entity recognition model, wherein the named entity recognition model includes: an R2RML mapping language layer, an LSTMCRF layer, and a PCNN segmented convolutional neural network layer;

[0065] R2RML mapping language layer, used to map structured data into RDF triples;

[0066] LSTMCRF layer, used to extract entities from unstructured and semi-structured data;

[0067] The PCNN segmented convolutional neural network layer is used to extract the relationship between devices from the text. The extracted entity and relationship data are finally stored in Jena to build the network element knowledge graph.

[0068] Specifically, knowledge is extracted and stored for structured, semi-structured and unstructured network data.

[0069] Knowledge is extracted from structured data (such as device configuration tables and performance monitoring data in the database), and the network data is mapped from the relational database to the RDF data set using the R2RML mapping language.

[0070] Knowledge is extracted from semi-structured and unstructured data (such as equipment logs, manuals, configuration files, etc.). The LSTMCRF named entity recognition model is used to extract network element entities (such as equipment names, interfaces, protocols, etc.). The PCNN segmented convolutional neural network is used to extract network element relationships (such as communication protocols between devices, link connections, etc.).

[0071] When using the LSTM-CRF named entity recognition model for entity extraction, the model construction process is as follows: establish an Embedding layer to map words into word vectors; establish an n-layer bidirectional LSTM layer to learn the before and after semantic information of words; establish a CRF layer to perform sequence annotation on each word;

[0072] The PCNN segmented convolutional neural network is used for relationship extraction. The model construction process is as follows: establish an Embedding word vector extraction layer to map words and word position information into word vectors; establish a convolution layer to perform segmented convolution operations on text; establish a max pooling layer; establish an output layer, and the activation function is the softmax function;

[0073] Finally, the extracted network element entities, attributes, and relationship data are stored in Jena (Apache's RDF triple database) to build a preliminary knowledge graph.

[0074] Furthermore, the input data of the rule learning model includes: RDF dataset, header coverage threshold, maximum rule length and minimum confidence threshold.

[0075] Furthermore, the triple data in the network element knowledge graph is input into the rule learning model to obtain the initial rule set including:

[0076] Generate several new rules based on the input data;

[0077] Test the new rules to determine whether the confidence of the new rules meets the preset conditions and obtain the initial rule set.

[0078] Specifically, the steps of the rule learning model are as follows:

[0079] 1. Data input: The input of the AMIE+ algorithm includes the RDF dataset, the header coverage threshold (minHC), the maximum rule length (maxLen), and the minimum confidence threshold (minConf).

[0080] 2. Rule generation:

[0081] AMIE+ first creates an empty rule queue with an empty rule header (no rule body).

[0082] Each iteration generates new rules from the data, checks whether they are closed-form rules, and calculates their PCA confidence scores.

[0083] 3. Rule detection:

[0084] If the PCA confidence is lower than (minConf), the rule will be discarded; otherwise, the rule is added to the queue.

[0085] The confidence of the generated sub-rule must be higher than the confidence of its parent rule. If this condition is not met, the sub-rule will not be output.

[0086] 4. Rule pruning and optimization:

[0087] AMIE+ provides pruning strategies and approximation strategies in the rule mining process to improve algorithm efficiency and reduce unnecessary rule generation.

[0088] 5. Output: Finally, AMIE+ outputs a set of rules including head coverage, confidence, and PCA confidence.

[0089] Furthermore, the initial rule set is optimized to obtain the best rule set including:

[0090] Assign weights to the initial rule set, prioritize the rules according to the weights, and obtain the best rule set.

[0091] Specifically, the steps of the rule optimization model are as follows:

[0092] The quality of rules generated by the AMIE+ algorithm may vary, and simply relying on thresholds may cause some valuable rules to be ignored. To this end, a rule optimization model is introduced to screen high-quality rules through weight optimization.

[0093] 1. Weight distribution:

[0094] Use the Maximum Likelihood Estimation (MLE) method to assign weights to each rule. The weight of each rule is automatically adjusted based on the performance of the rule in the training dataset.

[0095] 2.Threshold division:

[0096] The rules under the same relationship are divided into multiple levels according to the weights. They are divided into 5 threshold intervals. If the number of rules is less than 5, the performance of each rule is tested one by one.

[0097] 3. Screening:

[0098] Based on the weight test, the rules with the best performance are finally selected. These optimized rules are used in the subsequent reasoning stage.

[0099] Furthermore, the optimal rule set is inferred, and the inference results obtained include:

[0100] The best rule set is input into the multi-level PLS inference model. First, the input rules are divided into multiple sub-models. Each sub-model uses a subset of rules corresponding to its relationship. The sub-models are run in sequence, and the inference results are gradually passed until all sub-models complete the inference.

[0101] Integrate the inference results to obtain the inference results.

[0102] Specifically, the PSL (probabilistic soft logic) reasoning module uses the filtered rules for reasoning and infers the rules through the probability model. The specific steps are as follows:

[0103] 1. Reasoning input:

[0104] The PSL reasoning model takes the optimized rules, query predicates, evidence predicates, and known evidence atoms as input.

[0105] 2. Reasoning process:

[0106] PSL reasoning uses Maximum A Posteriori (MAP) to calculate the inferred probability of the query predicate based on known evidence.

[0107] PSL's lazy reasoning technology allows reasoning only for atoms with non-zero values, reducing memory usage and improving reasoning efficiency.

[0108] 3. Reasoning Optimization:

[0109] The PSL optimization method is based on a basic assumption: all atoms are assumed to be zero unless there is evidence that they are non-zero. This method reduces the computational burden of irrelevant data and significantly improves the efficiency of reasoning.

[0110] 4. Weight learning:

[0111] PSL also calculates the optimal weight of each rule through MLE, making the reasoning model more accurately adapt to the input evidence data.

[0112] In order to better handle the complex multiple relationships in RDF data, a multi-level reasoning model is introduced into the reasoning process to process the reasoning results of the rules layer by layer and reduce the uncertainty of the reasoning results. The steps are as follows:

[0113] 1. Hierarchical Model:

[0114] The input rule set is divided into multiple sub-models according to different target relations, and each sub-model focuses on the reasoning of a specific type of relation.

[0115] 2. Layer-by-layer reasoning:

[0116] In the first step, the sub-model (R1) is run and the preliminary inference results are obtained.

[0117] The result of (R1) is passed as a known condition to the sub-model (R2), and then (R2) continues reasoning based on the result of (R1).

[0118] This process is repeated, passing the inference results of each sub-model to the next sub-model until all sub-models complete inference.

[0119] 3. Results integration:

[0120] Finally, the reasoning results of all sub-models will be integrated to obtain the reasoning result of the entire RDF dataset.

[0121] The present invention proposes a network element association analysis method based on the AMIE+ algorithm. The purpose is to:

[0122] Dynamically adapt to changes in the network environment: One of the purposes of the present invention is to introduce the AMIE+ algorithm and utilize its automated association rule mining capabilities to dynamically generate association rules between network elements, thereby getting rid of the traditional method's reliance on static rules. This can better adapt to changes in the network environment and ensure the accuracy and timeliness of association analysis.

[0123] Processing incomplete data and improving the robustness of analysis: The present invention aims to solve the problem of analysis bias caused by incomplete data. Through the application of the AMIE+ algorithm, the present invention can still effectively perform network element association analysis in the case of missing data, thereby improving the robustness of data processing and the reliability of analysis results.

[0124] Improve analysis efficiency and support large-scale network environments: In order to cope with the problem of high computational complexity in large-scale network environments, this paper uses the efficient computing power of the AMIE+ algorithm to achieve rapid correlation analysis in a big data environment. This method can process large-scale network data in a reasonable time and meet the needs of real-time monitoring and rapid response.

[0125] Deeply explore the potential correlations between network elements: Another important purpose of the present invention is to discover the hidden potential correlations between network elements through the deep mining capability of the AMIE+ algorithm. Compared with traditional shallow analysis, the present invention can identify more deep and critical correlation patterns, which is of great significance for network optimization and fault prediction.

[0126] In summary, the comprehensive goal of the present invention is to provide a network element association analysis method based on the AMIE+ algorithm, aiming to overcome the deficiencies of the prior art, especially the defects in dynamics, data integrity, analysis efficiency and deep mining. Through the method of the present invention, it is possible to better support the management and operation of modern networks and improve the overall performance and stability of the network.

[0127] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A network element association analysis method based on AMIE+ algorithm, characterized in that: include: Obtain network data; Extracting entities and relationships from the network element data and constructing a network element knowledge graph; Inputting the triplet data in the network element knowledge graph into a rule learning model to obtain an initial rule set, wherein the rule learning model is constructed based on the AMIE+ algorithm; Optimizing the initial rule set to obtain an optimal rule set; The optimal rule set is inferred to obtain an inference result.

2. The network element association analysis method based on the AMIE+ algorithm according to claim 1, characterized in that: The network data includes: structured data, semi-structured data and unstructured data.

3. The network element association analysis method based on the AMIE+ algorithm according to claim 1, characterized in that: Before extracting entities and relationships from the network element data, the method further includes: classifying and cleaning the network element data.

4. The network element association analysis method based on the AMIE+ algorithm according to claim 2, characterized in that: Extracting entities and relationships from the network data includes: extracting entities and relationships using a named entity recognition model, wherein the named entity recognition model includes: an R2RML mapping language layer, an LSTMCRF layer, and a PCNN segmented convolutional neural network layer; The R2RML mapping language layer is used to map the structured data into RDF triples; The LSTMCRF layer is used to extract entities from the unstructured data and semi-structured data; The PCNN segmented convolutional neural network layer is used to extract the relationship between devices from the text.

5. The network element association analysis method based on the AMIE+ algorithm according to claim 1, characterized in that: The input data of the rule learning model includes: RDF data set, header coverage threshold, maximum rule length and minimum confidence threshold.

6. The network element association analysis method based on the AMIE+ algorithm according to claim 5, characterized in that: Inputting the triplet data in the network element knowledge graph into the rule learning model to obtain the initial rule set includes: Generate several new rules based on the input data; The new rules are tested to determine whether the confidence of the new rules meets a preset condition, and the initial rule set is obtained.

7. The network element association analysis method based on the AMIE+ algorithm according to claim 1, characterized in that: Optimizing the initial rule set to obtain the best rule set includes: Weights are assigned to the initial rule set, and rule priorities are divided according to the weights to obtain the optimal rule set.

8. The network element association analysis method based on the AMIE+ algorithm according to claim 1, characterized in that: Reasoning the optimal rule set and obtaining the reasoning result includes: Input the optimal rule set into the multi-level PLS inference model, divide the input optimal rule set into multiple sub-models, each sub-model uses a rule subset corresponding to its relationship, runs the sub-models in sequence, and gradually transfers the inference results until all sub-models complete the inference; The reasoning results are integrated to obtain the reasoning results.

Images