A communication network fault locating system and method based on big data analysis

Abstract

The application belongs to the technical field of communication network, and particularly relates to a communication network fault positioning system and method based on big data analysis, which comprises an offline training module and an online training module, wherein the offline training module and the online training module each comprise a data acquisition unit, a data cleaning unit, a modeling analysis unit, an optimization iteration unit and a model evaluation unit; the offline training module and the online training module are connected to a network management server and run relying on a network management system; each unit of each module is composed of atomized capabilities, realizing front-end and back-end separation, supporting high concurrency and atomized capability assembly; the alarm distinguishing and chaining mechanism of the fault root network element proposed in the scheme can realize the disclosure of part of alarms to after-sales service parties, realize intelligent alarm correlation compression and rapid fault positioning under a cloud network, and effectively solve the problems of insufficient manual operation and maintenance experience, difficult fault positioning, low processing efficiency and difficult customized service under a new cloud network.

Description

Technical Field

[0001] This invention relates to the field of communication network technology, specifically to a communication network fault location system and method based on big data analysis. Background Technology

[0002] The continuous introduction of new technologies and services in communication networks has made network structures and business processes more complex. The cloud core of the network is transforming towards IT-based, cloud-based, regional, and CU separation; the bearer network is transforming towards edge L3 and fixed-mobile convergence; the wireless network is transforming towards multi-standard, 4 / 5G convergence, and VoNR; 5GToB is developing rapidly; and CT is converging with IT applications. The difficulty of alarm correlation and fault location between different levels and different manufacturers will be greater, and improving operation and maintenance efficiency will face huge challenges.

[0003] Traditional communication networks primarily rely on monitoring based on maintenance experience and fixed physical network devices. Personnel use their experience to analyze alarm correlation rules, determine the problem's location based on network topology, and then notify other maintenance personnel for resolution. However, with the rapid development of cloud-based network architectures and the shift from CT to IT, the accumulation of maintenance experience by personnel lags far behind the pace of network development. Furthermore, the characteristics of cloud-based networks, such as general-purpose hardware servers, automatic virtual machine migration, and automatic scaling of virtual network elements, result in real-time network topology updates, fundamentally different from the fixed physical network devices and fixed topology of traditional network architectures. Relying on manual monitoring and management of network alarms in cloud-based networks is no longer practical. Therefore, the ability to quickly and accurately monitor the root network element causing a large number of alarms, and only dispatch work orders to it, thereby efficiently resolving problems and reducing work order volume, while accurately notifying the relevant customers of fault alarms and providing customized services, is crucial for rapid fault handling and ensuring customer satisfaction in the industry.

[0004] Typically, when a network device in a cloud-based network fails, a large number of alarm messages will be generated in a short period of time, affecting devices at the same horizontal level and across vertical levels. These alarms are mainly categorized as primary and derivative alarms, alarms specific to the network and alarms from other disciplines, device alarms, performance alarms, and dial-up alarms. While this provides maintenance personnel with more diverse ways to discover problems and richer analytical and location information, it also increases the difficulty of alarm handling. Monitoring and maintenance personnel find it difficult to quickly and accurately distinguish the root network element and the root cause of the problem from the reported alarms. For example, in the Central China region, a certain province's network cloud receives tens of thousands of alarms daily, but only about 10 faults actually require manual handling. Individual faults and operations on interconnected devices can generate a large number of related alarms. Maintenance personnel mainly spend their energy analyzing and processing each alarm or work order, making it difficult to quickly locate the root cause, consuming a lot of manpower and resources, and potentially leading to the omission of important alarms.

[0005] Furthermore, after providing 5G To B and cloud-based network services to external clients, operators also need to provide customized monitoring and maintenance services for each industry customer. Alarms generated by alarm network elements can generally be disclosed and traced within the enterprise to which the network element belongs. However, for after-sales service providers of cloud-based network elements, some alarms may not be suitable for disclosure because they involve the confidentiality of the enterprise to which the network element belongs or are not related to the primary issue. Therefore, how to disclose some alarms to after-sales service providers so that they can trace the cause of the fault while ensuring that confidential alarms are not leaked is a problem that needs to be solved after locating the root network element of the fault. Summary of the Invention

[0006] The purpose of this invention is to provide a communication network fault location system and method based on big data analysis to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a communication network fault location system based on big data analysis, comprising an offline training module and an online training module, wherein both the offline training module and the online training module include a data acquisition unit, a data cleaning unit, a modeling and analysis unit, an optimization and iteration unit, and a model evaluation unit;

[0008] Both the offline and online training modules are connected to the network management server and run on the network management system. Each unit of each module is composed of atomic capabilities, realizing front-end and back-end separation, supporting high concurrency, and atomic capability assembly. Various functional services are encapsulated in atomic capabilities to achieve rapid function orchestration and construction.

[0009] The offline training module trains from multiple dimensions to obtain a primary and secondary association rule table, including a primary rule table obtained by combining manual rule application. The primary rule table adopts a combination of expert experience evaluation and alarm clearing time evaluation, and then feeds it back to the primary and secondary association relationship to generate an optimization algorithm for optimization.

[0010] The offline training module's data acquisition unit mainly collects historical alarm data, network topology data, and resource data from the network management system, as well as manually generated rule tables; and outputs the data to the data cleaning unit according to the interface specifications; the online training module's data acquisition unit is mainly responsible for collecting the current alarm, topology data, and resource data from the network management system in real time;

[0011] The data cleaning unit is responsible for cleaning alarm, topology, and resource data, removing alarm, topology, and resource data with missing or incorrect fields; filtering out flashing alarms and short-term interruption alarms; extracting various key data fields required for fault location; and outputting the data to the modeling and analysis unit according to the interface specifications.

[0012] The modeling and analysis unit of the offline training module is responsible for processing alarms and, based on the resource pool information in the cloud network alarm information, prioritizing the spatial domain division of alarm data; the optimization and iteration unit updates the frequency of primary and secondary alarms and the frequency of binary terms in the association rule base synchronously based on the daily data operation results, performs probabilistic statistical analysis on the updated association rules, and adds new rules or reduces old rules through confidence and lift to maintain the real-time performance and rationality of the model;

[0013] In the model evaluation unit of the offline training module, the model evaluation unit verifies the rules based on topology data and supports experts in adjusting the rules in the association rule base based on their experience. In the model evaluation unit of the online training module, the system automatically judges whether the association rules are accurate based on the content of the operation and maintenance personnel's reply in the work order, and marks and reminds those rules that do not hit the root cause.

[0014] Preferably, the optimization algorithm uses the DBSCAN clustering algorithm to cluster batch alarms and obtain the cluster IDs of the occurrence clusters; it uses the Eclat association mining algorithm to realize the primary and secondary association analysis of alarms, using the combination of "network element name + alarm name" to quickly find alarm pairs with frequent binary terms; it performs rule verification on alarms based on alarm clearing time and removes alarms that are not within the centralized recovery time window; it generalizes the mined primary and secondary association rules based on network element type and stores the generalized rules in the database.

[0015] Preferably, in the modeling and analysis unit of the online training module, the alarm clustering and grouping of the real-time data cleaning results in the time domain and space is performed. Then, the association relationship of the real-time alarm data in each group is searched according to the rule relationship table in the association rule base. If there are corresponding secondary alarms and primary alarms in the same group, the primary alarm is located in real time, and the secondary alarms are merged and compressed. For root cause alarms, orders are dispatched, and the alarms corresponding to the fault root network element are uploaded to the chain.

[0016] Preferably, the current alarms are collected in real time within the data cleaning unit, and key fields required for fault location are extracted, including alarm occurrence time, alarm network element name, alarm network element ID, alarm network element type, alarm title, network management alarm ID, resource pool, and specialty. Alarms with missing key fields are removed, frequent alarms are merged, intermittent alarms and worthless alarms are filtered, and normalization preprocessing is performed to clean and transform all data.

[0017] Preferably, alarm data is divided into time domains based on the DBSCAN density clustering algorithm. According to operational experience, alarms caused by the same network element failure have a certain correlation in terms of time and space. Therefore, based on the resource pool information in the cloud network alarm information, the alarm data is first divided into spatial domains, and alarms belonging to the same resource pool are grouped together. Then, the DBSCAN clustering algorithm is used to automatically divide the time domain of batch alarms in combination with the occurrence time of the alarm data, and alarms that may be related are divided into one time window, avoiding root cause alarms and derived alarms not being in the same time window due to manual division, and finally obtaining the grouping of batch alarm data.

[0018] Preferably, fault location is achieved based on the Eclat association mining algorithm. In the root cause location scenario, the Eclat itemset is a combination of "network element name + alarm name", which is used for subsequent mining of frequent itemsets and association rules. Alarm pairs without association are filtered out by support, and the primary and secondary association rules of alarms and the lift degree are used to determine whether the associated alarms belong to co-alarms.

[0019] Preferably, the generalization of primary and secondary association rules is achieved based on probability statistics. Since the rules generated by the Eclat algorithm are too detailed and specific, in most cases, alarm data cannot be completely mapped one-to-one with alarms of the same network management alarm ID and the same network element ID. It is determined whether the primary and secondary alarms occur on the same network element and labeled with 0 and 1. For alarms with the same primary and secondary alarm ID, primary and secondary network element ID and whether they are on the same network element, logical merging is performed.

[0020] Compared with the prior art, the beneficial effects of the present invention are:

[0021] The proposed alarm differentiation and on-chain mechanism for fault root network elements can disclose a portion of alarms to after-sales service providers, enabling them to trace the cause of the fault, while ensuring that confidential alarms are not leaked. This provides customers with customized solutions for each customer, improves the efficiency of front-line network operation and maintenance through network intelligence, and achieves intelligent alarm correlation compression and rapid fault location in cloud networks based on density clustering spatiotemporal window division and alarm correlation big data analysis algorithms, and an intelligent fault location platform. It effectively solves the problems of insufficient manual operation and maintenance experience, difficulty in fault location, low processing efficiency, and difficulty in customized services in new cloud networks.

[0022] Based on a big data analysis algorithm for intelligent fault location, the platform automatically correlates all alarm information in cloud-based networks, effectively improving the compression rate and accuracy of the algorithm. Furthermore, the intelligent fault location platform, based on an open architecture and atomic capabilities, exhibits good adaptability and scalability, enabling the expansion and development of analytical applications and improving the efficiency of distributed platforms. Especially in environments where human experience is limited and fault location is challenging in cloud-based network architectures such as 5G and NFV, it can quickly and effectively generate alarm correlation rules across various cloud-based networks, locate root fault elements in real time, shorten fault handling time, and improve network quality and operational efficiency. Simultaneously, it can provide personalized and customized services to ensure a positive customer experience. Attached Figure Description

[0023] Figure 1 This is a system logic block diagram of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will now be described in conjunction with the accompanying drawings.

[0025] The embodiments described herein are clearly and completely presented. Obviously, the described embodiments are only a portion of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0026] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. Example

[0027] Please refer to Figure 1. This invention provides a technical solution for a communication network fault location system based on big data analysis, which includes an offline training module and an online training module.

[0028] Both the offline training module and the online training module include a data acquisition unit, a data cleaning unit, a modeling and analysis unit, an optimization and iteration unit, and a model evaluation unit;

[0029] The data acquisition unit in the offline training module collects historical data from the network management system, including alarm data, network topology data, resource data, and manually generated rule tables, and outputs the data to the data cleaning unit according to the interface specifications. The data acquisition unit in the online training module is responsible for collecting the current alarm, topology, and resource data of the network management system in real time.

[0030] The data cleaning unit cleans the collected data, removing alarm topologies and resource data with missing or incorrect fields; it filters out flashing alarms and short-term interruption alarms, extracts various key data fields required for fault location, and outputs the data to the modeling and analysis unit according to the interface specifications.

[0031] In addition to data cleaning, the online training module searches for and replaces missing topology or resource data from the offline training module with data from the most recent period.

[0032] The modeling and analysis unit of the offline training module is responsible for processing alarms. Based on the resource pool information in the cloud network alarm information, it prioritizes spatial domain division of alarm data. Then, based on the alarm occurrence time, it uses the DBSCAN clustering algorithm to cluster batch alarms and obtain the occurrence cluster ID. Based on the Eclat association mining algorithm, it performs alarm primary and secondary association analysis, using the combination of "network element name + alarm name" to quickly find alarm pairs with frequent binary terms. Based on the alarm clearance time, it performs rule verification on alarms and removes alarms that are not within the centralized recovery time window. Based on the network element type, it generalizes the mined primary and secondary association rules and stores the generalized rules in the database.

[0033] In the modeling and analysis unit of the online training module, the alarm clustering and grouping of the real-time data after cleaning is performed in the time domain and space. Then, based on the rule relationship table in the association rule base, the association relationship of the real-time alarm data in each group is searched. If there are corresponding secondary alarms and primary alarms in the same group, the primary alarm is located in real time, and secondary alarms are merged and compressed. For root cause alarms, orders are dispatched, and the alarms corresponding to the fault root network element are uploaded to the chain.

[0034] The optimization iteration unit updates the frequency of primary and secondary alarms and the frequency of binary terms in the association rule base synchronously based on the daily data operation results. It performs probabilistic statistical analysis on the updated association rules and adds new rules or reduces old rules based on confidence and lift, so as to maintain the real-time performance and rationality of the model.

[0035] The model evaluation unit verifies the rules based on topology data and supports experts in adjusting the rules in the association rule base based on their experience. The model evaluation unit in the online training module automatically judges whether the association rules are accurate based on the content of the operation and maintenance personnel's reply in the work order, and marks and reminds the rules that do not hit the root cause.

[0036] The data cleaning unit collects current alarms in real time, extracts key fields needed for fault location, such as alarm occurrence time, alarm network element name, alarm network element ID, alarm network element type, alarm title, network management alarm ID, resource pool, and specialty. It then removes alarms with missing key fields, merges frequent alarms, filters intermittent and worthless alarms, and performs preprocessing such as normalization. All data is cleaned and transformed. An example of alarm data is shown in the following figure:

[0037] Alarm data sample

[0038]

[0039] The alarm data is divided into time domains based on the DBSCAN density clustering algorithm. According to operational experience, alarms caused by the same network element failure have a certain correlation in terms of time and space. Therefore, based on the resource pool information in the cloud network alarm information, the alarm data is first divided into spatial domains. Alarms belonging to the same resource pool are grouped together. Then, the DBSCAN clustering algorithm is used to automatically divide the time domain of batch alarms in combination with the occurrence time of the alarm data. Alarms that may be related are grouped into one time window, avoiding root cause alarms and derived alarms not being in the same time window due to manual division. Finally, the grouping of batch alarm data is obtained.

[0040] Fault localization is achieved using the Eclat association mining algorithm. In the root cause localization scenario, the Eclat itemset is a combination of "network element name + alarm name", which is used for subsequent mining of frequent itemsets and association rules. Compared with traditional Apriori and FP-Growth algorithms, it can quickly find alarm pairs with frequent binary terms, filter out alarm pairs without correlation by support, and determine the primary and secondary association rules of alarms and whether the lift excludes related alarms from being co-alarm alarms based on confidence or conditional probability relationships. The following is a sample data of association rules:

[0041] Alarm primary and secondary association rules

[0042]

[0043] Then, rule verification is carried out. For the mined alarm groups, the accuracy of the rules is verified according to whether the alarm clearing times are concentrated within a certain period; because according to maintenance experience, for the alarms generated by the same fault, their clearing times are also concentrated within a certain time range; according to the time difference between the earliest alarm clearing time and the latest alarm clearing time in the group, denoted as T; for the alarm clearing time corresponding to the median (such as for 15 alarms, it is the clearing time A of the 8th alarm), calculate the time points of A - T / 2 minutes and A + T / 2 minutes and define them as the reasonable time interval; judge whether all alarm clearing times fall within the interval of

A - T / 2 minutes - A + T / 2 minutes

Y / N

[0044] Based on probability statistics, the generalization of the primary-secondary association rules is realized. Since the rules generated by the Eclat algorithm are too fine and specific, in most cases, alarm data cannot be completely and一一corresponded to the alarms with the same network management alarm ID and the same network element ID, and cannot be widely applied to the massive real-time alarm data. Therefore, this process generalizes the primary-secondary association rules; first, judge whether the primary and secondary alarms occur on the same network element, and label them with 0 and 1; perform logical merging processing on the alarms with the same primary-secondary alarm ID, primary-secondary network element ID, and whether they are on the same network element field; the specific merging logic is as follows:

[0045] Suppose there are two rules as follows. Since the alarm title, network management alarm ID, network element name, and network element ID一一correspond, for simplicity, only the network element ID and network management alarm ID are taken as examples, and the letters in the table are the corresponding numerical examples:

[0046] Sample data before the generalization of association rules

[0047]

[0048] For the primary-secondary alarm ID, primary-secondary network element ID, and whether they are on the same network element field, the rules are merged. The generalized rules are shown in the following figure:

[0049] Sample data after the generalization of association rules

[0050]

[0051] Taking rule ① in the generalized rules as an example, the three fields of the primary-secondary network element type, primary-secondary alarm ID, and whether they are on the same network element in the original rules are the same in value, so they are merged and aggregated;

[0052] In the generalization rules, although the primary and secondary network element types and primary and secondary alarm IDs of the original rules 3 and 4 are the same as those of rules 1 and 2, the values ​​of the "whether they are the same network element" field are different. Therefore, only rules 3 and 4 are merged and aggregated.

[0053] Rules ③ and ④ in the generalization rules are also merged based on the above logic, but since there are no rule items that can be merged, the frequency remains unchanged;

[0054] Finally, this set of generalized rules can, to some extent, weaken the restrictions imposed by network element IDs on rule application and expand the application scope of association rules;

[0055] Then, real-time location and dispatch are performed. Based on the real-time collection of current alarms, the key fields required for fault location are extracted. Similarly, according to the temporal and spatial characteristics of alarm data, alarms are clustered and grouped in the temporal and spatial domains. Then, according to the primary and secondary rule relationship table mined out, the real-time alarm data in each group is searched for correlation. If there are corresponding secondary alarms and primary alarms in the same group, the secondary alarms are deleted. Finally, the root cause alarm of the fault is located in real time and alarm compression is performed.

[0056] When dispatching work orders, only root cause alarms are dispatched according to the system's preset dispatch rules, and other secondary alarm information is attached to the work order; if the root cause alarm has already been dispatched, the secondary alarm information is added to the root cause alarm's work order again, and the added alarm is notified to the front-line operation and maintenance personnel via SMS to avoid omissions.

[0057] After locating the faulty root network element, the corresponding alarm for the faulty root network element is uploaded to the blockchain. The specific steps are as follows:

[0058] Step 1: Obtain the relevant information of the alarm corresponding to the faulty root network element. For example, obtain one or more of the following information from Table 1 "Alarm Data Sample": alarm occurrence time, resource pool, specialty, network element name, alarm title, device type, manufacturer alarm ID, and alarm text.

[0059] Step 2: Based on the pre-set "Sensitive Alarm Information List," determine whether the relevant information of the alarm corresponding to the faulty root network element matches at least one item in the sensitive alarm information list. The "Sensitive Alarm Information List" is pre-built based on information related to classified alarms. For example, if a network element named X triggers an alarm at a certain time T and within a certain resource pool W, which is a classified alarm, then the "Sensitive Alarm Information List" will contain the information "X+T+W." Therefore, it can be determined whether X, T, and W exist in the relevant information of the alarm corresponding to the faulty root network element. If they exist, it's a match; otherwise, it's a miss.

[0060] Step 3: Classify alarms that match at least one item in the sensitive alarm information list as "sensitive alarms" and classify alarms that do not match any item in the sensitive alarm information list as "non-sensitive alarms".

[0061] Step 4: Sensitive alarms are uploaded to the private blockchain of the enterprise to which the faulty root network element belongs: Non-sensitive alarms are uploaded to the private blockchain of the enterprise to which the faulty root network element belongs (this private blockchain is the exclusive blockchain of the enterprise and the after-sales service provider of the faulty root network element has no access rights), and to the public blockchain (this public blockchain is a blockchain that can be accessed by both the enterprise and the after-sales service provider of the faulty root network element).

[0062] Based on the alarms reported by the faulty root network element, the "belonging enterprise" field of the network element is searched in the resource data to determine the enterprise to which the faulty root network element belongs. Then, based on the pre-stored access addresses of the enterprise and its private chain, the connection to the private chain is realized. Based on the alarms reported by the faulty root network element, the "corresponding after-sales service provider" field of the network element is searched in the resource data to determine the after-sales service provider of the faulty root network element. Then, based on the pre-stored access addresses of the after-sales service provider and the public chain that the after-sales service provider has access to, the connection to the public chain is realized.

[0063] When optimizing the iterative unit, the five steps from 3.3.1 to 3.3.5 are repeated daily. The alarms generated by the network each day are cleaned, divided into time domains, and the relationships are mined, verified, and generalized. The resulting associations are then incorporated into the alarm primary and secondary relationship table mined by the system. The frequency of primary and secondary alarms and the frequency of binary terms are updated synchronously. The updated relationship table is subjected to probabilistic statistical analysis. New rules are added or old rules are reduced by confidence and lift, so as to maintain the real-time performance and rationality of the model.

[0064] It can learn new alarms incrementally every day (small cycle) and update it iteratively, while retraining the rules as a whole every six months (large cycle) to keep the expert experience rule base effective and applicable.

[0065] The model evaluation unit verifies the rules based on topology information and work order information. At the same time, it matches and compares the actual manual rules in the existing network with the rules generated by this solution. It supports manual merging, optimization, and addition of rules in the associated rule base to output more accurate rules.

[0066] Adding topology data as an auxiliary means of determining rule validity can filter out accompanying alarms; based on the content of the operation and maintenance personnel's reply in the work order dispatched by the root cause alarm, the system can automatically determine whether the association rule is accurate and record the rule that does not hit the root cause; the system can automatically send SMS or email reminders to the operation and maintenance personnel to pay attention when the fault hit rate is calculated daily and falls below the manually set threshold.

[0067] Then, the DBSCAN density clustering algorithm was implemented to cluster batch alarms based on the alarm occurrence time dimension, grouping potentially related alarms into the same time slice for subsequent correlation mining. The neighborhood distance and the number of neighborhood samples were tuned using the K-distance parameter tuning method and feedback on clustering results / human experience. Alarm data is closely inseparable in the time domain. If a fixed time window is used for division, it is difficult to accurately grasp the window interval of related alarms. However, the density clustering algorithm can flexibly adjust the alarm time window. The alarm windows of different batches are not the same, which can make alarms in the same batch as close to the same time window as possible.

[0068] By optimizing the Eclat algorithm, rule mining time can be shortened. Taking the professional alarm operation results of a certain month's operator network cloud as an example, the platform using the Eclat algorithm effectively improved the rule mining efficiency, while not requiring GPU processing and consuming minimal CPU, memory, and storage resources. An example of association algorithm mining is shown in the figure below:

[0069] Association algorithm mining efficiency

[0070]

[0071] Taking the network cloud operation results of a certain month as an example, the fault location algorithm based on clustering and alarm association rule mining effectively improved the original alarm dispatch ratio. The introduction of rule generalization processing can compress multiple alarms into a single rule, dispatching orders only to root cause alarms, thus enhancing the applicability and scalability of the rules, effectively improving the fault hit rate of work orders, and reducing the workload of frontline maintenance personnel. The comparison of network indicators before and after deployment is shown in the following figure:

[0072] Comparison of network cloud professional indicators before and after deployment

[0073]

[0074] By using big data correlation mining and analysis, we can reduce the reliance on resource information and the intervention of human experience, output more accurate fault location conclusions, and improve the interpretability and accuracy of rules.

[0075] It can disclose a portion of alarms to after-sales service providers, enabling them to trace the cause of the fault, while ensuring that confidential alarms are not leaked, providing customers with a customized "one-case-one-service" solution.

[0076] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A communication network fault location system based on big data analysis, comprising an offline training module and an online training module, characterized in that, Both the offline training module and the online training module include a data acquisition unit, a data cleaning unit, a modeling and analysis unit, an optimization and iteration unit, and a model evaluation unit; Both the offline and online training modules are connected to the network management server and run on the network management system. Each unit of each module is composed of atomic capabilities, realizing front-end and back-end separation, supporting high concurrency, and atomic capability assembly. Various functional services are encapsulated in atomic capabilities to achieve rapid function orchestration and construction. The offline training module trains from multiple dimensions to obtain a primary and secondary association rule table, including a primary rule table obtained by combining manual rule application. The primary rule table adopts a combination of expert experience evaluation and alarm clearing time evaluation, and then feeds it back to the primary and secondary association relationship to generate an optimization algorithm for optimization. The offline training module's data acquisition unit mainly collects historical alarm data, network topology data, and resource data from the network management system, as well as manually generated rule tables; and outputs the data to the data cleaning unit according to the interface specifications; the online training module's data acquisition unit is mainly responsible for collecting the current alarm, topology data, and resource data from the network management system in real time; The data cleaning unit is responsible for cleaning alarm, topology, and resource data, removing alarm, topology, and resource data with missing or incorrect fields; filtering out flashing alarms and short-term interruption alarms; extracting various key data fields required for fault location; and outputting the data to the modeling and analysis unit according to the interface specifications. The modeling and analysis unit of the offline training module is responsible for dividing the alarm data into spatial domains based on the resource pool information in the cloud network alarm information after processing. The optimization iteration unit updates the frequency of primary and secondary alarms and the frequency of binary terms in the association rule base synchronously based on the daily data operation results. It performs probabilistic statistical analysis on the updated association rules and adds new rules or reduces old rules based on confidence and lift, so as to maintain the real-time performance and rationality of the model. In the model evaluation unit of the offline training module, the model evaluation unit supports experts in adjusting the rules in the association rule base based on their experience, according to the topological data verification rules. In the model evaluation unit of the online training module, the system automatically determines whether the association rules are accurate based on the responses from maintenance personnel in the work order, and marks and reminds users of rules that do not hit the root cause. The optimization algorithm employs the DBSCAN clustering algorithm to cluster batch alarms and obtain the cluster IDs of the occurrence clusters; it uses the Eclat association mining algorithm to perform alarm primary and secondary association analysis, employing the combination of "network element name + alarm name" to quickly identify alarm pairs with frequent binary terms; it performs rule verification on alarms based on alarm clearance time, removing alarms that are not within the centralized recovery time window; and it generalizes the mined primary and secondary association rules based on network element type, and stores the generalized rules in the database. In the modeling and analysis unit of the online training module, the alarm clustering and grouping of the real-time data cleaning results in the time domain and space is performed. Then, based on the rule relationship table in the association rule base, the association relationship of the real-time alarm data in each group is searched. If there are corresponding secondary alarms and primary alarms in the same group, the primary alarm is located in real time, and secondary alarms are merged and compressed. For root cause alarms, orders are dispatched, and the alarms corresponding to the fault root network element are uploaded to the chain. The data cleaning unit collects current alarms in real time, extracts key fields required for fault location, including alarm occurrence time, alarm network element name, alarm network element ID, alarm network element type, alarm title, network management alarm ID, resource pool, and specialty. It removes alarms with missing key fields, merges frequent alarms, filters intermittent alarms and worthless alarms, and performs normalization preprocessing to clean and transform all data. Based on the DBSCAN density clustering algorithm, alarm slice time domain division is implemented. According to operation and maintenance experience, alarms caused by the same network element failure have a certain correlation in terms of time and space. Therefore, based on the resource pool information in the cloud network alarm information, alarm data is first divided into spatial domains, and alarms belonging to the same resource pool are grouped together. Furthermore, by combining the occurrence time of alarm data with the DBSCAN clustering algorithm, the time domain of batch alarms is automatically divided, and alarms that may be related are grouped into one time window. This avoids root cause alarms and derived alarms not being in the same time window due to manual division, and finally obtains the grouping of batch alarm data. Fault localization is achieved based on the Eclat association mining algorithm. In the root cause localization scenario, the Eclat item set is a combination of "network element name + alarm name", which is used for subsequent mining of frequent itemsets and association rules. Alarm pairs with no relationship are filtered out by support, and the primary and secondary association rules of alarms are determined based on confidence and the lift is used to exclude whether the associated alarms belong to co-alarms. The generalization of primary and secondary association rules is achieved based on probability statistics. Since the rules generated by the Eclat algorithm are too detailed and specific, in most cases, alarm data cannot be completely matched one-to-one with alarms of the same network management alarm ID and the same network element ID. The system determines whether primary and secondary alarms occur on the same network element and labels them with 0 and 1. Alarms with the same primary and secondary alarm ID, primary and secondary network element ID, and whether they belong to the same network element are logically merged.

Images