5G alarm data mining method and device

Abstract

The present invention provides a 5G alarm data mining method and device, comprising: obtaining fifth-generation mobile communication 5G alarm data; filtering and processing the 5G alarm data based on a Spark framework to obtain 5G raw alarm data; performing data cleaning on the 5G raw alarm data to obtain processed 5G raw alarm data; compressing the processed 5G raw alarm data based on a distributed frequent pattern growth (FP-growth) algorithm to obtain a pruned alarm type frequent item set; labeling and encoding the pruned alarm type frequent item set based on the alarm type to obtain an encoded frequent item set; and inputting the encoded frequent item set into a pre-trained multi-task deep neural network model to output a target result. The present invention effectively improves the accuracy and efficiency of fault root cause location using 5G alarm data.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for mining 5G alarm data. Background Art

[0002] The fifth generation of mobile communications th 5G generation, as a new type of network architecture, has the characteristics of large amount of alarm data and more difficult to locate the root cause of the alarm.

[0003] Traditional fourth generation mobile communications th The root cause mining method for 5G (4G) alarms collects a large amount of 4G alarm data and mines the root cause of the alarm using an artificial expert rule knowledge base or an algorithm related to association rules. Mining the root cause of 5G alarms is an urgent problem. Summary of the Invention

[0004] The present invention provides a 5G alarm data mining method and device to solve the problem of mining the root cause of 5G alarms.

[0005] In a first aspect, the present invention provides a 5G alarm data mining method, comprising:

[0006] Acquire fifth-generation mobile communication 5G alarm data based on the operation and maintenance center (OMC) system, filter and process the 5G alarm data based on the Spark framework to obtain 5G raw alarm data, and store the 5G raw alarm data in a Hadoop cluster;

[0007] Performing data cleaning on the 5G original alarm data to obtain processed 5G original alarm data;

[0008] Compressing the processed 5G original alarm data based on the distributed frequent pattern growth fp-growth algorithm to obtain a pruned alarm type frequent item set;

[0009] Performing data labeling on the pruned frequent itemsets of the alarm type based on the alarm type to obtain labeled frequent itemsets;

[0010] encoding the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets;

[0011] Inputting the encoded frequent itemsets into a pre-trained multi-task deep neural network model and outputting a target result, wherein the multi-task deep neural network model includes a first-level classification model and a second-level classification model;

[0012] Among them, the multi-task deep neural network model takes the 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding as input, and is trained using the fault types corresponding to the 5G original alarm data samples as sample labels.

[0013] In one embodiment, the compressed 5G original alarm data after processing is performed based on the distributed frequent pattern growth fp-growth algorithm to obtain the pruned alarm type frequent itemset, specifically including:

[0014] Dividing the processed 5G original alarm data based on a sliding time window of a preset size and step length to obtain transactions corresponding to the 5G original alarm data within the sliding time window;

[0015] Calculate the support of the alarm type in the transaction based on the Spark framework;

[0016] Filtering the alarm types in the transaction based on a preset minimum support threshold to obtain filtered transactions;

[0017] Performing a pre-pruning operation on the filtered transactions to generate frequent item sets;

[0018] Pruning the frequent itemsets based on a dual-correlation factor algorithm of a time series of 5G alarms and a network element topology to obtain a pruned frequent itemset;

[0019] A frequent pattern FP tree is constructed based on the pruned frequent item sets, and the FP tree is traversed to obtain the pruned alarm type frequent item sets.

[0020] In one embodiment, the pruning of the frequent itemsets based on the dual-correlation algorithm of the time series and network element topology of the 5G alarm specifically includes:

[0021] Calculate the error rate of the frequent itemsets not belonging to the same network element topology structure based on the dual-correlation factor algorithm of the time series of 5G alarms and the network element topology;

[0022] If the error rate is greater than a preset threshold, the frequent itemsets are pruned.

[0023] In one embodiment, the algorithm based on the dual correlation factors of the time series of 5G alarms and the network element topology calculates the error rate of the frequent itemsets not belonging to the same network element topology structure, specifically including:

[0024] Using formulas (1) and (2), calculate the error rate of the frequent itemsets not being in the same network element topology:

[0025]

[0026]

[0027] Among them, e(q) represents the number of sample misjudgments under node q, c(t,r) q represents the spatiotemporal correlation coefficient of the samples under the node q, s(q) represents the total number of samples under the node q, t represents the time parameter, r represents the network element topology parameter, α represents the weight of the number of sample misjudgments, β represents the weight of the spatiotemporal correlation coefficient, max(y tm -y tn ) represents the maximum time series interval between two samples in the frequent item set, y tm and y tn represents the time series value corresponding to the sample in the frequent item set, c r (y ri ,y ri+1 ) represents the network element topology structure correlation coefficient between two samples in the frequent item set, y ri and y ri+1 represents the network element topology parameter corresponding to the sample in the frequent item set, q represents the frequent item set corresponding to the sliding time window, and the sample represents the 5G alarm data in the frequent item set.

[0028] In one embodiment, the multi-task deep neural network model is trained using 5G raw alarm data samples that have undergone data cleaning, compression, data labeling, and encoding as input and using the fault types corresponding to the 5G raw alarm data samples as sample labels, specifically including:

[0029] Input the 5G original alarm data sample that has undergone data cleaning, compression, data labeling, and encoding processing into the first-level classification model, and output the first target result; input the 5G original alarm data sample that has undergone data cleaning, compression, data labeling, and encoding processing into the second-level classification model, and output the second target result;

[0030] Calculating a first loss function value based on the first target result and the first type of fault corresponding to the 5G original alarm data sample, and calculating a second loss function value based on the second target result and the second type of fault corresponding to the 5G original alarm data sample;

[0031] Using a backpropagation algorithm, adjusting the various parameters of the first-level classification model starting from the output layer of the first-level classification model so as to minimize the value of the first loss function, and adjusting the various parameters of the second-level classification model starting from the output layer of the second-level classification model so as to minimize the value of the second loss function;

[0032] Determine whether the training end condition is met. If so, save the parameters of the first-level classification model and the second-level classification model of the current iteration to obtain a trained multi-task deep neural network model.

[0033] In one embodiment, the step of labeling the pruned frequent itemsets of the alarm type based on the alarm type to obtain the labeled frequent itemsets specifically includes:

[0034] Based on the alarm type, multi-granularity labeling and merge labeling are performed on the pruned alarm type frequent itemsets to obtain labeled frequent itemsets.

[0035] In one embodiment, performing data cleaning on the 5G original alarm data to obtain processed 5G original alarm data specifically includes:

[0036] Performing data completion processing on the 5G original alarm data to obtain first preprocessed data;

[0037] performing redundancy removal processing on the first preprocessed data to obtain second preprocessed data;

[0038] Information integrity processing is performed on the second preprocessed data to obtain processed 5G original alarm data.

[0039] In a second aspect, the present invention provides a 5G alarm data mining device, comprising:

[0040] A data screening unit is used to obtain fifth-generation mobile communication 5G alarm data based on the operation and maintenance center OMC system, screen and process the 5G alarm data based on the spark framework to obtain 5G original alarm data, and store the 5G original alarm data in a hadoop cluster;

[0041] A data cleaning unit, configured to perform data cleaning processing on the 5G original alarm data to obtain processed 5G original alarm data;

[0042] a data compression unit, configured to compress the processed 5G original alarm data based on a distributed frequent pattern growth (fp-growth) algorithm to obtain a pruned alarm type frequent item set;

[0043] a data labeling unit, configured to label the pruned frequent itemsets of the alarm type based on the alarm type to obtain labeled frequent itemsets;

[0044] a data encoding unit, configured to encode the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets;

[0045] An output unit is used to input the encoded frequent item set into a pre-trained multi-task deep neural network model and output a target result. The multi-task deep neural network model includes a first-level classification model and a second-level classification model; wherein the multi-task deep neural network model takes the 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding as input, and is trained with the fault types corresponding to the 5G original alarm data samples as sample labels.

[0046] In a third aspect, the present invention provides an electronic device comprising a memory and a memory storing a computer program, wherein when the processor executes the program, the steps of the 5G alarm data mining method described in the first aspect are implemented.

[0047] In a fourth aspect, the present invention provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the steps of the 5G alarm data mining method described in the first aspect.

[0048] The 5G alarm data mining method and device provided by the present invention collect 5G alarm data through the operation and maintenance center OMC system, screen and process the 5G alarm data to obtain 5G original alarm data, perform data cleaning on the 5G original alarm data to improve the quality of the 5G original alarm data, compress the processed 5G original alarm data based on the distributed FP-growth algorithm, load balance the performance differences between Hadoop cluster nodes, label and encode the pruned alarm type frequent item sets, and input the encoding results into a pre-trained multi-task deep neural network model, effectively improving the accuracy and efficiency of fault root cause location of 5G alarm data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 This is a flow chart of the 5G alarm data mining method provided by the present invention;

[0051] Figure 2 Schematic diagram of the framework of the method for collecting 5G original alarm data provided by the present invention;

[0052] Figure 3 This is a schematic diagram of the framework of the multi-task deep neural network model training process provided by the present invention;

[0053] Figure 4 Schematic diagram of the framework for root cause location of 5G raw alarm data based on a multi-task deep neural network model provided by the present invention;

[0054] Figure 5 It is a structural diagram of the 5G alarm data mining device provided by the present invention;

[0055] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0057] In order to solve the problem of mining the root cause of 5G alarms, the present invention provides a 5G alarm data mining method. Figure 1 FIG. 1 is a flow chart of a 5G alarm data mining method provided by an embodiment of the present invention, such as Figure 1 As shown, the 5G alarm data mining method provided by the present invention includes:

[0058] Step 100: Acquire fifth-generation mobile communication 5G alarm data based on the operation and maintenance center OMC system, filter and process the 5G alarm data based on the spark framework to obtain 5G original alarm data, and store the 5G original alarm data in the hadoop cluster.

[0059] 5G alarm data indicates alarms generated by abnormal conditions such as 5G network communication equipment failures. Common 5G alarm data includes wireless equipment failure alarms, dynamic and environmental equipment failure alarms, core network equipment failure alarms, data network equipment failure alarms, and transmission equipment failure alarms. Table 1 shows the content of 5G alarm data.

[0060] Table 1 Contents of 5G alarm data

[0061]

[0062] The Operation Maintenance Center (OMC) system refers to the Operation Maintenance Center system, which is used to collect 5G alarm data from various network elements in 5G network communication equipment such as core network and wireless network.

[0063] The Spark framework represents a computing engine framework in big data technology and is used to filter 5G alarm data obtained through the socket communication interface.

[0064] Hadoop cluster refers to the Hadoop cluster server in big data technology, which is used to store data.

[0065] 5G network communication equipment reports 5G alarm data to the OMC system. Electronic equipment obtains the 5G alarm data collected in the OMC system through the socket communication interface. Based on spark technology, the 5G alarm data obtained by the socket communication interface is filtered to obtain the 5G original alarm data. The 5G original alarm data is stored in the Hadoop cluster through the message queue.

[0066] In one implementation, a network management system (NMS) is used to monitor and reconnect the process of 5G alarm data obtained by the socket communication interface, thereby ensuring real-time and full collection of 5G alarm data.

[0067] In one implementation, the working status of the Hadoop cluster is automatically inspected, and servers with abnormal working status are obtained in real time, thereby ensuring the stable operation of each server in the Hadoop cluster.

[0068] Step 101: Perform data cleaning on the 5G original alarm data to obtain processed 5G original alarm data.

[0069] Data cleaning processing includes removing / completing missing data, modifying data with format and content errors, removing data with logical errors, and removing redundant data.

[0070] The electronic equipment performs data cleaning processing on the 5G original alarm data, and performs missing data completion processing, redundant data removal processing and information integrity processing in sequence, and finally obtains the processed 5G original alarm data.

[0071] Step 102: compress the processed 5G original alarm data based on the distributed frequent pattern growth fp-growth algorithm to obtain a pruned alarm type frequent item set.

[0072] In order to mine valuable data information from massive 5G raw alarm data, the electronic device performs correlation analysis on the processed 5G raw alarm data based on the distributed frequent-pattern-growth (fp-growth) algorithm, generates association rules, and compresses the processed 5G raw alarm data based on the association rules to obtain the pruned frequent item set of alarm types.

[0073] Furthermore, the distribution of the pruned alarm type frequent itemsets is verified based on expert verification to avoid uneven data distribution and evaluate the rationality of the data.

[0074] Association rules are used to instruct electronic devices to discover associations or correlations from a data set, that is, to identify frequently occurring attribute value sets from the data set, also known as frequent item sets, and then use these frequent item sets to create rules that describe the association relationships.

[0075] Frequent itemsets are used to indicate that when the support of a data item set exceeds a preset threshold, the data item set is called a frequent itemset.

[0076] Support is used to indicate how frequently a data item appears in the same data set.

[0077] Data items are used to indicate data types.

[0078] Step 103: perform data labeling on the pruned alarm type frequent itemsets based on the alarm types to obtain labeled frequent itemsets.

[0079] Optionally, data labeling includes multi-granularity labeling and merged labeling.

[0080] Multi-granularity labeling is used to indicate the labeling of large-category fault types and small-category fault types for the pruned alarm type frequent itemsets.

[0081] Merge labeling is used to indicate the merging of the pruned alarm type frequent item sets that have a single labeled small class of fault types and contain different 5G original alarm data.

[0082] Step 104: Encode the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets.

[0083] In one embodiment, the encoding process adopts a binary encoding method. The electronic device judges each 5G original alarm data in the marked frequent item set based on the alarm type A corresponding to the marked frequent item set. If the alarm type of the 5G original alarm data is A, the 5G original alarm data is encoded as 1. If the alarm type of the 5G original alarm data is not A, it is confirmed that the 5G original alarm data is encoded as 0. After encoding all the 5G original alarm data in the marked frequent item set, a binary sequence is obtained.

[0084] Step 105: Input the encoded frequent item set into a pre-trained multi-task deep neural network model to output the target result, wherein the multi-task deep neural network model includes a first-level classification model and a second-level classification model; wherein the multi-task deep neural network model takes the 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding as input, and is trained with the fault type corresponding to the 5G original alarm data sample as the sample label.

[0085] It should be noted that the target result is the output result corresponding to the encoded frequent item set, including the output result of the first-level classification model and the output result of the second-level classification model.

[0086] In one embodiment, for a frequent item set within a time window, the encoded frequent item set is input into a first-level classification model to obtain the output result of the first-level classification model, the encoded frequent item set is input into a second-level classification model to obtain the output result of the second-level classification model, and the output result A with a top-1 confidence level is obtained from the output result of the first-level classification model. The output result A is associated with all the output results of the second-level classification model, and the output result B is screened out, wherein the output result B represents the output result of the second-level classification model that belongs to the output result A.

[0087] Furthermore, the top-3 confidence data are obtained from the output result B as the effective output result of the second-level classification model.

[0088] Furthermore, based on the pre-trained multi-task deep neural network model, all valid output results corresponding to the frequent itemset are counted. For all valid output results, the idea of ​​voting is used to calculate the output results with the highest triggering frequency and the top-3 confidence level within the time window as the target results; if the number of all output results corresponding to the frequent itemset is less than 3, all output results are used as target results.

[0089] It should be noted that the output result of the first-level classification model is the major fault type corresponding to the encoded frequent item set, and the output result of the second-level classification model is the minor fault type corresponding to the encoded frequent item set.

[0090] The 5G alarm data mining method provided by the present invention collects 5G alarm data through the operation and maintenance center OMC system, screens and processes the 5G alarm data to obtain 5G original alarm data, performs data cleaning on the 5G original alarm data to improve the quality of the 5G original alarm data, compresses the processed 5G original alarm data based on the distributed FP-growth algorithm, load balances the performance differences between Hadoop cluster nodes, annotates and encodes the pruned alarm type frequent item sets, and inputs the encoding results into a pre-trained multi-task deep neural network model, effectively improving the accuracy and efficiency of fault root cause location of 5G alarm data.

[0091] Combine Figure 2 This section explains the principles of collecting 5G raw alarm data. Figure 2 It is a schematic diagram of the framework of the method for collecting 5G original alarm data provided by an embodiment of the present invention.

[0092] like Figure 2 As shown in the figure, Ne represents a 5G base station, which is used to generate 5G alarm data. The OMC system represents an operation and maintenance center system, which is used to collect 5G alarm data of each network element in 5G network communication equipment such as the core network and wireless network. The NMS network management system represents a network management system, which is used to monitor the process of obtaining 5G alarm data through the socket communication interface and reconnect the socket communication interface. The Hadoop cluster represents a cluster server, which is used to store data.

[0093] Among them, the Ne device reports the 5G alarm data to the OMC system, the socket communication interface obtains the collected 5G alarm data from the OMC system, and filters the 5G alarm data obtained from the socket interface based on spark technology to obtain the 5G original alarm data, and stores the 5G original alarm data in the hadoop cluster in the form of a message queue.

[0094] In one implementation, the dynamic ring equipment, core network equipment, data network equipment and transmission equipment report the 5G alarm data to the OMC system, the socket communication interface obtains the collected 5G alarm data from the OMC system, and filters the 5G alarm data obtained from the socket interface based on spark technology to obtain the 5G original alarm data, and stores the 5G original alarm data in the Hadoop cluster in the form of a message queue.

[0095] Furthermore, in order to ensure the real-time and full collection of 5G alarm data, the NMS network management system monitors the process of obtaining 5G alarm data through the socket communication interface in real time and reconnects the socket communication interface.

[0096] Furthermore, in order to ensure the stable operation of each server in the Hadoop cluster, the working status of the Hadoop cluster is automatically inspected, and servers with abnormal working status are obtained in real time.

[0097] Based on any of the above embodiments, the compressed data of the processed 5G original alarm data is compressed based on the distributed frequent pattern growth fp-growth algorithm to obtain the pruned alarm type frequent item set, specifically including:

[0098] Dividing the processed 5G original alarm data based on a sliding time window of a preset size and step length to obtain transactions corresponding to the 5G original alarm data within the sliding time window;

[0099] Calculate the support of the alarm type in the transaction based on the Spark framework;

[0100] Filtering the alarm types in the transaction based on a preset minimum support threshold to obtain filtered transactions;

[0101] Performing a pre-pruning operation on the filtered transactions to generate frequent item sets;

[0102] Pruning the frequent itemsets based on a dual-correlation factor algorithm of a time series of 5G alarms and a network element topology to obtain a pruned frequent itemset;

[0103] A frequent pattern FP tree is constructed based on the pruned frequent item sets, and the FP tree is traversed to obtain the pruned alarm type frequent item sets.

[0104] In one implementation, the size of the sliding time window is set to 180s, the step length is 60s, all 5G original alarm data within a sliding time window is regarded as a transaction, and the sliding time window is used to divide all processed 5G original alarm data.

[0105] Specifically, the electronic device uses a sliding time window to divide all processed 5G original alarm data to obtain transactions corresponding to all 5G original alarm data within the sliding time window.

[0106] Furthermore, for each transaction, all 5G original alarm data in the transaction are further divided based on the alarm type that occurs in the transaction. The frequency of occurrence of each alarm type is calculated through the spark framework, which is used as the support of the alarm type corresponding to the frequency. Based on the preset minimum support threshold, the alarm types in the transaction are filtered to obtain the alarm types that meet the preset conditions.

[0107] In one embodiment, for each transaction, alarm types that meet preset conditions are sorted from large to small based on the support of the alarm types, and a set of alarm types ranked in the top 80% is selected as the filtered transaction.

[0108] It should be noted that, for all 5G original alarm data in each transaction, each 5G original alarm data corresponds one-to-one to an alarm type, and each alarm type corresponds to at least one 5G original alarm data.

[0109] Furthermore, the electronic device pre-prunes each filtered transaction to generate frequent itemsets. Based on a dual-correlation factor algorithm based on the time series of 5G alarms and network element topology, each frequent itemset is pruned to obtain a pruned frequent itemset. A frequent-pattern (FP) tree is constructed for the pruned frequent itemsets, and the FP tree is traversed to obtain a pruned frequent itemset of the alarm type. Traversing the FP tree effectively improves the efficiency of reading 5G alarm data.

[0110] In one implementation, the distribution of 5G original alarm data is verified by expert verification, which can effectively avoid uneven data distribution and improve data quality.

[0111] The 5G alarm data mining method provided by the embodiment of the present invention divides all 5G original alarm data based on a sliding time window, uses the spark framework and preset conditions to filter the transactions corresponding to each sliding time window, pre-prunes the transaction set, prunes the frequent item sets based on the dual-correlation factor algorithm of the 5G alarm time series and network element topology, constructs an FP tree for the pruned frequent item sets, and then generates association rules. The association rules are used to compress and verify the 5G original alarm data to obtain the pruned alarm type frequent item sets, effectively avoiding uneven data distribution and improving the quality of 5G alarm data.

[0112] Based on any of the foregoing embodiments, the pruning of the frequent itemsets based on the dual-correlation factor algorithm of the time series and network element topology of the 5G alarm specifically includes:

[0113] Calculate the error rate of the frequent itemsets not belonging to the same network element topology structure based on the dual-correlation factor algorithm of the time series of 5G alarms and the network element topology;

[0114] If the error rate is greater than a preset threshold, the frequent itemsets are pruned.

[0115] The dual-correlation factor algorithm of time series and network element topology of 5G alarms includes two dimensions: time series and network element topology. It calculates the time series interval value and network element topology structure correlation coefficient of any two 5G original alarm data in the frequent item set, and then calculates the error rate when the frequent item sets do not belong to the same network element topology structure.

[0116] Furthermore, it is determined whether the error rate of the frequent itemsets not belonging to the same network element topology structure meets a preset condition. If so, a pruning operation is performed on the frequent itemsets.

[0117] The preset condition is that the error rate of frequent item sets not belonging to the same network element topology structure is greater than a preset threshold.

[0118] The 5G alarm data mining method provided by an embodiment of the present invention calculates the error rate of the frequent item sets that do not belong to the same network element topology structure based on the dual-correlation factor algorithm of the time series of 5G alarms and the network element topology. When the error rate of the frequent item sets that do not belong to the same network element topology structure meets the preset conditions, the frequent item sets are pruned, thereby effectively improving the quality of the 5G alarm data, and thereby improving the accuracy and efficiency of the root cause positioning of the 5G alarm data.

[0119] Based on any of the foregoing embodiments, the algorithm based on the dual correlation factors of the time series of 5G alarms and the network element topology calculates the error rate in which the frequent itemsets do not belong to the same network element topology structure, specifically including:

[0120] Using formulas (1) and (2), calculate the error rate of the frequent itemsets not being in the same network element topology:

[0121]

[0122]

[0123] Among them, e(q) represents the number of sample misjudgments under node q, c(t,r) q represents the spatiotemporal correlation coefficient of the samples under the node q, s(q) represents the total number of samples under the node q, t represents the time parameter, r represents the network element topology parameter, α represents the weight of the number of sample misjudgments, β represents the weight of the spatiotemporal correlation coefficient, max(y tm -y tn ) represents the maximum time series interval between two samples in the frequent item set, y tm and y tn represents the time series value corresponding to the sample in the frequent item set, c r (y ri ,y ri+1 ) represents the network element topology structure correlation coefficient between two samples in the frequent item set, y ri and yri+1 represents the network element topology parameter corresponding to the sample in the frequent item set, q represents the frequent item set corresponding to the sliding time window, and the sample represents the 5G alarm data in the frequent item set.

[0124] Specifically, the pruning condition is f(q)≥f(q'), where f(q') represents a preset threshold. When the error rate of node q is greater than the preset threshold and the node q is not in the same network element topology structure, the node q is pruned so that the sample error and spatial correlation index in the pruned node q are lower after weighting.

[0125] It should be noted that if the time series intervals between samples in a frequent item set are small and they belong to the same network element topology, the spatiotemporal correlation coefficients of the samples in the frequent item set are higher; on the contrary, if the time series intervals between samples in a frequent item set are large, or they belong to different network element topologies, the spatiotemporal correlation coefficients of the samples in the frequent item set are lower. Among them, if the samples in a frequent item set belong to the edge of the sliding time window and belong to different network element topologies, the spatiotemporal correlation coefficients of the frequent item set are the lowest.

[0126] The 5G alarm data mining method provided by an embodiment of the present invention calculates the error rate of the frequent item sets that do not belong to the same network element topology structure based on the dual-correlation factor algorithm of the time series of 5G alarms and the network element topology. When the error rate of the frequent item sets that do not belong to the same network element topology structure meets the preset conditions, the frequent item sets are pruned, thereby effectively improving the quality of the 5G alarm data, and thereby improving the accuracy and efficiency of the root cause positioning of the 5G alarm data.

[0127] Based on any of the above embodiments, the multi-task deep neural network model is trained using 5G original alarm data samples that have undergone data cleaning, compression, data labeling, and encoding as input and using the fault types corresponding to the 5G original alarm data samples as sample labels, specifically including:

[0128] Input the 5G original alarm data sample that has undergone data cleaning, compression, data labeling, and encoding processing into the first-level classification model, and output the first target result; input the 5G original alarm data sample that has undergone data cleaning, compression, data labeling, and encoding processing into the second-level classification model, and output the second target result;

[0129] Calculating a first loss function value based on the first target result and the first type of fault corresponding to the 5G original alarm data sample, and calculating a second loss function value based on the second target result and the second type of fault corresponding to the 5G original alarm data sample;

[0130] Using a backpropagation algorithm, adjusting the various parameters of the first-level classification model starting from the output layer of the first-level classification model so as to minimize the value of the first loss function, and adjusting the various parameters of the second-level classification model starting from the output layer of the second-level classification model so as to minimize the value of the second loss function;

[0131] Determine whether the training end condition is met. If so, save the parameters of the first-level classification model and the second-level classification model of the current iteration to obtain a trained multi-task deep neural network model.

[0132] The first target result is the data result corresponding to the 5G original alarm data sample, and the second target result is the data result corresponding to the 5G original alarm data sample.

[0133] In one implementation, the first type of fault is a major type of fault, and the second type of fault is a minor type of fault.

[0134] Specifically, the 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding processing are used as input, and the fault types corresponding to the 5G original alarm data samples are used as sample labels to train the multi-task deep neural network model. The number of input and output neurons of the multi-task deep neural network model is determined according to the 5G original alarm data samples and the fault type sample labels corresponding to the 5G original alarm data samples, and the number of hidden layers and neurons is determined by empirical rules, exhaustive search and other methods.

[0135] In one embodiment, if the number of feature points of the 5G original alarm data sample that has undergone data cleaning, compression, data labeling and encoding processing is 20, and the number of feature points of the first type of fault type corresponding to the 5G original alarm data sample is 5, then the number of nodes in the input layer in the first-level classification model is 20, the number of neuron nodes in the output layer in the first-level classification model is 5, the number of hidden layers is 2, and the number of nodes in each hidden layer is 8.

[0136] In one implementation, weight initialization is performed using methods such as Xavier initialization or He initialization.

[0137] In one embodiment, the Tanh activation function, the ReLU activation function and the Sigmoid activation function are used to train the multi-task deep neural network model respectively. According to the training results, it can be concluded that the ReLU activation function can solve the problem of gradient disappearance in the positive interval, and at the same time make some neurons 0, thereby leading to the sparsity of the multi-task deep neural network model, and reducing the mutual dependence between the parameters in the multi-task deep neural network model, alleviating the overfitting problem, and its calculation speed and convergence speed are faster than the Tanh activation function and the Sigmoid activation function. Therefore, the ReLU activation function is used to train the multi-task deep neural network model.

[0138] In one embodiment, a dropout strategy is used to train a multi-task deep neural network model, and a small number of neurons are shielded with a certain probability to avoid overfitting and enhance the robustness of the model.

[0139] It should be noted that the 5G original alarm data that has undergone data cleaning, compression, data labeling and encoding processing is used as a sample, and the first-level classification model is trained with the first type of fault type corresponding to the 5G original alarm data as the sample label; the 5G original alarm data that has undergone data cleaning, compression, data labeling and encoding processing is used as a sample, and the second-level classification model is trained with the second type of fault type corresponding to the 5G original alarm data as the sample label. The training process of the two models can be carried out simultaneously or separately, without any particular order.

[0140] The 5G alarm data mining method provided by an embodiment of the present invention takes 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding processing as input, and trains a multi-task deep neural network model with the fault type corresponding to the 5G original alarm data sample as the sample label to obtain a trained task deep neural network model, thereby effectively improving the accuracy and efficiency of fault root cause locating of 5G alarm data.

[0141] Combine Figure 3 Illustrate the training process of a multi-task deep neural network model. Figure 3 This is a schematic diagram of the framework of the multi-task deep neural network model training process provided by an embodiment of the present invention.

[0142] like Figure 3As shown, the 5G original alarm data samples are subjected to data cleaning processing, association rule processing and encoder encoding processing to obtain preprocessed 5G original alarm data samples, and the preprocessed 5G original alarm data samples are input into the input layer of the multi-task deep neural network model to obtain output results. Based on the output results of the output layer and the fault type sample labels corresponding to the 5G original alarm data samples, the dropout strategy is used to train the multi-task deep neural network model to obtain a trained multi-task deep neural network model.

[0143] Combine Figure 4 This paper describes how to locate the root cause of 5G raw alarm data based on a multi-task deep neural network model. Figure 4 This is a schematic diagram of a framework for root cause location of 5G raw alarm data based on a multi-task deep neural network model provided by an embodiment of the present invention.

[0144] like Figure 4 As shown, Model 1 represents the first-level classification model, and Model 2 represents the second-level classification model.

[0145] The alarm transaction coding data is input into the first-level classification model, and multiple major fault types corresponding to the alarm transaction coding data are output, which are fault major category 1, fault major category 2, ... and fault major category a in sequence; the alarm transaction coding data is input into the second-level classification model, and multiple minor fault types corresponding to the alarm transaction coding data are output, which are fault minor category 1, fault minor category 2, fault minor category 3, fault minor category 4, ... and fault minor category b in sequence.

[0146] Determine the root cause of the 5G original alarm data based on the major fault type and the minor fault type corresponding to the alarm transaction coding data.

[0147] Based on any of the above embodiments, the step of labeling the pruned frequent itemsets of the alarm type based on the alarm type to obtain the labeled frequent itemsets specifically includes:

[0148] Based on the alarm type, multi-granularity labeling and merge labeling are performed on the pruned alarm type frequent itemsets to obtain labeled frequent itemsets.

[0149] Multi-granularity labeling is used to indicate the labeling of large-category fault types and small-category fault types for the pruned alarm type frequent itemsets based on the alarm types.

[0150] In one implementation, for the pruned alarm type frequent item set A, the major fault type to which it is labeled is 1, and the minor fault type to which it is labeled is 1-1; for the pruned alarm type frequent item set B, the major fault type to which it is labeled is 1, and the minor fault type to which it is labeled is 1-2; for the pruned alarm type frequent item set C, the major fault type to which it is labeled is 2, and the minor fault type to which it is labeled is 2-1; for the pruned alarm type frequent item set D, the major fault type to which it is labeled is 2, and the minor fault type to which it is labeled is 2-2. Then A, B, C, and D are the data set P of the first granularity, A and B are the first data subset Q1 in the second granularity, and C and D are the second data subset Q2 in the second granularity, where the first granularity is the major fault type and the second granularity is the minor fault type.

[0151] In one implementation, the pruned alarm type frequent item set corresponding to the wireless device failure is labeled as a major fault type 1, the pruned alarm type frequent item set corresponding to the dynamic environment equipment failure is labeled as a major fault type 2, the pruned alarm type frequent item set corresponding to the core network equipment failure is labeled as a major fault type 3, the pruned alarm type frequent item set corresponding to the data network equipment failure is labeled as a major fault type 4, and the pruned alarm type frequent item set corresponding to the transmission equipment failure is labeled as a major fault type 5.

[0152] In one embodiment, wireless device failures include clock failures, license failures, remote radio unit (RRU) failures, baseband unit (BBU) failures, and network management disconnection failures, wherein the pruned alarm type frequent item set corresponding to the clock failure belongs to a subclass fault type of 1-1, the pruned alarm type frequent item set corresponding to the license failure belongs to a subclass fault type of 1-2, the pruned alarm type frequent item set corresponding to the RRU failure belongs to a subclass fault type of 1-3, the pruned alarm type frequent item set corresponding to the BBU failure belongs to a subclass fault type of 1-4, and the pruned alarm type frequent item set corresponding to the network management disconnection failure belongs to a subclass fault type of 1-5.

[0153] Merge labeling is used to indicate the merging of pruned alarm type frequent item sets that can be labeled with a single small class of fault types but contain different 5G original alarm data.

[0154] In one implementation, the pruned alarm type frequent item set A is {a, b}, the pruned alarm type frequent item set B is {a, b, c}, and the subclass fault type label to which A and B belong is 1-1, then A∩B={a, b}, where A∩B is used to indicate the merging processing of A and B.

[0155] It should be noted that based on the alarm type, the pruned alarm type frequent item set can be labeled with major fault types, and based on the major fault type, the pruned alarm type frequent item set can be further labeled with minor fault types.

[0156] It should be noted that multi-granularity labeling is performed on all pruned alarm type frequent itemsets, and merge labeling is performed on some pruned alarm type frequent itemsets that meet the merge labeling conditions.

[0157] The 5G alarm data mining method provided in an embodiment of the present invention performs multi-granularity labeling and merge labeling on the pruned alarm type frequent item set based on the alarm type, which can effectively improve the reliability of the multi-task deep neural network model and further improve the accuracy and efficiency of fault root cause location of 5G alarm data.

[0158] Based on any of the foregoing embodiments, performing data cleaning on the 5G original alarm data to obtain processed 5G original alarm data specifically includes:

[0159] Performing data completion processing on the 5G original alarm data to obtain first preprocessed data;

[0160] performing redundancy removal processing on the first preprocessed data to obtain second preprocessed data;

[0161] Information integrity processing is performed on the second preprocessed data to obtain processed 5G original alarm data.

[0162] The electronic device obtains the stored 5G original alarm data from the Hadoop cluster, performs data cleaning on the 5G original alarm data, and obtains the processed 5G original alarm data.

[0163] In one embodiment, data completion processing is performed on the 5G original alarm data. For each missing data in the post-verified 5G original alarm data, a file synchronization request is sent to the OMC system through the socket communication interface to obtain the file fed back by the OMC. Based on the fed-back file, the alarm content corresponding to the alarm ID of the missing data is obtained, and the missing data is completed according to the alarm content to obtain the first preprocessed data.

[0164] In one embodiment, the first preprocessed data is de-redundant, and the de-redundant is used to indicate that data with a low alarm level and no or little impact on actual business in the first preprocessed data is not written into the Hadoop cluster to obtain second preprocessed data.

[0165] The method of removing redundant processing can ensure that the second pre-processed data written into the Hadoop cluster is data with a high correlation with the occurrence of the fault, thereby avoiding the influence of redundant data with low correlation on the association rule operation and neural network training based on the original 5G alarm data.

[0166] In one embodiment, information integrity processing is performed on the second preprocessed data, and the information integrity processing is used to indicate that data that is missing important field information in the second preprocessed data is not written into the Hadoop cluster. If the data that is missing important field information belongs to data with a higher alarm level, data completion processing is performed on the data, and the data is repeatedly collected to obtain the processed 5G original alarm data.

[0167] Information integrity processing can effectively ensure the integrity of 5G original alarm data information and prevent incomplete information from affecting association rule operations and neural network training based on 5G original alarm data.

[0168] The 5G alarm data mining method provided by the embodiment of the present invention sequentially performs data completion processing, redundancy removal processing, and information integrity processing on the 5G original alarm data to obtain processed 5G original alarm data, thereby improving the integrity and effectiveness of the 5G original alarm data.

[0169] The 5G alarm data mining device provided by the present invention is described below. The 5G alarm data mining device described below and the 5G alarm data mining method described above can be referenced to each other.

[0170] Figure 5 Schematic diagram of a 5G alarm data mining device provided by an embodiment of the present invention, such as Figure 5 As shown, the 5G alarm data mining device includes: a data screening unit 500, a data cleaning unit 501, a data compression unit 502, a data marking unit 503, a data encoding unit 504 and an output unit 505, wherein:

[0171] The data screening unit 500 is configured to obtain fifth-generation mobile communication 5G alarm data based on the operation and maintenance center (OMC) system, screen and process the 5G alarm data based on the Spark framework to obtain 5G raw alarm data, and store the 5G raw alarm data in a Hadoop cluster.

[0172] The data cleaning unit 501 is configured to perform data cleaning processing on the 5G original alarm data to obtain processed 5G original alarm data;

[0173] A data compression unit 502 is configured to compress the processed 5G original alarm data based on a distributed frequent pattern growth (fp-growth) algorithm to obtain a pruned alarm type frequent item set;

[0174] A data labeling unit 503 is configured to label the pruned frequent itemsets of the alarm type based on the alarm type to obtain labeled frequent itemsets;

[0175] A data encoding unit 504 is configured to encode the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets;

[0176] The output unit 505 is used to input the encoded frequent item set into a pre-trained multi-task deep neural network model and output the target result. The multi-task deep neural network model includes a first-level classification model and a second-level classification model; wherein the multi-task deep neural network model takes the 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding as input, and is trained with the fault type corresponding to the 5G original alarm data sample as the sample label.

[0177] The 5G alarm data mining device provided by the present invention collects 5G alarm data through the operation and maintenance center OMC system, filters and processes the 5G alarm data to obtain 5G original alarm data, performs data cleaning on the 5G original alarm data to improve the quality of the 5G original alarm data, compresses the processed 5G original alarm data based on the distributed FP-growth algorithm, load balances the performance differences between Hadoop cluster nodes, annotates and encodes the pruned alarm type frequent item sets, and inputs the encoding results into a pre-trained multi-task deep neural network model, effectively improving the accuracy and efficiency of fault root cause location of 5G alarm data.

[0178] Optionally, the data compression unit 502 is further configured to:

[0179] Dividing the processed 5G original alarm data based on a sliding time window of a preset size and step length to obtain transactions corresponding to the 5G original alarm data within the sliding time window;

[0180] Calculate the support of the alarm type in the transaction based on the Spark framework;

[0181] Filtering the alarm types in the transaction based on a preset minimum support threshold to obtain filtered transactions;

[0182] Performing a pre-pruning operation on the filtered transactions to generate frequent item sets;

[0183] Pruning the frequent itemsets based on a dual-correlation factor algorithm of a time series of 5G alarms and a network element topology to obtain a pruned frequent itemset;

[0184] A frequent pattern FP tree is constructed based on the pruned frequent item sets, and the FP tree is traversed to obtain the pruned alarm type frequent item sets.

[0185] Optionally, the pruning of the frequent itemsets based on the dual-correlation factor algorithm of the time series and network element topology of the 5G alarm specifically includes:

[0186] Calculate the error rate of the frequent itemsets not belonging to the same network element topology structure based on the dual-correlation factor algorithm of the time series of 5G alarms and the network element topology;

[0187] If the error rate is greater than a preset threshold, the frequent itemsets are pruned.

[0188] Optionally, the calculating of the error rate of the frequent itemsets not belonging to the same network element topology structure based on the dual-correlation factor algorithm of the time series of 5G alarms and the network element topology specifically includes:

[0189] Using formulas (1) and (2), calculate the error rate of the frequent itemsets not being in the same network element topology:

[0190]

[0191]

[0192] Among them, e(q) represents the number of sample misjudgments under node q, c(t, r) q represents the spatiotemporal correlation coefficient of the samples under the node q, s(q) represents the total number of samples under the node q, t represents the time parameter, r represents the network element topology parameter, α represents the weight of the number of sample misjudgments, β represents the weight of the spatiotemporal correlation coefficient, max(y tm -y tn ) represents the maximum time series interval between two samples in the frequent item set, y tm and y tn represents the time series value corresponding to the sample in the frequent item set, c r (y ri ,y ri+1 ) represents the network element topology structure correlation coefficient between two samples in the frequent item set, y ri and y ri+1represents the network element topology parameter corresponding to the sample in the frequent item set, q represents the frequent item set corresponding to the sliding time window, and the sample represents the 5G alarm data in the frequent item set.

[0193] Optionally, the output unit 505 is further configured to:

[0194] Input the 5G original alarm data sample that has undergone data cleaning, compression, data labeling, and encoding processing into the first-level classification model, and output the first target result; input the 5G original alarm data sample that has undergone data cleaning, compression, data labeling, and encoding processing into the second-level classification model, and output the second target result;

[0195] Calculating a first loss function value based on the first target result and the first type of fault corresponding to the 5G original alarm data sample, and calculating a second loss function value based on the second target result and the second type of fault corresponding to the 5G original alarm data sample;

[0196] Using a backpropagation algorithm, adjusting the various parameters of the first-level classification model starting from the output layer of the first-level classification model so as to minimize the value of the first loss function, and adjusting the various parameters of the second-level classification model starting from the output layer of the second-level classification model so as to minimize the value of the second loss function;

[0197] Determine whether the training end condition is met. If so, save the parameters of the first-level classification model and the second-level classification model of the current iteration to obtain a trained multi-task deep neural network model.

[0198] Optionally, the data tagging unit 503 is further configured to:

[0199] Based on the alarm type, multi-granularity labeling and merge labeling are performed on the pruned alarm type frequent itemsets to obtain labeled frequent itemsets.

[0200] Optionally, the data cleaning unit 501 is further configured to:

[0201] Performing data completion processing on the 5G original alarm data to obtain first preprocessed data;

[0202] performing redundancy removal processing on the first preprocessed data to obtain second preprocessed data;

[0203] Information integrity processing is performed on the second preprocessed data to obtain processed 5G original alarm data.

[0204] The 5G alarm data mining device provided by the present invention can achieve Figure 1The various processes implemented by the method embodiment achieve the same technical effect and are not described here again to avoid repetition.

[0205] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call a computer program in the memory 630 to execute the steps of the 5G alarm data mining method, for example, including:

[0206] Acquire fifth-generation mobile communication 5G alarm data based on the operation and maintenance center (OMC) system, filter and process the 5G alarm data based on the Spark framework to obtain 5G raw alarm data, and store the 5G raw alarm data in a Hadoop cluster;

[0207] Performing data cleaning on the 5G original alarm data to obtain processed 5G original alarm data;

[0208] Compressing the processed 5G original alarm data based on the distributed frequent pattern growth fp-growth algorithm to obtain a pruned alarm type frequent item set;

[0209] Performing data labeling on the pruned frequent itemsets of the alarm type based on the alarm type to obtain labeled frequent itemsets;

[0210] encoding the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets;

[0211] Inputting the encoded frequent itemsets into a pre-trained multi-task deep neural network model and outputting a target result, wherein the multi-task deep neural network model includes a first-level classification model and a second-level classification model;

[0212] Among them, the multi-task deep neural network model takes the 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding as input, and is trained using the fault types corresponding to the 5G original alarm data samples as sample labels.

[0213] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0214] On the other hand, the present invention further provides a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer is capable of performing the 5G alarm data mining method provided by the above methods, the method comprising:

[0215] Acquire fifth-generation mobile communication 5G alarm data based on the operation and maintenance center (OMC) system, filter and process the 5G alarm data based on the Spark framework to obtain 5G raw alarm data, and store the 5G raw alarm data in a Hadoop cluster;

[0216] Performing data cleaning on the 5G original alarm data to obtain processed 5G original alarm data;

[0217] Compressing the processed 5G original alarm data based on the distributed frequent pattern growth fp-growth algorithm to obtain a pruned alarm type frequent item set;

[0218] Performing data labeling on the pruned frequent itemsets of the alarm type based on the alarm type to obtain labeled frequent itemsets;

[0219] encoding the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets;

[0220] Inputting the encoded frequent itemsets into a pre-trained multi-task deep neural network model and outputting a target result, wherein the multi-task deep neural network model includes a first-level classification model and a second-level classification model;

[0221] Among them, the multi-task deep neural network model takes the 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding as input, and is trained using the fault types corresponding to the 5G original alarm data samples as sample labels.

[0222] On the other hand, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, wherein the computer program is configured to cause the processor to execute the methods provided in the above embodiments, for example, including:

[0223] Acquire fifth-generation mobile communication 5G alarm data based on the operation and maintenance center (OMC) system, filter and process the 5G alarm data based on the Spark framework to obtain 5G raw alarm data, and store the 5G raw alarm data in a Hadoop cluster;

[0224] Performing data cleaning on the 5G original alarm data to obtain processed 5G original alarm data;

[0225] Compressing the processed 5G original alarm data based on the distributed frequent pattern growth fp-growth algorithm to obtain a pruned alarm type frequent item set;

[0226] Performing data labeling on the pruned frequent itemsets of the alarm type based on the alarm type to obtain labeled frequent itemsets;

[0227] encoding the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets;

[0228] Inputting the encoded frequent itemsets into a pre-trained multi-task deep neural network model and outputting a target result, wherein the multi-task deep neural network model includes a first-level classification model and a second-level classification model;

[0229] Among them, the multi-task deep neural network model takes the 5G original alarm data samples that have undergone data cleaning, compression, data labeling and encoding as input, and is trained using the fault types corresponding to the 5G original alarm data samples as sample labels.

[0230] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NANDFLASH), solid-state drives (SSDs)), etc.

[0231] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0232] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0233] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A 5G alarm data mining method, characterized in that: include: Acquire fifth-generation mobile communication 5G alarm data based on the operation and maintenance center (OMC) system, filter and process the 5G alarm data based on the Spark framework to obtain 5G raw alarm data, and store the 5G raw alarm data in a Hadoop cluster; Performing data cleaning on the 5G original alarm data to obtain processed 5G original alarm data; Compressing the processed 5G original alarm data based on the distributed frequent pattern growth fp-growth algorithm to obtain a pruned alarm type frequent item set; Performing data labeling on the pruned frequent itemsets of the alarm type based on the alarm type to obtain labeled frequent itemsets; encoding the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets; Inputting the encoded frequent itemsets into a pre-trained multi-task deep neural network model and outputting a target result, wherein the multi-task deep neural network model includes a first-level classification model and a second-level classification model; The multi-task deep neural network model is trained using 5G original alarm data samples that have undergone data cleaning, compression, data labeling, and encoding as input and the fault types corresponding to the 5G original alarm data samples as sample labels; The processed 5G original alarm data is compressed based on the distributed frequent pattern growth fp-growth algorithm to obtain a pruned alarm type frequent item set, including: Prune frequent itemsets based on the dual-correlation factor algorithm of 5G alarm time series and network element topology to obtain pruned frequent itemsets; The algorithm based on the dual-correlation factors of the time series and network element topology of the 5G alarm is used to prune the frequent itemsets, specifically including: Calculate the error rate of the frequent itemsets not belonging to the same network element topology structure based on the dual-correlation factor algorithm of the time series of 5G alarms and the network element topology; If the error rate is greater than a preset threshold, pruning the frequent itemsets; The algorithm based on the dual correlation factors of the time series of 5G alarms and the network element topology calculates the error rate of the frequent itemsets not belonging to the same network element topology structure, specifically including: Using formulas (1) and (2), calculate the error rate when the frequent itemsets are not in the same network element topology: （1） （2） in, Indicates that the node The number of sample misclassifications under Indicates that the node The spatiotemporal correlation coefficient of the samples under Indicates that the node The total number of samples under represents the time parameter, Indicates the network element topology parameters, Indicates the weight of the number of sample misjudgments, represents the weight of the spatiotemporal correlation coefficient, It represents the maximum time series interval between two samples in the frequent itemset. and Represents the time series value corresponding to the sample in the frequent item set, represents the network element topology correlation coefficient between two samples in the frequent item set, and represents the network element topology parameters corresponding to the sample in the frequent item set, represents the frequent itemset corresponding to the sliding time window, and sample represents the 5G alarm data in the frequent itemset.

2. The 5G alarm data mining method according to claim 1, characterized in that: The compressed 5G original alarm data after processing is performed based on the distributed frequent pattern growth fp-growth algorithm to obtain the pruned alarm type frequent item set, specifically including: Dividing the processed 5G original alarm data based on a sliding time window of a preset size and step length to obtain transactions corresponding to the 5G original alarm data within the sliding time window; Calculate the support of the alarm type in the transaction based on the Spark framework; Filtering the alarm types in the transaction based on a preset minimum support threshold to obtain filtered transactions; Performing a pre-pruning operation on the filtered transactions to generate frequent item sets; Pruning the frequent itemsets based on a dual-correlation factor algorithm of a time series of 5G alarms and a network element topology to obtain a pruned frequent itemset; A frequent pattern FP tree is constructed based on the pruned frequent item sets, and the FP tree is traversed to obtain the pruned alarm type frequent item sets.

3. The 5G alarm data mining method according to claim 1, characterized in that: The multi-task deep neural network model is trained using 5G raw alarm data samples that have undergone data cleaning, compression, data labeling, and encoding as input and using the fault types corresponding to the 5G raw alarm data samples as sample labels. Specifically, it includes: Input the 5G original alarm data sample that has undergone data cleaning, compression, data labeling, and encoding processing into the first-level classification model, and output the first target result; input the 5G original alarm data sample that has undergone data cleaning, compression, data labeling, and encoding processing into the second-level classification model, and output the second target result; Calculating a first loss function value based on the first target result and the first type of fault corresponding to the 5G original alarm data sample, and calculating a second loss function value based on the second target result and the second type of fault corresponding to the 5G original alarm data sample; Using a backpropagation algorithm, adjusting the various parameters of the first-level classification model starting from the output layer of the first-level classification model so as to minimize the value of the first loss function, and adjusting the various parameters of the second-level classification model starting from the output layer of the second-level classification model so as to minimize the value of the second loss function; Determine whether the training end condition is met. If so, save the parameters of the first-level classification model and the second-level classification model of the current iteration to obtain a trained multi-task deep neural network model.

4. The 5G alarm data mining method according to claim 1, characterized in that: The step of labeling the pruned frequent itemsets of the alarm type based on the alarm type to obtain the labeled frequent itemsets specifically includes: Based on the alarm type, multi-granularity labeling and merge labeling are performed on the pruned alarm type frequent itemsets to obtain labeled frequent itemsets.

5. The 5G alarm data mining method according to claim 1, characterized in that: The data cleaning process of the 5G original alarm data to obtain the processed 5G original alarm data specifically includes: Performing data completion processing on the 5G original alarm data to obtain first preprocessed data; performing redundancy removal processing on the first preprocessed data to obtain second preprocessed data; Information integrity processing is performed on the second preprocessed data to obtain processed 5G original alarm data.

6. A 5G alarm data mining device, characterized in that: include: A data screening unit is used to obtain fifth-generation mobile communication 5G alarm data based on the operation and maintenance center OMC system, screen and process the 5G alarm data based on the spark framework to obtain 5G original alarm data, and store the 5G original alarm data in a hadoop cluster; A data cleaning unit, configured to perform data cleaning processing on the 5G original alarm data to obtain processed 5G original alarm data; a data compression unit, configured to compress the processed 5G original alarm data based on a distributed frequent pattern growth (fp-growth) algorithm to obtain a pruned alarm type frequent item set; a data labeling unit, configured to label the pruned frequent itemsets of the alarm type based on the alarm type to obtain labeled frequent itemsets; a data encoding unit, configured to encode the labeled frequent itemsets based on a multi-granularity encoder to obtain encoded frequent itemsets; An output unit is configured to input the encoded frequent itemsets into a pre-trained multi-task deep neural network model and output a target result, wherein the multi-task deep neural network model includes a first-level classification model and a second-level classification model; wherein the multi-task deep neural network model is trained using 5G original alarm data samples that have undergone data cleaning, compression, data labeling, and encoding as input and using the fault types corresponding to the 5G original alarm data samples as sample labels; The processed 5G original alarm data is compressed based on the distributed frequent pattern growth fp-growth algorithm to obtain a pruned alarm type frequent item set, including: Prune frequent itemsets based on the dual-correlation factor algorithm of 5G alarm time series and network element topology to obtain pruned frequent itemsets; The algorithm based on the dual-correlation factors of the time series and network element topology of the 5G alarm is used to prune the frequent itemsets, specifically including: Calculate the error rate of the frequent itemsets not belonging to the same network element topology structure based on the dual-correlation factor algorithm of the time series of 5G alarms and the network element topology; If the error rate is greater than a preset threshold, pruning the frequent itemsets; The algorithm based on the dual correlation factors of the time series of 5G alarms and the network element topology calculates the error rate of the frequent itemsets not belonging to the same network element topology structure, specifically including: Using formulas (1) and (2), calculate the error rate when the frequent itemsets are not in the same network element topology: （1） （2） in, Indicates that the node The number of sample misclassifications under Indicates that the node The spatiotemporal correlation coefficient of the samples under Indicates that the node The total number of samples under represents the time parameter, Indicates the network element topology parameters, Indicates the weight of the number of sample misjudgments, represents the weight of the spatiotemporal correlation coefficient, It represents the maximum time series interval between two samples in the frequent itemset. and Represents the time series value corresponding to the sample in the frequent item set, represents the network element topology correlation coefficient between two samples in the frequent item set, and represents the network element topology parameters corresponding to the sample in the frequent item set, represents the frequent itemset corresponding to the sliding time window, and sample represents the 5G alarm data in the frequent itemset.

7. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of the 5G alarm data mining method according to any one of claims 1 to 5 are implemented.

8. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, which is used to enable the processor to execute the steps of the 5G alarm data mining method according to any one of claims 1 to 5.

Images