Dynamic threshold alarm method and system based on communication big data AI time sequence capacity prediction

Abstract

The invention discloses a dynamic threshold value alarm method and system based on communication big data AI time sequence capacity prediction, belongs to the technical field of communication, and aims to solve the technical problems that a traditional capacity prediction algorithm is slow in prediction, needs a compensation value, is large in prediction deviation, and is high in prediction efficiency. The technical scheme includes that the method includes data collection and preprocessing, to be specific, collecting various time sequence original data indexes in real time from a performance management system of a communication network, and performing time summarization, network element summarization and KPI (key performance indicator) calculation processing on the collected data to obtain preprocessed data; feature analysis and modeling: carrying out feature analysis on the preprocessed data, identifying a distribution rule of the data, pre-defining the number of training data sets and the number of prediction data sets of an AI prediction model through a foreground configuration page, and selecting the AI prediction model according to data distribution features and a use scene; packaging the AI prediction model through a Restful API (Application Program Interface); and generating alarm information according to the dynamic threshold.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a dynamic threshold warning method and system based on communication big data AI time series capacity prediction. Background Art

[0002] With the rapid development of big data and artificial intelligence (AI) technologies, the structure of communication networks has become increasingly complex, and the data volume has increased sharply. In particular, time series data such as site availability, network traffic, call drop rate, etc. have become important data sources for monitoring and warning, and are the basis for network development consulting, network planning, and optimization. Currently, almost all operators have an urgent new demand for network prediction. Traditional capacity prediction algorithms, such as enhanced capacity prediction linear algorithms, Snowden time algorithms, etc., have slow algorithm prediction, require compensation values, have large prediction deviations, and do not support multi-index factor prediction algorithms; moreover, the original system using the static threshold warning method has difficulty meeting the requirements of the current complex business scenarios, and there are problems such as inaccurate and lagging warning information, and it cannot effectively reflect the future operation of the system in a timely manner. Summary of the Invention

[0003] The technical task of the present invention is to provide a dynamic threshold warning method and system based on communication big data AI time series capacity prediction to solve the problems of slow prediction, requiring compensation values, large prediction deviations of traditional capacity prediction algorithms, not supporting multi-index factor prediction, and inaccurate warning information of the static threshold algorithm.

[0004] The technical task of the present invention is achieved in the following manner. A dynamic threshold warning method based on communication big data AI time series capacity prediction is as follows:

[0005] Data collection and preprocessing: Real-time collect various types of time series raw data indicators from the performance management system of the communication network, and perform time aggregation, network element aggregation, and KPI calculation processing on the collected data to obtain preprocessed data;

[0006] Feature analysis and modeling: Perform feature analysis on the preprocessed data, identify the distribution law of the data, and predefine the number of training data sets and prediction data sets of the AI prediction model through the front-end configuration page, and select the AI prediction model according to the data distribution characteristics and usage scenarios; among them, the AI prediction model includes a moving average model or a time series analysis model; the time series analysis model includes an ARIMA model, an AUTO ARIMA model, a Random Forest model, and a GBDT model;

[0007] Restful API interface encapsulates the AI prediction model: Use Flask to encapsulate the Restful API call of the AI prediction model;

[0008] The Dashboard calls the AI prediction and displays data: Through the Dashboard configuration page, select the corresponding training data set from the aggregated high-dimensional data, configure the used AI prediction model, and after querying, call the Restful API to obtain the results of the AI prediction, and display the historical data and training data set data on the front-end page;

[0009] Generate alarm information according to the dynamic threshold: Based on the historical data and training data set obtained by the AI prediction model, generate alarms according to the configured dynamic threshold rules; among them, the dynamic threshold rule refers to dynamically adjusting the threshold used to determine the abnormal or normal state according to the real-time changes of communication data.

[0010] Preferably, the time-series raw data metrics include the site availability rate, site traffic, site call drop rate, CPU utilization rate, memory utilization rate, and network bandwidth utilization rate of 2G Domain, 3G Domain, LTE Domain, 5G Domain, Core Domain, Transmission Domain of multiple manufacturers or directly connected devices;

[0011] Time aggregation is to aggregate 15-minute data into data at hourly, daily, weekly, and monthly granularities;

[0012] Network element aggregation is to aggregate Cell data into Site, City, and Region levels;

[0013] When calculating the KPI, use the original metric configuration formula to obtain the KPI value.

[0014] Preferably, the data collection and preprocessing are specifically as follows:

[0015] Trigger the collection task every five minutes, collect data from the peripheral system using the SFTP / SNMP protocol, where the EMS file formats include csv, xml, and binary, and obtain the corresponding device metrics from the SNMP directly connected devices and generate pia files;

[0016] Convert the EMS file into a unified format pia file through the parsing rules;

[0017] Store the parsed pia file into the Hbase database;

[0018] Obtain the raw data from the Hbase database, and complete the KPI calculation, time aggregation, and network element aggregation for the raw data;

[0019] Store the aggregated high-dimensional data into Hive for storage.

[0020] Preferably, the Restful API call for encapsulating the AI prediction model using Flask is specifically as follows:

[0021] Use Flask to encapsulate the Restful API interface, receive input training data, and pass the training data to the AI prediction model for prediction processing;

[0022] Return the prediction results of the AI prediction model;

[0023] At the same time, run the Restful API in Docker, directly call the Restful API to obtain AI prediction data, and solve the platform deployment and basic environment problems of multiple operating system versions.

[0024] Preferably, the Dashboard calls the AI prediction and displays data as follows:

[0025] The Dashboard configures the selection domain and the AI prediction model;

[0026] Select the entity to be queried according to the AI prediction model;

[0027] Select the metrics to be queried and predicted;

[0028] Select the time granularity and time range of the training result set, and the time range should meet the requirements of the number of training sets of the subsequent AI prediction model;

[0029] Configure the AI prediction model algorithm;

[0030] Call the API of the AI prediction model;

[0031] Return the result set of the AI prediction;

[0032] Display the predicted Chart in the icon page, configure the comparison of historical values for the prediction results, and verify the prediction accuracy.

[0033] More preferably, generate alarm information according to the dynamic threshold as follows:

[0034] Use the AI prediction model to predict the data within the set future time, and obtain the prediction base value;

[0035] According to the historical data, determine the distribution range of the data, and divide it into the upper threshold sample space and the lower threshold sample space;

[0036] According to the distribution of the sample points with the largest deviation, perform secondary threshold calculation on the sample space without the maximum deviation sample points to obtain a more reasonable dynamic threshold range; and introduce a tolerance coefficient to further optimize the dynamic threshold to meet the requirements of different business scenarios.

[0037] A dynamic threshold alarm system based on AI time series capacity prediction of communication big data, the system includes:

[0038] The data collection and preprocessing module is used to collect various time series raw data indicators from the performance management system of the communication network in real time, and to perform time aggregation, network element aggregation and KPI calculation processing on the collected data to obtain preprocessed data;

[0039] The feature analysis and modeling module is used to perform feature analysis on preprocessed data, identify data distribution patterns, and predefine the number of training data sets and prediction data sets for the AI ​​prediction model through the front-end configuration page, and select the AI ​​prediction model according to the data distribution characteristics and usage scenarios; among them, the AI ​​prediction model includes the moving average model or the time series analysis model; the time series analysis model includes the ARIMA model, the AUTO ARIMA model, the Random Forest model, and the GBDT model;

[0040] Model encapsulation module, used to encapsulate the Restful API call of AI prediction model using Flask;

[0041] The calling module is used to select the corresponding training data set through the Dashboard configuration page for the aggregated high-dimensional data, configure the AI ​​prediction model to be used, call the Restful API after querying to obtain the AI ​​prediction results, and display the historical data and training data set data on the front page;

[0042] Dynamic threshold generation module, which is used to generate alarms based on historical data and training data sets obtained by AI prediction models according to the configured dynamic threshold rules

[0043] Preferably, the data collection and preprocessing module includes:

[0044] The collection submodule is used to trigger the collection task every five minutes and use the SFTP / SNMP protocol to collect data from the peripheral system. The EMS file format includes csv, xml and binary. The SNMP direct connection device obtains the indicators of the corresponding device and generates a pia file.

[0045] The parsing submodule is used to convert the EMS file into a unified format pia file according to the parsing rules;

[0046] The storage submodule is used to store the parsed pia file into the Hbase database;

[0047] The summary submodule is used to obtain the original data from the Hbase database and complete KPI calculation, time summary and network element summary for the original data;

[0048] The post-aggregation storage submodule is used to store the aggregated high-dimensional data into Hive for storage.

[0049] An electronic device, comprising: a memory and at least one processor;

[0050] Wherein, a computer program is stored on the memory;

[0051] The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the dynamic threshold warning method based on communication big data AI time series capacity prediction as described above.

[0052] A computer-readable storage medium, in which a computer program is stored, and the computer program can be executed by a processor to implement the dynamic threshold warning method based on communication big data AI time series capacity prediction as described above.

[0053] The dynamic threshold warning method and system based on communication big data AI time series capacity prediction of the present invention have the following advantages:

[0054] (1) The present invention dynamically adjusts the warning threshold by AI intelligent analysis and prediction of time series data in the communication network, realizes efficient and accurate fault and capacity early warning, effectively solves the disadvantages of slow prediction, requiring compensation values, large prediction deviation, and not supporting multi-index factor prediction of traditional capacity prediction algorithms, and at the same time introduces a dynamic threshold to solve the problem of inaccurate warning information of the static threshold algorithm;

[0055] (2) The result of the preprocessing of the present invention can reflect the operation status of the communication network and equipment in real time, and uses monitoring tools such as Dashboard and warning to monitor indicators such as network performance and availability;

[0056] (3) Through the dynamic threshold generation mechanism of the present invention, the warning threshold can be dynamically adjusted according to the change rules of real-time data and historical data, improving the accuracy of warning. Combined with the AI prediction algorithm, it can adapt to the prediction warning requirements of different business scenarios and complex network environments, respond in time or expand capacity, and improve the stability and reliability of the communication network;

[0057] (4) The present invention continuously optimizes the algorithm and model according to the actual situation of the warning and the feedback of the operation and maintenance personnel, improving the accuracy and effectiveness of the warning; at the same time, analyzing the warning historical data to discover potential problems and trends, providing a reference for future operation and maintenance and capacity expansion work;

[0058] (5) The present invention uses four AI time series prediction models, including ARIMA, AUTO ARIMA, RandomForest, and GBDT, encapsulates the interface for calling the AI algorithm using Flask, and deploys the algorithm in Docker to ensure availability on multiple operating system platforms; at the same time, through dynamic threshold configuration, the accuracy of the prediction warning algorithm is improved;

[0059] (6) The present invention, through functions such as preprocessing of historical data of traffic and network and other indicators, data aggregation and analysis, data prediction by AI (Artificial Intelligence) time series algorithm, and dynamic threshold warning, completely describes the implementation principles of prediction, display, and warning. Based on the historical laws of relevant indicator data, it solves the shortcomings of the original prediction algorithm, can greatly improve the accuracy and efficiency of traffic and network prediction, and meets the needs of operators for efficient network operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The present invention will be further described below with reference to the accompanying drawings.

[0061] Appendix Figure 1 It is a flowchart of a dynamic threshold warning method for AI time series capacity prediction based on communication big data;

[0062] Appendix Figure 2 It is a screenshot of the prediction display effect interface for the ARIMA algorithm;

[0063] Appendix Figure 3 It is a screenshot of the prediction display effect interface for the AUTO ARIMA algorithm;

[0064] Appendix Figure 4 It is a screenshot of the prediction display effect interface for the Random Forest algorithm;

[0065] Appendix Figure 5 It is a screenshot of the prediction display effect interface for the GBDT algorithm;

[0066] Appendix Figure 6 It is a screenshot of the interface for determining the distribution range of data based on historical data and dividing it into the upper threshold sample space and the lower threshold sample space. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] The dynamic threshold warning method and system for AI time series capacity prediction based on communication big data of the present invention will be described in detail below with reference to the accompanying drawings of the specification and specific embodiments.

[0068] Embodiment 1:

[0069] As shown in the appendix Figure 1 This embodiment discloses a dynamic threshold warning method for AI time series capacity prediction based on communication big data, and the method is as follows:

[0070] S1. Data collection and preprocessing: Real-time collect various types of time-series raw data indicators from the performance management system of the communication network, and perform time aggregation, network element aggregation, and KPI calculation processing on the collected data to obtain preprocessed data;

[0071] S2. Feature Analysis and Modeling: Perform feature analysis on the preprocessed data, identify the distribution patterns of the data, and predefine the number of training datasets and prediction datasets for the AI prediction model through the front-end configuration page. Select the AI prediction model according to the data distribution characteristics and usage scenarios. Among them, the AI prediction model includes a moving average model or a time series analysis model.

[0072] S3. Wrap the AI Prediction Model with a Restful API: Use Flask to wrap the Restful API call of the AI prediction model.

[0073] S4. Dashboard Calls the AI Prediction and Displays Data: Select the corresponding training datasets through the Dashboard configuration page for the aggregated high-dimensional data, configure the used AI prediction model, query and then call the Restful API to obtain the results of the AI prediction, and display the historical data and training dataset data on the front-end page.

[0074] S5. Generate Alarm Information According to the Dynamic Threshold: Based on the historical data and training datasets obtained by the AI prediction model, generate alarms according to the configured dynamic threshold rules. Among them, the dynamic threshold rules refer to dynamically adjusting the threshold for determining abnormal or normal states according to the real-time changes of communication data.

[0075] S6. Alarm Optimization and Feedback: Continuously optimize the algorithms and models according to the actual situation of the alarms and the feedback from the operation and maintenance personnel to improve the accuracy and effectiveness of the alarms. At the same time, analyze the alarm historical data to discover potential problems and trends, providing references for future operation and maintenance and capacity expansion work.

[0076] The time-series raw data metrics in step S1 of this embodiment include the 2G Domain, 3G Domain, LTE Domain, 5G Domain, Core Domain, Transmission Domain of multiple vendors, or the site availability rate, site traffic, site call drop rate, CPU utilization rate, memory utilization rate, and network bandwidth utilization rate of directly connected devices.

[0077] Time aggregation is to aggregate 15-minute data into data at hourly, daily, weekly, and monthly granularities.

[0078] Network element aggregation is to aggregate Cell data into Site, City, and Region levels.

[0079] When calculating KPIs, use the original metric configuration formula to obtain the KPI values.

[0080] The data collection and preprocessing in step S1 of this embodiment are specifically as follows:

[0081] S101. Trigger the data collection task every five minutes, collect data from the peripheral system using the SFTP / SNMP protocol. Among them, the EMS file format includes csv, xml, and binary. The SNMP directly connected device obtains the indicators of the corresponding device and generates pia files;

[0082] S102. Convert the EMS file into a unified format pia file through parsing rules;

[0083] S103. Store the parsed pia file into the Hbase database;

[0084] S104. Obtain the original data from the Hbase database, and complete KPI calculation, time aggregation, and network element aggregation for the original data;

[0085] S105. Store the aggregated high-dimensional data into Hive for storage.

[0086] The time series analysis models in step S2 of this embodiment include the ARIMA model, AUTO ARIMA model, RandomForest model, and GBDT model;

[0087] Among them, the ARIMA model (Autoregressive Integrated Moving Average model), that is, the differential integrated moving average autoregressive model, is one of the time series prediction analysis methods.

[0088] Advantages: The model is simple and easy to implement; it can handle linear and non-linear trends; it can check residual autocorrelation.

[0089] Disadvantages: It may not be able to fit non-linear trends or complex time series: For very complex non-linear time series, the ARIMA model may not provide an accurate fit. It requires a large amount of data: Effective modeling and prediction of the ARIMA model usually require a large amount of historical data.

[0090] AUTO ARIMA is an automated time series prediction model. Based on the ARIMA model, it can automatically select the best model parameters to provide accurate prediction results.

[0091] Advantages: AUTO ARIMA can automatically select the best ARIMA model parameters without manual adjustment by the user, greatly saving time and effort and reducing the possibility of human errors. Based on the ARIMA (autoregressive moving average) model, AUTO ARIMA can accurately predict and analyze time series data.

[0092] Disadvantages: When dealing with large-scale datasets, the computational cost of AUTO ARIMA may be relatively high, requiring a long calculation time. Model complexity: Although AUTO ARIMA automates the selection of model parameters, the ARIMA model itself may be relatively complex and not easy to understand and interpret.

[0093] Random Forest is an ensemble learning method mainly used for classification and regression tasks. It consists of multiple decision trees, and improves the accuracy and stability of the model by integrating the prediction results of these decision trees.

[0094] Advantages: High prediction accuracy: By integrating multiple decision trees, Random Forest can usually provide higher prediction accuracy than a single decision tree. Strong robustness: Each tree only sees part of the data and part of the features, making the model more robust and less susceptible to noisy data. Handling high-dimensional data: Random Forest performs well in handling high-dimensional data and is not easily affected by the number of features.

[0095] Disadvantages: High computational resource consumption: Since a large number of decision trees need to be trained, Random Forest consumes a large amount of computational resources and memory. Poor model interpretability: The Random Forest model itself is relatively complex, and it is difficult to explain the decision-making path of a single decision tree, which may not be very applicable in scenarios that require high interpretability and transparency.

[0096] GBDT (Gradient Boosting Decision Tree) is an ensemble learning method used for classification and regression tasks. It combines multiple decision trees to improve the prediction ability of the model.

[0097] Advantages: GBDT can usually provide relatively high prediction accuracy, especially on complex datasets. It improves the prediction performance by gradually improving the model. GBDT can handle various types of features, including numerical and categorical data.

[0098] Disadvantages: Since GBDT is an iterative process and each step needs to fit the residuals, the training time may be relatively long, especially on large datasets. GBDT has multiple hyperparameters (such as learning rate, number of trees, and depth of trees), and these hyperparameters need to be tuned through cross-validation, which may be time-consuming.

[0099] A summary comparison of the above four algorithms is shown in the following table:

[0100]

[0101] This embodiment provides a front-end configuration page, which can pre-define the number of training data sets and prediction data sets of the AI prediction model. The optimal algorithm practices of various models are pre-configured, and customers can select appropriate algorithms according to the usage scenarios as shown in the following table:

[0102] Algorithm Name Algorithm Type Tranining Dataset Number Forecast Num Period ARIMA-120,5,5 ARIMA 120 5 5 AutOARIMA-96,14,5 AutoARIMA 96 14 5 RandomForest150,14,5 RandomForest 150 14 5 GBDT96,5,5 GBDT 96 5 5

[0103] The specific implementation of encapsulating the Restful API of the AI prediction model using Flask in step S3 of this embodiment is as follows:

[0104] S301. Use Flask to encapsulate the Restful API interface, receive the input training data, and pass the training data to the AI prediction model for prediction processing;

[0105] S302. Return the prediction result of the AI prediction model;

[0106] S303. At the same time, run the Restful API in Docker, and directly call the Restful API to obtain the AI prediction data, so as to solve the platform deployment and basic environment problems of multi-operating system versions.

[0107] The specific implementation of the Dashboard calling the AI prediction and displaying data in step S4 of this embodiment is as follows:

[0108] S401. The Dashboard configures the selection domain and the AI prediction model;

[0109] S402. Select the entity to be queried according to the AI prediction model;

[0110] S403. Select the metrics to be queried and predicted;

[0111] S404. Select the time granularity and time range of the training result set, and the time range should meet the requirements of the number of training sets of the subsequent AI prediction model;

[0112] S405. Configure the AI prediction model algorithm;

[0113] S406. Call the API of the AI prediction model;

[0114] S407. Return the result set of the AI prediction;

[0115] S408. Display the predicted Chart in the icon page, and configure the historical value comparison of the prediction result to verify the prediction accuracy.

[0116] The prediction display effects of the ARIMA model, AUTO ARIMA model, Random Forest model and GBDT model are shown in Appendix Figure 2 、AppendixFigure 3 , attached Figure 4 and attached Figure 5 as shown

[0117] Specifically, generating an alarm message according to the dynamic threshold in step S5 of this embodiment is as follows:

[0118] S501. Use the AI prediction model to predict the data within a set future time, and obtain the prediction base value;

[0119] S502. According to the historical data, determine the distribution range of the data, and divide it into an upper threshold sample space and a lower threshold sample space, as attached Figure 6 as shown

[0120] S503. According to the distribution of the sample points with the largest deviation, perform a secondary threshold calculation on the sample space without the largest deviation sample points to obtain a more reasonable dynamic threshold range; and introduce a tolerance coefficient to further optimize the dynamic threshold to meet the requirements of different business scenarios.

[0121] Embodiment 2:

[0122] This embodiment provides a dynamic threshold alarm system based on AI time series capacity prediction of communication big data. The system includes:

[0123] A data collection and preprocessing module, which is used to collect various types of original time series data indicators from the performance management system of the communication network in real time, and perform time aggregation, network element aggregation, and KPI calculation processing on the collected data to obtain preprocessed data;

[0124] A feature analysis and modeling module, which is used to perform feature analysis on the preprocessed data, identify the distribution law of the data, and predefine the number of training data sets and the number of prediction data sets of the AI prediction model through the front-end configuration page, and select the AI prediction model according to the data distribution characteristics and usage scenarios; among them, the AI prediction model includes a moving average model or a time series analysis model; the time series analysis model includes an ARIMA model, an AUTO ARIMA model, a Random Forest model, and a GBDT model;

[0125] A model encapsulation module, which is used to encapsulate the Restful API call of the AI prediction model using Flask;

[0126] A calling module, which is used to select the corresponding training data set through the Dashboard configuration page for the aggregated high-dimensional data, configure the used AI prediction model, query and call the Restful API to obtain the results of AI prediction, and display the historical data and the training data set data on the front-end page;

[0127] A dynamic threshold generation module, which is used to generate alarms according to the configured dynamic threshold rules based on the historical data and training data set obtained by the AI prediction model

[0128] The data collection and preprocessing module in this embodiment includes:

[0129] An acquisition sub-module, which is used to trigger an acquisition task every five minutes, collect data from the peripheral system using the SFTP / SNMP protocol, where the EMS file format includes csv, xml, and binary, and the SNMP directly connected device obtains the metrics of the corresponding device and generates a pia file;

[0130] A parsing sub-module, which is used to convert the EMS file into a unified format pia file through parsing rules;

[0131] An warehousing sub-module, which is used to warehouse the parsed pia file into the Hbase database;

[0132] A summarization sub-module, which is used to obtain the original data from the Hbase database and complete KPI calculation, time summarization, and network element summarization for the original data;

[0133] A post-summarization warehousing sub-module, which is used to warehouse the summarized high-dimensional data into Hive for storage.

[0134] Embodiment 3:

[0135] This embodiment also provides an electronic device, including: a memory and a processor;

[0136] Wherein, the memory stores computer execution instructions;

[0137] The processor executes the computer execution instructions stored in the memory, so that the processor executes the dynamic threshold alarm method based on communication big data AI time series capacity prediction in any embodiment of the present invention.

[0138] The processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0139] The memory can be used to store computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory and invoking the data stored in the memory, the processor realizes various functions of the electronic device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function, etc.; the data storage area can store the data created according to the use of the terminal, etc. In addition, the memory can also include high-speed random access memory, and can also include non-volatile memory, such as hard disks, memory, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash memory cards, at least one magnetic disk storage period, flash memory devices, or other volatile solid-state storage devices.

[0140] Embodiment 4:

[0141] This embodiment also provides a computer-readable storage medium, which stores multiple instructions. The instructions are loaded by the processor to enable the processor to execute the dynamic threshold warning method based on communication big data AI time series capacity prediction in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided. On this storage medium, software program codes for realizing the functions of any one of the above embodiments are stored, and the computer (or CPU or MPU) of the system or device reads and executes the program codes stored in the storage medium.

[0142] In this case, the program code read from the storage medium itself can realize the functions of any one of the above embodiments. Therefore, the program code and the storage medium storing the program code constitute a part of the present invention.

[0143] Embodiments of the storage medium for providing program codes include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer via a communication network.

[0144] In addition, it should be clear that not only can the functions of any one of the above embodiments be realized by executing the program code read by the computer, but also by making the operating system operating on the computer, etc. based on the instructions of the program code to complete part or all of the actual operations.

[0145] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then based on the instructions of the program code, the CPU or the like installed on the expansion board or the expansion unit is made to execute part or all of the actual operations, thereby implementing the functions of any one of the above embodiments.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic threshold alarm method based on communication big data AI time series capacity prediction, characterized in that: The method is as follows: Data collection and preprocessing: Collect various time-series raw data indicators from the performance management system of the communication network in real time, and perform time aggregation, network element aggregation, and KPI calculation on the collected data to obtain preprocessed data; Feature analysis and modeling: Perform feature analysis on preprocessed data to identify data distribution patterns, and predefine the number of training data sets and prediction data sets for the AI ​​prediction model through the front-end configuration page, and select the AI ​​prediction model based on the data distribution characteristics and usage scenarios; AI prediction models include moving average models or time series analysis models; time series analysis models include ARIMA models, AUTO ARIMA models, Random Forest models, and GBDT models; Restful API interface encapsulates AI prediction model: Use Flask to encapsulate the Restful API call of the AI ​​prediction model; Dashboard calls AI prediction and displays data: select the corresponding training data set through the Dashboard configuration page for the aggregated high-dimensional data, configure the AI ​​prediction model to be used, call the Restful API after querying to obtain the AI ​​prediction results, and display the historical data and training data set data on the front page; Generate alarm information based on dynamic thresholds: Generate alarms based on historical data and training data sets obtained by the AI ​​prediction model and the configured dynamic threshold rules.

2. According to claim 1, the dynamic threshold alarm method based on communication big data AI time series capacity prediction is characterized in that: The time series raw data indicators include site availability, site traffic, site call drop rate, CPU utilization, memory utilization, and network bandwidth utilization of 2G Domain, 3G Domain, LTE Domain, 5GDomain, Core Domain, Transmission Domain, or directly connected devices of multiple vendors; Time aggregation is to aggregate 15-minute data into hourly, daily, weekly, and monthly granularity data; Network element aggregation is to aggregate Cell data to the Site, City, and Region levels; When calculating KPI, the original indicator configuration formula is used to obtain the KPI value.

3. The dynamic threshold alarm method based on communication big data AI time series capacity prediction according to claim 1 is characterized in that: The data collection and preprocessing are as follows: The collection task is triggered every five minutes, and the SFTP / SNMP protocol is used to collect data from the peripheral system. The EMS file formats include csv, xml and binary. The SNMP direct connection device obtains the indicators of the corresponding device and generates a pia file. Convert the EMS file into a unified format pia file through parsing rules; Store the parsed pia file into the Hbase database; Get the original data from the Hbase database, and complete KPI calculation, time aggregation, and network element aggregation for the original data; The aggregated high-dimensional data is stored in Hive.

4. The dynamic threshold alarm method based on communication big data AI time series capacity prediction according to claim 1 is characterized in that: The Restful API call of the AI ​​prediction model encapsulated by Flask is as follows: Use Flask to encapsulate the Restful API interface, receive input training data, and pass the training data to the AI ​​prediction model for prediction processing; Returns the prediction results of the AI ​​prediction model; At the same time, run the Restful API in Docker and directly call the Restful API to obtain AI prediction data.

5. The dynamic threshold alarm method based on communication big data AI time series capacity prediction according to claim 1 is characterized in that: The Dashboard calls AI prediction and displays data as follows: Dashboard configuration selection domain and AI prediction model; Select entities to query based on AI prediction models; Select the metric to query and predict; Select the time granularity and time range of the training result set. The time range must meet the requirements for the number of training sets for the subsequent AI prediction model. Configure AI prediction model algorithm; Call the API of the AI ​​prediction model; Returns the result set of AI prediction; The forecast chart is displayed on the icon page, and the forecast results are compared with historical values ​​to verify the accuracy of the forecast.

6. The dynamic threshold alarm method based on communication big data AI time series capacity prediction according to any one of claims 1 to 5, characterized in that: The specific alarm information generated according to the dynamic threshold is as follows: Use AI prediction models to predict data within a set period of time in the future and obtain the prediction base value; According to historical data, determine the distribution range of data and divide it into upper threshold sample space and lower threshold sample space; According to the distribution of the sample point with the maximum deviation, a secondary threshold calculation is performed on the sample space without the maximum deviation sample point to obtain a more reasonable dynamic threshold range; A tolerance coefficient is introduced to further optimize the dynamic threshold to meet the needs of different business scenarios.

7. A dynamic threshold warning system based on communication big data AI time series capacity prediction, characterized in that: The system includes: The data collection and preprocessing module is used to collect various time series raw data indicators from the performance management system of the communication network in real time, and to perform time aggregation, network element aggregation and KPI calculation processing on the collected data to obtain preprocessed data; The feature analysis and modeling module is used to perform feature analysis on preprocessed data, identify data distribution patterns, and predefine the number of training data sets and prediction data sets for the AI ​​prediction model through the front-end configuration page, and select the AI ​​prediction model according to the data distribution characteristics and usage scenarios; among them, the AI ​​prediction model includes the moving average model or the time series analysis model; the time series analysis model includes the ARIMA model, the AUTO ARIMA model, the Random Forest model, and the GBDT model; Model encapsulation module, used to encapsulate the Restful API call of AI prediction model using Flask; The calling module is used to select the corresponding training data set through the Dashboard configuration page for the aggregated high-dimensional data, configure the AI ​​prediction model to be used, call the Restful API after querying to obtain the AI ​​prediction results, and display the historical data and training data set data on the front page; The dynamic threshold generation module is used to generate alarms based on the historical data and training data sets obtained by the AI ​​prediction model according to the configured dynamic threshold rules.

8. The dynamic threshold warning system based on communication big data AI time series capacity prediction according to claim 7 is characterized in that: The data collection and preprocessing modules include: The collection submodule is used to trigger the collection task every five minutes and use the SFTP / SNMP protocol to collect data from the peripheral system. The EMS file format includes csv, xml and binary. The SNMP direct connection device obtains the indicators of the corresponding device and generates a pia file. The parsing submodule is used to convert the EMS file into a unified format pia file according to the parsing rules; The storage submodule is used to store the parsed pia file into the Hbase database; The summary submodule is used to obtain the original data from the Hbase database and complete KPI calculation, time summary and network element summary for the original data; The post-aggregation storage submodule is used to store the aggregated high-dimensional data into Hive for storage.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the dynamic threshold alarm method based on communication big data AI time series capacity prediction as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the dynamic threshold alarm method based on communication big data AI time series capacity prediction as described in any one of claims 1 to 6.

Images