A 5GC-based index anomaly closed-loop management method and device

Abstract

The application relates to the technical field of network control, and particularly discloses a 5GC-based index abnormality closed-loop management method and device, which comprises the following steps: acquiring high-weight index time sequence data, filtering the data according to specific business rules of each index, inputting the training data into a trained first model to obtain future period data and maximum and minimum values of the period; and associating the future period data with an abnormality judgment rule to obtain index-level abnormality time sequence information, including an index abnormality level, abnormality score content and the like. The application realizes accurate prediction of index time sequence data in combination with a machine learning algorithm, obtains abnormality information of predicted index data in association with an abnormality judgment standard, realizes whole-process closed-loop management through individualized check setting and whole-process visual monitoring of abnormal network elements, enables system maintenance personnel to predict abnormal indexes of the network elements in advance, and avoids losses caused by untimely processing.

Description

Technical Field

[0001] This invention relates to the field of network control technology, and specifically to a method and apparatus for closed-loop management of indicator anomalies based on 5GC. Background Technology

[0002] Typically, when implementing closed-loop management for network element anomaly early warning, real-time periodic analysis and prediction are performed using high-weight indicators of specific network elements in the existing network. This data is then correlated with anomaly thresholds for related calculations. Automated verification of indicator predictions against blacklists and work order dispatch standards determines whether a work order should be dispatched. Combined with automated analysis of network element anomaly causes, this enables pre-monitoring of anomaly indicators and rapid fault location. After resolution, the province to which the network element belongs can query recent data to ensure anomaly clearance; otherwise, the anomaly is escalated and further follow-up is initiated. This achieves a closed-loop management process for anomaly network element early warning, alerting maintenance personnel to monitor network element faults in advance and reducing business risks. However, current 5GC core network element indicator anomaly early warning management processes cannot dispatch work orders based on personalized user needs. Errors in work order dispatching can occur in special scenarios such as network element cutovers, increasing maintenance workload and reducing alarm maintenance efficiency. Furthermore, the high alarm frequency of different indicators for the same network element makes comprehensive analysis of anomalies impossible.

[0003] In summary, to address the existing problems, this invention proposes a closed-loop management method and device for indicator anomalies based on 5GC. Summary of the Invention

[0004] The purpose of this invention is to provide a closed-loop management method and device for abnormal indicators based on 5GC, in order to solve the problems of the current full-process management of abnormal indicators of 5GC network elements, which cannot dispatch orders and issue warnings according to the personalized needs of users, and there are situations such as incorrect order dispatch in special scenarios such as network element cutover, which increases the workload of operation and maintenance, reduces the efficiency of alarm operation and maintenance, and the alarm frequency of different indicators of the same network element is high, making it impossible to comprehensively analyze its abnormal situation.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a closed-loop management method for abnormal indicators based on 5GC, the method comprising the following steps:

[0006] Step S100: Obtain time series data of high-weight indicators, filter the data according to the specific business rules of each indicator to obtain training data, and input the training data into the trained first model to obtain future time period data and the maximum and minimum values ​​of the future time period data;

[0007] Step S200: Associate the future time period data with the anomaly judgment rules to obtain indicator-level anomaly time series information, wherein the indicator-level anomaly time series information includes indicator anomaly level and anomaly score content;

[0008] Step S300: Perform automated comparison and verification based on the blacklist and dispatch conditions set by the user and the time-series information of indicator-level anomalies over a future period.

[0009] Step S400: If the verification passes, the system will automatically analyze the reasons for the abnormal network element indicators and send them along with the work order information to the province where the network element belongs, thereby improving the efficiency of manual monitoring and verification.

[0010] Step S500: Monitor network elements that may be abnormal in real time within the province, and quickly resolve network element abnormalities based on the analysis results of the abnormal network element indicators, and follow up on the handling.

[0011] Step S600: Monitor the indicator data of the abnormal network element over a recent period of time to determine whether the abnormality has been cleared. If it has returned to normal, the case can be closed; otherwise, the abnormality is upgraded and follow-up processing continues.

[0012] In a preferred embodiment of the present invention, the high-weight index time series data in step S100 is obtained in real time from the high-weight performance index time series data information of four types of network elements in the current network, namely SMF, AMF, UPF and UDM.

[0013] As a preferred embodiment of the present invention, before step S100, the method further includes: collecting the original performance index file on the network element, obtaining index data by parsing the file, storing the information in the Elasticsearch cluster, and then periodically obtaining the collected basic information of the network element index through the Elasticsearch cluster.

[0014] As a preferred embodiment of the present invention, the basic information of the network element indicator includes node, network element name, data time, indicator name and actual value.

[0015] In a preferred embodiment of the present invention, in step S200, the anomaly assessment is to periodically predict the indicator data for a future period of time through a timed task, and automatically match the indicator anomaly assessment criteria to obtain the anomaly score and anomaly level information of the indicator for a future period of time.

[0016] In a preferred embodiment of the present invention, in step S300, the blacklist and dispatch conditions are set by combining the abnormal dispatch conditions and blacklist of network elements in different provinces.

[0017] In a preferred embodiment of the present invention, in step S400, when the abnormal network element indicator meets the dispatch verification conditions, the system will automatically perform an analysis of the cause of the abnormality of the network element and send it to the home province together with the work order information.

[0018] In a preferred embodiment of the present invention, in step S100, the training data is collected from the time series data of high-weight performance indicators from the Elasticsearch cluster 10 days ago. After data cleaning and preprocessing, the training data is input into the first model to obtain performance indicator data value information for a future period of time. At the same time, the training of the first model requires the arrangement and combination of all its parameters to form a grid, that is, the grid search method is used to find the optimal parameters for modeling the first model. The evaluation mechanism of the optimal parameters uses cross-validation.

[0019] In a preferred embodiment of the present invention, the first model is an XGBoost model; the mean square error (MSE) is used to evaluate the performance of the model.

[0020] A closed-loop management device for abnormal indicators based on 5GC, the device comprising:

[0021] Acquisition module: Used to acquire time series data of high-weight indicators for a future period of time;

[0022] The association module is used to associate the predicted time series data of indicators with the indicator anomaly threshold judgment criteria to obtain indicator-level anomaly information in the future period, including indicator anomaly scores, anomaly levels, etc.

[0023] Verification module: Used for periodic and automated verification of whether the indicator prediction information meets the user-set order dispatch criteria. If it meets the criteria, an order is dispatched; otherwise, no order is dispatched. The blacklist setting can meet special scenarios such as network element cutover, that is, in this scenario, no order dispatch verification is required even if the network element is abnormal.

[0024] The work order dispatch module is used to notify the province where the abnormal network element is located in advance, monitor and investigate the abnormal situation of the network element in a timely manner based on one or more abnormal indicators under the work order, and automatically analyze the results of the abnormality of the indicator to improve the efficiency of manual investigation.

[0025] Monitoring module: Used for real-time monitoring of abnormal network elements after processing, and determines whether the abnormality has been cleared by obtaining the recent values ​​of one or more indicators of the abnormal network element;

[0026] Closed-loop module: Used to close the loop for network elements that have recovered; otherwise, it performs an anomaly escalation operation and continues to follow up on the anomaly handling.

[0027] Compared with the prior art, the beneficial effects of the present invention are:

[0028] 1. By combining machine learning algorithms to achieve accurate prediction of time-series data of indicators, and by associating anomaly judgment criteria to obtain anomaly information of predicted indicator data, the system achieves closed-loop management of the entire process through personalized verification settings and full-process visual monitoring of abnormal network elements, enabling system maintenance personnel to know the abnormal indicators of network elements in advance and avoid losses due to untimely handling.

[0029] 2. To meet the needs of multiple scenarios, a blacklist setting has been added to avoid abnormal network element indicators at certain times due to special scenarios such as network element cutover, which may lead to incorrect order dispatch.

[0030] 3. Users in each province can set up anomaly detection settings according to their own needs, and the system will perform periodic task scheduling to realize automatic verification and order dispatch functions. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0032] Figure 1 This is a flowchart of a closed-loop management method for abnormal indicators based on 5GC according to the present invention;

[0033] Figure 2 This is a structural diagram of a closed-loop management device for abnormal indicators based on 5GC according to the present invention. Detailed Implementation

[0034] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0035] Please see Figure 1-2 This invention provides a method and apparatus for closed-loop management of indicator anomalies based on 5GC, the method comprising the following steps:

[0036] Step S100: Obtain time series data of high-weight indicators, filter the data according to the specific business rules of each indicator, and combine grid search and XGBoost to obtain data for future time periods and the maximum and minimum values ​​of that time period.

[0037] Step S200: Obtain indicator-level anomaly time series information by associating indicator data with future time periods according to the anomaly judgment rules, including indicator anomaly level and anomaly score content;

[0038] Step S300: Perform automated comparison and verification based on the blacklist and dispatch conditions set by the user and the time-series information of indicator-level anomalies over a future period;

[0039] Step S400: If the verification passes, the system will automatically analyze the cause of the network element indicator anomaly and send it along with the work order information to the province where the network element belongs, thereby improving the efficiency of manual monitoring and verification.

[0040] Step S500: Monitor network elements that may be abnormal in real time within the province, quickly resolve network element abnormalities based on the results of the abnormality cause analysis, and follow up on the handling.

[0041] Step S600: Monitor the indicator data of the abnormal network element over a recent period of time to determine whether the abnormality has been cleared. If it has returned to normal, the case can be closed; otherwise, the abnormality is upgraded and follow-up processing continues.

[0042] Furthermore, in step S100, the high-weight indicator time-series data is obtained in real time from the high-weight performance indicator time-series data information of the four types of network elements in the current network. The four types of network element information are SMF (Session Management function), AMF (Access and Mobility Management Function), UPF (The User plane function) and UDM (The Unified Data Management function).

[0043] Furthermore, prior to step S100 of this method, the method further includes: collecting the original performance indicator file on the network element, obtaining indicator data by parsing the file, storing the information in the Elasticsearch cluster, and then periodically obtaining the collected basic information of the network element indicators through the Elasticsearch cluster.

[0044] Furthermore, the basic information of the indicator includes the node, network element name, data time, indicator name, and actual value.

[0045] Furthermore, in step S200, the anomaly assessment is performed by periodically predicting indicator data for a future period through a timed task, and automatically matching the indicator anomaly assessment criteria to obtain the anomaly score and anomaly level information of the indicator for a future period.

[0046] Furthermore, in step S300, the blacklist and dispatch conditions are set in combination with the abnormal dispatch conditions and blacklist of network elements in different provinces.

[0047] Furthermore, in step S400, when the abnormal network element indicator meets the dispatch verification conditions, the system will automatically perform an analysis of the cause of the abnormality of the network element and send it to the home province along with the work order information.

[0048] Furthermore, the future time period data needs to be obtained by combining training data and the first model, and then associated with anomaly judgment rules to obtain indicator-level anomaly time series information. The first model is the XGBoost model. The training data is collected from the 10-day high-weight performance indicator time series data of the Elasticsearch cluster. After data cleaning and preprocessing, the training data is input into the XGBoost model to obtain performance indicator data value information for a future period. At the same time, the training of the XGBoost model requires the permutation and combination of all its parameters to form a grid, that is, to use the grid search method to find the optimal parameters for XGBoost modeling. The evaluation mechanism of the optimal parameters uses the cross-validation method.

[0049] Furthermore, the first model of this method uses mean squared error (MSE) to evaluate its performance.

[0050] Please see Figure 1 In the specific implementation, the high-weight performance index time-series data information of the four types of network elements (SMF, AMF, UPF, UDM) in the existing network is first obtained in real time. Before step S100 in this embodiment, the original performance index file is collected on the network element, the index data is obtained by parsing the file, and the information is stored in the Elasticsearch cluster. Then, the basic information of the collected network element index is obtained through the Elasticsearch cluster at regular intervals. The basic information of the index includes node, network element name, data time, index name, actual value, etc.

[0051] Using the time-series data of the aforementioned indicators as the data source for the algorithm model, the system periodically predicts indicator data for a future period through scheduled tasks, and automatically matches the indicator anomaly judgment criteria to obtain information such as the anomaly score and anomaly level of the indicators for the future period. Combined with different provinces, the system can set local network element anomaly dispatch conditions and blacklists, avoiding network element anomaly alarms caused by special circumstances such as network element cutover. When an abnormal network element indicator meets the dispatch verification conditions, the system automatically analyzes the cause of the anomaly and sends it along with the work order information to the home province. Provincial maintenance personnel then focus on real-time monitoring of potentially abnormal network element indicators. Combined with the results of the indicator anomaly cause analysis, the system can quickly locate the fault and improve maintenance efficiency. Afterwards, by monitoring whether the values ​​of the relevant indicators of the abnormal network element tend to return to normal over a recent period, it can be determined whether the fault has been cleared. If cleared, the work order can be closed; otherwise, the anomaly is escalated and further follow-up processing is initiated.

[0052] The training data was collected from the Elasticsearch cluster using high-weight performance index time-series data from 10 days prior. After data cleaning and preprocessing, the training data was input into the XGBoost model to obtain performance index values ​​for a future period. The training of the XGBoost model required permutations and combinations of all its parameters to form a grid, i.e., using a grid search method to find the optimal parameters for XGBoost modeling. The evaluation mechanism for the optimal parameters used cross-validation. The first model's performance was evaluated using mean squared error (MSE).

[0053] MSE is calculated by dividing the sum of squared errors by the sample size. The sum of squared errors is the sum of the squared errors between the predicted and actual values ​​during the fitting process of a linear regression model. A smaller MSE value indicates a closer match between the predicted and actual values, and a better fit. The formula for calculating MSE is:

[0054]

[0055] Where n is the number of samples; y i The true value of the sample; is the predicted value for the sample; i is the sample number.

[0056] After the above training and testing, we obtained the optimal parameter combination as follows: learning rate of 0.2, n_esti-mators of 90, max_depth of 10, min_child_weight of 1, and CV of 3 based on practical experience.

[0057] The optimal model is obtained through the above training. Combined with user-defined dispatch settings and minute-level high-performance indicator anomaly prediction information, intelligent automatic verification is achieved. Firstly, dispatch settings are divided into provincial users and national users. Provincial users can only set network elements within their province, while national users can set network elements across all provinces. Regarding priority, if both provincial and national users set the same network element, the provincial user setting condition takes precedence.

[0058] Secondly, regarding order verification, it is divided into several aspects: in terms of time, it is divided into periodic and non-periodic, and only data within the verification period is subject to order verification; in terms of the continuity of anomalies, it is divided into continuous and non-continuous, and order verification is performed based on the setting of conditions and whether the actual data is continuously abnormal; in terms of the number of times, the standard for the number of anomalies can be manually entered. When the number of anomalies in the data reaches the corresponding standard within the specified time, it satisfies our order dispatch logic.

[0059] For data that has already been assigned, even if the assignment criteria are met again in the subsequent time, no new assignment will be made to prevent the same network element from repeatedly assigning assignments in a short period of time. The data will only be re-verified after the current work order is completed and closed.

[0060] Example 2

[0061] Please see Figure 2 A closed-loop management device for abnormal indicators based on 5GC, the device comprising:

[0062] Acquisition Module: Used to acquire time-series data of high-weight indicators for a future period. It reads historical high-weight indicator data from Elasticsearch and performs operations such as missing value handling, time-series denoising, and outlier detection. It then filters the data by combining various network elements and specific business rules for each indicator. Finally, it uses grid search and XGBoost to obtain data for future periods, along with the maximum and minimum values ​​for that period.

[0063] The association module is used to associate the predicted time series data of indicators with the indicator anomaly threshold judgment criteria to obtain indicator-level anomaly information in the future period, including indicator anomaly scores, anomaly levels, etc.

[0064] Verification module: Used for periodic and automated verification of whether the predicted indicator information meets the user-defined dispatch criteria. If it does, a dispatch is made; otherwise, no dispatch is made. The blacklist setting can meet special scenarios such as network element cutover, where no dispatch verification is required even if a network element malfunctions.

[0065] Work order dispatch module: Used to notify the province where the abnormal network element is located in advance, and to monitor and investigate the abnormal situation of the network element in a timely manner based on one or more abnormal indicators under the work order. Combined with the automatic analysis results of the cause of the abnormality of the indicator, it improves the efficiency of manual investigation.

[0066] Monitoring module: Used for real-time monitoring of abnormal network elements after processing. It determines whether the abnormality has been completely cleared by obtaining the recent values ​​of one or more indicators of the abnormal network element.

[0067] Closed-loop module: Used to close the loop for network elements that have recovered; otherwise, it performs an anomaly escalation operation and continues to follow up on the anomaly handling.

[0068] For example, the processor fetches instructions one by one from the memory, analyzes the instructions, and then performs the corresponding operations according to the instructions, generating a series of control commands to enable the various parts of the computer to act automatically, continuously and in coordination, forming an organic whole, realizing program input, data input, and calculation and output of results. The arithmetic or logical operations generated in this process are all performed by the arithmetic unit; the memory includes a read-only memory (ROM), which is used to store computer programs, and the memory is provided with external protection devices.

[0069] For example, a computer program can be divided into one or more modules, one or more of which are stored in memory and executed by a processor to perform the present invention. The one or more modules can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a terminal device.

[0070] Those skilled in the art will understand that the above description of the service equipment is merely an example and does not constitute a limitation on the terminal equipment. It may include more or fewer components than described above, or a combination of certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0071] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the terminal device, connecting various parts of the user terminal via various interfaces and lines.

[0072] The aforementioned memory can be used to store computer programs and / or modules. The aforementioned processor implements various functions of the aforementioned terminal device by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating device, the application program required for at least one function (such as information collection template display function, product information publishing function, etc.), etc.; the data storage area may store data created based on the use of the berth status display device (such as product information collection templates corresponding to different product types, product information that different product providers need to publish, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0073] If the modules / units integrated into the terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the modules / units in the above-described embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the functions of the various device embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0074] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0075] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A closed-loop management method for abnormal indicators based on 5GC, characterized in that, The method includes the following steps: Step S100: Obtain time series data of high-weight indicators, filter the data according to the specific business rules of each indicator to obtain training data, and input the training data into the trained first model to obtain future time period data and the maximum and minimum values ​​of the future time period data; Step S200: Associate the future time period data with the anomaly judgment rules to obtain indicator-level anomaly time series information, wherein the indicator-level anomaly time series information includes indicator anomaly level and anomaly score content; Step S300: Perform automated comparison and verification based on the blacklist and dispatch conditions set by the user and the time-series information of indicator-level anomalies over a future period. Step S400: If the verification passes, the system will automatically analyze the reasons for the abnormal network element indicators and send them along with the work order information to the province where the network element belongs, thereby improving the efficiency of manual monitoring and verification. Step S500: Monitor network elements that may be abnormal in real time within the province, and quickly resolve network element abnormalities based on the analysis results of the abnormal network element indicators, and follow up on the handling. Step S600: Monitor the indicator data of the abnormal network element over a recent period of time to determine whether the abnormality has been cleared. If it has returned to normal, the case can be closed. Otherwise, if the upgrade fails, continue to follow up and process; the high-weight index time series data in step S100 is the real-time acquisition of the high-weight performance index time series data information of the four types of network elements in the current network, namely SMF, AMF, UPF and UDM. Before step S100, the method further includes: collecting raw performance indicator files on the network element, obtaining indicator data by parsing the files, storing the information in the Elasticsearch cluster, and periodically obtaining the collected basic information of the network element indicators through the Elasticsearch cluster; the basic information of the network element indicators includes the node, network element name, data time, indicator name, and actual value; in step S200, the anomaly assessment is performed by periodically predicting the indicator data for a future period through a scheduled task, and automatically matching the indicator anomaly assessment criteria to obtain the anomaly score and anomaly level information of the indicator for a future period.

2. The method according to claim 1, characterized in that, In step S300, the blacklist and dispatch conditions are set by combining the abnormal dispatch conditions and blacklist of network elements in different provinces.

3. The method according to claim 2, characterized in that, In step S400, when the abnormal network element indicator meets the dispatch verification conditions, the system will automatically analyze the cause of the abnormality of the network element and send it to the home province along with the work order information.

4. The method according to claim 1, characterized in that: In step S100, the training data is collected from the time series data of high-weight performance indicators from the Elasticsearch cluster 10 days ago. After data cleaning and preprocessing, the training data is input into the first model to obtain performance indicator data value information for a future period. At the same time, the training of the first model requires the permutation and combination of all its parameters to form a grid, that is, the grid search method is used to find the optimal parameters for modeling the first model. The evaluation mechanism of the optimal parameters uses cross-validation.

5. The method according to claim 4, characterized in that: The first model is the XGBoost model; The first model uses mean squared error (MSE) to evaluate its performance.

6. A closed-loop management device for abnormal indicators based on 5GC, characterized in that, The apparatus is used to implement the method according to any one of claims 1 to 5, the apparatus comprising: Acquisition module: Used to acquire time series data of high-weight indicators for a future period of time; The association module is used to associate the predicted time series data of indicators with the indicator anomaly threshold judgment criteria to obtain indicator-level anomaly information in the future period, including indicator anomaly score and anomaly level content. Verification module: Used for periodic and automated verification of whether the predicted indicator information meets the user-defined dispatch criteria. If it does, a dispatch is made; otherwise, no dispatch is made. The blacklist setting can meet special scenarios, where no dispatch verification is required even if the network element is abnormal. The work order dispatch module is used to notify the province where the abnormal network element is located in advance, monitor and investigate the abnormal situation of the network element in a timely manner based on one or more abnormal indicators under the work order, and automatically analyze the results of the abnormality of the indicator to improve the efficiency of manual investigation. Monitoring module: Used for real-time monitoring of abnormal network elements after processing, and determines whether the abnormality has been cleared by obtaining the recent values ​​of one or more indicators of the abnormal network element; Closed-loop module: Used to close the loop for network elements that have recovered; otherwise, it performs an anomaly escalation operation and continues to follow up on the anomaly handling.

Images