Network data change detection device, method and program

Abstract

To detect a change of statistical information that becomes a momentum for considering again a fault detection AI model of a network by update or e-learning, and determine its cause.SOLUTION: A correct answer label creation function part 101 generates correct answer label data for each piece of network domain information. In a network information accumulation function part 103, statistical information on a network is accumulated. A data set creation function part 102 creates a data set to which a factor of a change of the statistical information is applied as a correct answer label on the basis of a date and time when the change occurs in the statistical information and a date and time when an event managed by the network domain information occurs. An AI model creation function part 104 creates a data change detection AI model by applying the data set to a machine learning algorithm with a teacher. A data change determination function part 105 detects the change of a data distribution on the basis of the data change detection AI model, and determines its cause.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a network data change detection device, method, and program, and more particularly to a network data change detection device, method, and program that can determine the cause of a data change based on network domain information. [Background technology]

[0002] Patent Document 1 discloses a technology for determining whether a change point is a change point based on the differences between data to be determined and comparison data extracted from consecutive data, and determining whether the data to be determined is a change point based on whether the number of data immediately preceding the data to be determined is greater than a predetermined threshold.

[0003] Non-Patent Document 1 discloses a framework for dealing with changes in data characteristics for fault prediction in edge clouds, which uses reinforcement learning to select the optimal adaptation method when changes in data characteristics occur, while taking into consideration the requirements of edge cloud operators, and also selects the optimal amount of data required during adaptation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-232 [Non-patent literature]

[0005] [Non-Patent Document 1] Paper "Automated Concept Drift Handling for Fault Prediction in Edge Clouds Using Reinforcement Learning", IEEE Transactions on Network and Service Management (Volume: 19, Issue: 2, June 2022), Behshid Shayesteh; Chunyan Fu; Amin Ebrahimzadeh; Roch H. Glitho [https: / / ieeexplore.ieee.org / document / 9718523] Summary of the Invention [Problem to be solved by the invention]

[0006] When using AI in disaster recovery operations, the data distribution in communication networks and the behavior of streaming data change daily, and so previously usable learning models can suddenly become unusable. Therefore, in order to maintain the inference accuracy of AI, there is a demand for the ability to review learning models at the appropriate time and in the appropriate way depending on the cause of the change.

[0007] For example, network statistics are used when selecting a method to deal with changes in data characteristics. In communication networks, data changes are induced by tasks and external events that are not caused by faults, and it is necessary to select the optimal method for reviewing the fault detection AI model depending on the cause of each data change.

[0008] However, neither Patent Document 1 nor Non-Patent Document 1 could identify the cause of the data change, and therefore the fault detection AI model could not be properly reviewed, resulting in unnecessary updates being made while necessary updates were not made to the fault detection AI model.

[0009] The object of the present invention is to solve the above technical problems and provide a network data change detection device, method, and program that can determine significant changes in data distribution and their causes by utilizing information specific to network fault operations, and optimize the timing and method of reviewing fault detection AI models. [Means for solving the problem]

[0010] In order to achieve the above object, the present invention provides a network data change detection device that detects changes in network data and determines the cause, and is equipped with means for acquiring network statistical information, means for acquiring network domain information specific to network fault operations, and means for determining, based on the statistical information and network domain information, changes in the statistical information that may serve as an opportunity to review the network fault detection AI model and the cause of those changes. [Effects of the Invention]

[0011] According to the present invention, by detecting changes in the distribution and behavior of network data and quickly updating the learning model according to the cause of the change, it becomes possible to maintain the inference accuracy of AI for automating network operation tasks. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 2 is a functional block diagram of a data change detection unit to which the present invention is applied. [Figure 2] FIG. 1 is a functional block diagram of a fault detection AI model management system to which the present invention is applied. [Figure 3] FIG. 2 is a functional block diagram showing the configuration of a main part of a data change detection unit. [Figure 4] FIG. 10 is a functional block diagram for explaining the operation in a training mode. [Figure 5] FIG. 2 is a functional block diagram for explaining an operation in an operation mode. [Figure 6] FIG. 1 shows an example of a data set (part 1). [Figure 7]FIG. 10 is a diagram showing an example of a dataset (part 2). [Figure 8] FIG. 10 is a diagram showing an example of a data set (part 3). [Figure 9] 1 is a sequence flow illustrating the operation of a fault detection AI model management system. DETAILED DESCRIPTION OF THE INVENTION

[0013] The following describes in detail an embodiment of the present invention with reference to the drawings. Figure 1 is a block diagram for explaining the function of a data change detection unit 1 to which the present invention is applied, and also explains the operation of a known data change handling unit 2 connected in the subsequent stage.

[0014] Based on network domain information and statistical information on network data, the data change detection unit 1 detects changes in the characteristics of statistical information that may trigger an update or re-learning of the AI ​​learning model that detects network faults (hereinafter sometimes referred to as the fault detection AI model), and then determines the cause and notifies the data change handling unit 2.

[0015] Here, network domain information is information specific to network fault operations, i.e., a general term for all information that operators refer to when carrying out their work in operating communication networks, and is information that manages events that may cause unusual changes in the statistical information of network data.

[0016] The network domain information includes planned work information, trouble ticket information, network performance data, calendar information of various events that may affect traffic characteristics, etc. The statistical information includes statistical information specific to the network or network node, such as traffic volume, throughput number, number of sessions, number of connected UEs (terminals), CPU utilization, etc.

[0017] When a significant characteristic change is detected in the statistical information and the cause of the change is determined, the data change handling unit 2 takes action such as updating the fault detection AI model depending on the nature and cause of the characteristic change. In this embodiment, depending on the magnitude of the characteristic change, the following is selected and executed: relearning the AI ​​model, switching to an existing AI model, updating the existing AI model, etc.

[0018] In this configuration, when the data change detection unit 1 detects a significant change in the characteristics of the statistical information, it refers to the network domain information. If, for example, it finds that a network connection configuration change was performed at a specific date and time in the planned work information and recognizes a causal relationship between the two, it determines that the network connection configuration change was the cause of the characteristic change. The details of the characteristic change and its cause are notified to the data change handling unit 2.

[0019] FIG. 2 is a functional block diagram showing the configuration of the main parts of a fault detection AI model management system to which the present invention is applied. In addition to the data change detection unit 1 and data change handling unit 2, the main components include a statistical information database (DB) 3 and a group of fault detection AIs 4.

[0020] Such a fault detection AI model management system can be configured by implementing applications (programs) that realize each function on at least one general-purpose computer or server equipped with a CPU, ROM, RAM, bus, interface, etc. Alternatively, it can be configured as a dedicated or single-function machine in which part of the application is implemented as hardware or software.

[0021] In this embodiment, the communication network is a group of facilities including communication devices such as routers and switches and transmission lines for providing mobile phone and Internet connection services, and the statistical information DB3 collects PM (Performance Management) data on the performance status of these devices, and converts and stores this data as KPI (Key Performance Indicator) information required for the operator's monitoring work.

[0022] The data change detection unit 1 collects KPI data from the statistical information DB 3, and collects network domain information from a dedicated system 5 used by an operator for monitoring work. In this embodiment, it collects CM (Configuration Management: configuration management data) data from a network configuration management system 501, collects planned work related information from a planned work management system 502, and collects FM (Fault Management: fault management data) data from a fault ticket management system 503.

[0023] Furthermore, when the data change detection unit 1 detects a significant change in the distribution of KPI data, it determines the cause by referring to the network domain information and notifies the data change handling unit 2 of the content and cause of the change in data distribution.

[0024] The data change handling unit 2 selects a review method for the corresponding fault detection AI model from the group of fault detection AI models based on the notified change in data distribution and its cause, and performs reviews such as updating and relearning depending on the selection result.

[0025] In this embodiment, first, (If) it is determined whether the data change has been experienced in the past. (Then) If it is an experienced data change, a measure is taken to switch the existing AI model corresponding to the most similar data distribution.

[0026] Else, if the data change is unprecedented but the maximum change is within the threshold, then measures are taken to update the AI ​​model using both the previous data and the new data, including data from before the change in data distribution occurred.

[0027] Else if the data has changed but has been addressed in the past, then no action is taken.

[0028] Otherwise (Else Then), a measure is taken to newly train (retrain) the AI ​​model using only new data since the data distribution changed.

[0029] FIG. 3 is a functional block diagram showing the configuration of the main parts of the data change detection unit 1, and its main components are a correct label creation function unit 101, a data set creation function unit 102, a network information accumulation function unit 103, an AI model creation function unit 104, and a data change determination function unit 105.

[0030] The data change detection unit 1 has a "training mode" in which the AI ​​model creation function unit 104 creates an AI learning model for data change detection (hereinafter referred to as a data change detection AI model) based on KPI data of NW statistical information and network domain information, as shown in Figure 4, and an "operation mode" in which the data change determination function unit 105 applies KPI data of NW statistical information to the data change detection AI model to determine significant data changes and their causes that could trigger a review of the fault detection AI model, as shown in Figure 5.

[0031] The correct label creation function unit 101 generates correct label data for each piece of network domain information. If the network domain information is planned work information, the correct label data is generated by collecting the implementation date and time of the planned work, the content, the target node of the work, and FM / CM data from a dedicated system.

[0032] The network information storage function unit 103 stores statistical information of the network and nodes, such as the number of sessions, the number of throughputs, the number of connected UEs, the CPU utilization rate, and the packet error rate, as distribution data of time and date.

[0033] The dataset creation function unit 102 refers to the correct label data based on the date and time when the KPI data distribution changed significantly, and identifies network domain information that is recognized to have a causal relationship with the change in data distribution. Then, it creates a dataset in which the cause of the change in data distribution is assigned as a correct label to the date and time when the change in distribution occurred in the KPI data, the KPI name, and the target node. Each item of KPI data is determined by the operator selecting in advance from the statistical information a portion necessary for monitoring, and calculating the average, median, amount of change (slope), etc.

[0034] Figure 6 is a diagram showing an example of a dataset created by the dataset creation function unit 102. Here, the number of sessions for communication equipment area ASMF01 remained at 270,000 units until 00:02:00 on December 8, 2022, but decreased to 140,000 units at 00:03:00 on the same day, and has continued to decrease since then, indicating that the data distribution of the number of sessions has changed significantly.

[0035] On the other hand, if the network configuration information for areaASMF01 has changed since around 3 minutes and 0 seconds ago, it can be determined that the change in the number of sessions in areaASMF01 is due to the change in the network configuration information. Therefore, the dataset creation function unit 102 assigns a "network configuration change" label as the correct answer label to data samples collected after 0:03 minutes and 0 seconds on December 8, and saves them as a dataset for machine learning of the data change detection AI model in the AI ​​model creation function unit 104.

[0036] FIG. 7 is a diagram showing another example of a dataset created by the dataset creation function unit 102, and here, attention is focused on the throughput number and the number of connected terminals (Active UE) of the communication equipment "areaASMF01" as statistical information.

[0037] Until 17:03:00 on December 8, 2022, the throughput and number of connected terminals remained stable at approximately 1.8 million and approximately 2.2 million, respectively, but at 17:04:00, these numbers rose to approximately 1.9 million and approximately 2.7 million, respectively, and continued to rise thereafter to approximately 3.8 million and approximately 3.3 million, respectively, indicating that the data distribution on the network has changed significantly.

[0038] On the other hand, since the network domain data information for areaASMF01 indicates that an event (fireworks display) was scheduled to begin at approximately 4 minutes and 0 seconds after the event, it can be determined that the significant changes in the number of sessions and the number of connected terminals in areaASMF01 are due to the holding of the event. Therefore, the dataset creation function unit 102 assigns the label "event (fireworks display)" as the correct answer label to the data samples collected after 17:04:00 on December 8th, and saves them as a dataset for machine learning of the data change detection AI model.

[0039] FIG. 8 is a diagram showing yet another example of a data set created by the data set creation function unit 102, and the explanation here focuses on the CPU utilization rate and packet error rate of the communication facility "areaASMF01."

[0040] Until 21:43:00 on December 8, 2022, the CPU utilization rate and packet error rate remained at 30% and 0%, respectively, but at 21:45:00 they rose to 80% and 10%, respectively, and have continued to rise since then, indicating that the data distribution of CPU utilization rate and packet error rate has changed significantly.

[0041] On the other hand, because a fault (abnormal CPU congestion) was detected from around 9:44:00 PM on December 8th in the network domain data information for areaASMF01, it can be determined that the significant changes in CPU usage and packet error rate in areaASMF01 are caused by the fault (abnormal CPU congestion). Therefore, the dataset creation function unit 102 assigns the label "fault (abnormal CPU congestion)" as the correct answer label to the data samples collected after 9:44:00 PM on December 8th, and saves them as a dataset for machine learning of the data change detection AI model.

[0042] Returning to Figure 3, the AI ​​model creation function unit 104 applies the dataset to a supervised machine learning algorithm to create a data change detection AI model for detecting whether a significant change has occurred in the data distribution of the KPI data and determining the cause of that change.

[0043] The data change determination function unit 105 detects whether a change in data distribution has occurred by applying newly collected KPI data in operation mode to the data change detection AI model generated in the training mode, and if a change in data is detected, it notifies the data change handling unit 2 of the change, including the cause.

[0044] Thereafter, the data change handling unit 2 selects a method for reviewing the fault detection AI model according to a pre-set conditional branch, depending on the content and cause of the change in data distribution notified by the data change detection unit 1, and takes appropriate measures such as updating depending on the selection result.

[0045] FIG. 9 is a sequence flow showing the operation of the fault detection AI model management system to which the present invention is applied in training mode.

[0046] At time t1, PM / CM data is collected from the network and stored in the statistical information DB 3. At time t2, the data change detection unit 1 collects FM data from the trouble ticket management system 503. At time t3, the data change detection unit 1 collects planned work related information from the planned work management system 502. At time t4, the data change detection unit 1 collects KPI data from the statistical information DB 3.

[0047] At time t5, the data change detection unit 1 determines whether or not there has been a change in data distribution and the cause thereof based on the collected data and domain information. At time t6, the data change handling unit 2 is notified of the whether or not there has been a change in data distribution and the cause thereof.

[0048] At time t7, a method for reviewing the fault detection AI model is selected in the data change handling unit 2. At time t8, based on the result of the selection, re-learning, switching, updating, or the like of the fault detection AI model is selectively performed.

[0049] Furthermore, according to the above embodiment, it becomes possible to maintain the inference accuracy of AI for automating network operation tasks, thereby making it possible to provide a variety of entertainment to many people regardless of geographic or economic disparities. As a result, it becomes possible to contribute to Goal 9 "Build resilient infrastructure and promote inclusive and sustainable industrialization" and Goal 11 "Make cities inclusive, safe, resilient and sustainable" of the United Nations-led Sustainable Development Goals (SDGs). [Explanation of symbols]

[0050] 1...Data change detection unit, 2...Data change handling unit, 3...Statistical information database (DB), 4...Fault detection AI group, 101...Correct label creation function unit, 102...Data set creation function unit, 103...Network information accumulation function unit, 104...AI model creation function unit, 105...Data change determination function unit, 501...Network configuration management system, 502...Planned work management system, 503...Fault ticket management system

Claims

1. A network data change detection device that detects changes in data related to network statistical information and determines the cause, a means for obtaining network statistics; means for acquiring network domain information specific to the network fault service; The network domain information is information specific to network failure operations, and is general information that operators refer to when performing operations in communication network operations, and is information that manages events that may cause unusual changes in the statistical information, A network data change detection device characterized by determining changes in statistical information and their causes that could trigger a review of the network fault detection AI model based on the statistical information and network domain information.

2. 2. The network data change detection device according to claim 1, wherein the network domain information includes at least one of planned work information, trouble ticket information, FM / CM data, and event calendar information in which traffic characteristics change.

3. It has a training mode and an operational mode, In the training mode, creating a data set that represents a causal relationship between the data change and the network domain information based on the time when the change occurred in the statistical information and the time when the event managed by the network domain information occurred; 3. A network data change detection device according to claim 1, wherein a data change detection AI model is created based on the data set to detect a data change that triggers a review of the fault detection AI model and determine the cause of the change.

4. In the operation mode, The network data change detection device described in claim 3, characterized in that statistical information is applied to the created data change detection AI model to detect data changes that may trigger a review of the fault detection AI model and determine the cause of the changes.

5. A method for detecting changes in network data, in which a computer detects changes in data related to network statistical information and determines the cause, comprising: Get network statistics, Obtain network domain information specific to network fault operations, Based on the statistical information and the network domain information, determine changes in the statistical information that will trigger a review of the network fault detection AI model and the causes thereof; A method for detecting changes in network data, characterized in that the network domain information is information specific to network failure operations, is general information that operators refer to when carrying out operations in communication network operations, and is information that manages events that may cause unusual changes to the statistical information.

6. A network data change detection program that detects changes in data related to network statistical information and determines the cause, Steps to get network statistics and Procedures for obtaining network domain information specific to network fault operations; and determining, based on the statistical information and the network domain information, a change in the statistical information that will trigger a review of the network fault detection AI model and the cause of the change; A network data change detection program characterized in that the network domain information is information specific to network failure operations, is general information that operators refer to when carrying out operations in communication network operations, and is information that manages events that may cause unusual changes to the statistical information.

Images