Determination device, determination method, and determination program

Abstract

A determination device 1 comprises a recovery determination / transition determination unit 14 that, with regard to metrics related to a decomposed traffic volume in which a traffic volume is decomposed by a prescribed classification axis, when the number of co-occurring abnormal metrics in which an abnormality is observed in an increase in the traffic volume reaches a quality deterioration occurrence reference value, extracts only a degree of abnormality in the traffic volume of the co-occurring abnormal metrics, and transitions to a recovery determination.

Description

Determination device, determination method, and determination program

[0001] The present disclosure relates to a determination device, a determination method, and a determination program.

[0002] In some cases, degradation in service quality cannot be detected by alarms or device performance information. Therefore, there is a technology that focuses on the time-series co-occurrence of multiple indicators (Non-Patent Document 1).

[0003] In Non-Patent Document 1, traffic volume is decomposed into multiple traffic volumes by port number etc., and the decomposed traffic volumes are called metrics. All metrics related to control signals are set as metrics to be judged as co-occurrences, and for all of these metrics, the period during which the number of anomalous metrics (co-occurrence count) showing abnormal increases in traffic volume is equal to or exceeds the quality degradation occurrence reference value is set as a quality degradation period, and the time when the quality degradation occurred and the time when recovery was completed are detected from this quality degradation period.

[0004] Hayashi, et al., "Proposal of a Service Quality Degradation Detection Technology Focusing on Time-Series Co-occurrence of Multiple Indicators," Institute of Electronics, Information and Communication Engineers, 2023 General Conference, B-14-10, March 8, 2023

[0005] As the total number of users affected by quality degradation gradually decreases due to the gradual resolution of congestion, the traffic volume of each abnormal metric gradually decreases toward the time of recovery completion. Therefore, near the time of recovery completion, it becomes difficult to distinguish between the steady-state metrics where no abnormal increase in traffic volume was observed.

[0006] However, in the past, the recovery completion time was detected based on all metrics, including both abnormal metrics and steady-state metrics. This resulted in problems such as the steady-state metrics affecting the number of co-occurrences near the recovery completion time, making it impossible to correctly detect the recovery completion time.

[0007] The present disclosure has been made in consideration of the above circumstances, and an object of the present disclosure is to provide a technology that can improve the accuracy of determining the recovery completion time in a communication service.

[0008] A determination device according to one aspect of the present disclosure is a determination device that determines recovery from quality degradation of a communication service, and includes a recovery determination transition determination unit that, when the number of co-occurring abnormal metrics showing an abnormality in the increase in traffic volume reaches a quality degradation occurrence reference value for metrics related to traffic volume after decomposition of the traffic volume along a predetermined classification axis, extracts only the degree of abnormality in the traffic volume of the co-occurring abnormal metrics and transitions to a recovery determination.

[0009] In one aspect of the present disclosure, a determination method for determining recovery from quality degradation of a communication service is provided, in which a determination device extracts only the degree of abnormality in the traffic volume of the co-occurring abnormal metrics, which show an abnormality in the increase in traffic volume, for metrics related to the traffic volume after decomposition of the traffic volume along a predetermined classification axis, when the number of co-occurring abnormal metrics, which show an abnormality in the increase in traffic volume, reaches a quality degradation occurrence reference value, and proceeds to a recovery determination.

[0010] A determination program according to an aspect of the present disclosure causes a computer to function as the determination device.

[0011] According to the present disclosure, it is possible to provide a technology that can improve the accuracy of determining the recovery completion time in a communication service.

[0012] Fig. 1 is a diagram showing the functional block configuration of a determination device. Fig. 2 is a diagram showing the processing flow of the determination device. Fig. 3 is a reference diagram when explaining the processing flow. Fig. 4 is a reference diagram when explaining the processing flow. Fig. 5 is a diagram showing the difference between the prior art and this embodiment. Fig. 6 is a diagram showing the hardware configuration of the determination device.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0014] [Summary of the Present Disclosure] To solve the above problem, in the present disclosure, instead of all metrics used in the co-occurrence determination (all metrics including both anomalous metrics and steady-state metrics), only co-occurring anomalous metrics are extracted and then the process moves to recovery determination. Furthermore, in the recovery determination, the time when a certain percentage of the number of co-occurring anomalous metrics falls below the recovery completion threshold for the degree of anomaly in traffic volume is determined to be the recovery completion time.

[0015] In this way, only co-occurring abnormal metrics are extracted, and the recovery completion time is determined as the time when a certain percentage of the number of co-occurring abnormal metrics falls below the recovery completion threshold for the abnormality level of traffic volume.This eliminates the influence of steady-state metrics that are not related to quality degradation, and makes it possible to correctly determine the recovery completion time even if the abnormality level of traffic volume for the abnormal metrics decreases gradually.

[0016] In addition to the above-mentioned problems, conventionally, the recovery completion time was determined using the same threshold as the quality degradation occurrence reference value, so even metrics indicating quality degradation may fall below the threshold before the recovery from the quality degradation is completed. This causes problems such as the determined quality degradation period being shorter than the actual quality degradation period, making it impossible to correctly detect the recovery completion time.

[0017] To solve this problem, in this disclosure, recovery is determined using a threshold different from the quality degradation occurrence reference value. That is, recovery is determined using a threshold for the traffic volume of the abnormal metrics (the above-mentioned recovery completion threshold) rather than a threshold for the number of abnormal metrics (the quality degradation occurrence reference value). The recovery completion threshold is, for example, a value calculated based on the traffic volume for a predetermined period of time in the past, starting from the time when the quality degradation occurred and the number of abnormal metrics reached the quality degradation occurrence reference value.

[0018] In this way, the recovery completion time is determined using the recovery completion threshold for the degree of abnormality of the traffic volume of the abnormal metrics, so that the recovery completion time can be determined more accurately even if the abnormal metrics decrease slowly.

[0019] [Configuration of Determination Device] FIG. 1 is a diagram showing the functional block configuration of a determination device 1 according to this embodiment.

[0020] The determination device 1 is an information processing device that determines recovery from a deterioration in the quality of a communication service. The determination device 1 includes a metrics extraction unit 11, a control signal metrics extraction unit 12, an abnormal metrics number calculation unit 13, a recovery determination transition determination unit 14, and a recovery determination unit 15.

[0021] The metrics extraction unit 11 has the function of acquiring flow data from the flow information DB, decomposing the time-series data of traffic volume contained in the acquired flow data into multiple traffic volumes using a predetermined classification axis, and extracting the decomposed traffic data (time-series data of traffic volume) as metrics.

[0022] Traffic can be identified from the name of the metrics. The metrics may be anything related to traffic volume. The units of traffic volume are, for example, bps (bits per second), pps (packets per second), or the number of flows per hour. The predetermined classification axis is, for example, port number or protocol number. Port numbers, etc. may be further classified into upstream and downstream directions.

[0023] The control signal metrics extraction unit 12 has a function of extracting metrics of control signals from the metrics extracted by the metrics extraction unit 11. The control signals are, for example, Session Initiation Protocol (SIP) and Internet Control Message Protocol (ICMP).

[0024] The abnormal metrics number calculation unit 13 has the function of treating all the control signal metrics extracted by the control signal metrics extraction unit 12 as metrics to be judged as co-occurrences, and calculating the degree of abnormality of the current traffic volume for each of those metrics from the difference or ratio between the current traffic volume and the past traffic volume, and calculating the number of abnormal metrics whose degree of abnormality has reached a predetermined abnormal value (the number of co-occurring abnormal metrics where an abnormality is observed in the increase in traffic volume).

[0025] In addition, the abnormal metrics number calculation unit 13 may use not only the metrics of the control signals extracted by the control signal metrics extraction unit 12, but also all the metrics extracted by the metrics extraction unit 11 and metrics other than the control signals.

[0026] The recovery judgment transition determination unit 14 has the function of determining whether the number of co-occurring abnormal metrics has reached a predetermined quality degradation occurrence reference value, and when the number of co-occurring abnormal metrics has reached the predetermined quality degradation occurrence reference value, extracting only the degree of abnormality of the traffic volume of the co-occurring abnormal metrics, setting it as the recovery judgment target metric, and transitioning to recovery judgment.

[0027] The recovery determination unit 15 has the function of determining whether the degree of abnormality of the traffic volume of abnormal metrics corresponding to a predetermined percentage or a predetermined rate of the number of extracted abnormal metrics falls below a predetermined recovery completion threshold, and determining the time when the degree of abnormality of the traffic volume of abnormal metrics corresponding to a predetermined percentage or a predetermined rate of the number of extracted abnormal metrics falls below the predetermined recovery completion threshold as the recovery completion time.

[0028] That is, the recovery determination unit 15 determines as the recovery completion time the time when a certain percentage of the number of abnormal metrics extracted by the recovery determination transition determination unit 14 falls below the recovery completion threshold for the degree of abnormality in traffic volume. For example, the recovery completion threshold is a value calculated using the degree of abnormality in traffic volume for a predetermined time period in the past from the time when the quality degradation occurred when the number of abnormal metrics reached the quality degradation occurrence reference value.

[0029] In addition to the above functions, the recovery determination unit 15 has the function of determining whether the number of extracted abnormal metrics has become less than the number of abnormal metrics immediately before the time when a predetermined quality degradation occurrence reference value has been reached, and determining that the time when the number of extracted abnormal metrics has become less than the number of abnormal metrics immediately before the time when a quality degradation occurred when the predetermined quality degradation occurrence reference value has been reached is the recovery completion time.

[0030] The above two functions of the recovery determination unit 15 can be selected arbitrarily by the user.

[0031] [Operation of Determination Device] FIG. 2 is a diagram showing a processing flow of the determination device 1 according to this embodiment.

[0032] Step S1: The metrics extractor 11 extracts a plurality of metrics from the flow information DB. Each metric includes the current traffic volume and the past traffic volume (for example, the traffic volume at the same time last week) (see FIG. 3).

[0033] Step S2: The control signal metrics extractor 12 extracts a plurality of metrics related to control signals from the plurality of metrics extracted in step S1.

[0034] Step S3: The abnormal metrics number calculation unit 13 sets all metrics extracted in step S2 as metrics to be subjected to co-occurrence judgment, calculates the degree of abnormality of the current traffic volume for each of those metrics from the difference or ratio between the current traffic volume and the past traffic volume, and calculates the number of abnormal metrics whose degree of abnormality has reached an abnormal value at each time (see the upper left of Figure 4).

[0035] 4, Metrics 1 and 2 are abnormal metrics because an abnormality is observed in the increase in traffic volume. Metric 3 is a steady-state metric because no abnormality is observed in the increase in traffic volume.

[0036] Step S4: The recovery determination transition determination unit 14 determines whether the number of abnormal metrics is equal to or greater than a predetermined quality degradation occurrence reference value. If the number of abnormal metrics is equal to or greater than the quality degradation occurrence reference value, the process proceeds to step S5. If the number of abnormal metrics is not equal to or greater than the quality degradation occurrence reference value, the process returns to step S3 and the number of abnormal metrics is calculated.

[0037] Step S5: If the number of abnormal metrics becomes equal to or greater than the quality degradation occurrence reference value, the recovery determination transition determination unit 14 determines the time when the number of abnormal metrics became equal to or greater than the quality degradation occurrence reference value as the time when the quality degradation occurred, extracts only the degree of abnormality in the traffic volume of the abnormal metrics, sets it as the metric to be subjected to recovery determination, and transitions to recovery determination (see the bottom left of Figure 4).

[0038] Step S6: The recovery determination unit 15 calculates the time after the occurrence of the quality degradation at which each abnormal metric extracted in step S5 falls below the recovery completion threshold, and sets this time as the recovery determination time for each metric (see the upper right corner of Figure 4).

[0039] The recovery completion threshold may be, for example, the average value of the degree of abnormality of the traffic volume for the past 24 hours starting from the time when the quality degradation occurred.

[0040] Step S7: The recovery determination unit 15 determines the recovery completion time as the time (recovery determination time) when a certain percentage (e.g., 80%, 90%, 100%, etc.) of the number of abnormal metrics extracted in step S5 falls below the recovery completion threshold for the degree of abnormality in traffic volume (see the bottom right of Figure 4).

[0041] (Variation 1) Instead of steps S6 and S7, the recovery determination unit 15 may determine as the recovery completion time the time when the number of abnormal metrics becomes less than the number of abnormal metrics immediately before the time when the quality degradation occurred due to the recovery of some of the abnormal metrics extracted in step S5.

[0042] (Modification 2) In steps S4 and S5, it is determined whether the number of abnormal metrics is equal to or greater than the quality degradation occurrence reference value. However, it may be determined whether the number of abnormal metrics exceeds the quality degradation occurrence reference value.

[0043] [Differences Between the Prior Art and the Present Embodiment] The differences between the prior art and the present embodiment will be explained again.

[0044] 5A and 5B are diagrams showing the difference between the conventional technology and this embodiment, where Fig. 5A shows the conventional technology and Fig. 5B shows this embodiment.

[0045] In the past, all metrics related to control signals (including both abnormal metrics and steady-state metrics) were used as metrics for co-occurrence determination, and the number of co-occurrences for all of these metrics was determined. The period during which the number of co-occurrences was equal to or greater than the quality degradation threshold was then determined as a quality degradation period, and the time when the number of co-occurrences fell below the quality degradation threshold was determined as the recovery completion time. In other words, in the past, the recovery completion time was determined directly from all metrics, including the steady-state metrics.

[0046] On the other hand, in this embodiment, the method for determining the number of co-occurrences is the same as in the conventional method, but when the number of co-occurrences exceeds the quality degradation occurrence reference value, only the co-occurring abnormal metrics are extracted as the recovery determination target metrics, rather than all metrics included in the group of metrics subject to co-occurrence determination, and the process moves to recovery determination. After that, the time when some percentage of the abnormal metrics fall below the recovery completion threshold for the degree of traffic volume abnormality and return to a steady value is determined to be the recovery completion time.

[0047] In other words, this embodiment does not use all metrics used in the co-occurrence judgment, but only uses the co-occurring abnormal metrics, and determines the recovery completion time as the time when a certain percentage of the co-occurring abnormal metrics falls below the recovery completion threshold for the degree of traffic volume anomaly and returns to a steady value. This makes it possible to eliminate the influence of steady-state metrics that are not related to quality degradation, and to correctly judge the recovery completion time even if the metrics decrease gradually.

[0048] [Effect] According to this embodiment, only the degree of anomaly in the traffic volume of co-occurring abnormal metrics is extracted and then recovery judgment is made. This makes it possible to eliminate the influence of steady-state metrics that are not related to quality degradation, and provides a technology that can improve the accuracy of determining the recovery completion time in communication services.

[0049] Furthermore, according to this embodiment, the recovery completion time is determined to be the time when a certain percentage of the extracted abnormal metrics fall below the recovery completion threshold for the abnormality level of traffic volume. Therefore, even if the time at which each metric returns to a steady state value differs, the recovery completion time can be determined uniformly, and a technology can be provided that can further improve the accuracy of determining the recovery completion time in communication services.

[0050] Furthermore, according to this embodiment, the recovery completion threshold is a value calculated using the traffic volume for a predetermined period of time in the past from when the number of abnormal metrics reached the quality degradation occurrence reference value, so it is possible to provide a technology that can further improve the accuracy of determining the recovery completion time in a communication service.

[0051] [Others] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0052] The determination device 1 of the present embodiment described above can be realized, for example, by using a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in Fig. 6. The memory 902 and the storage 903 are storage devices. In the computer system, the CPU 901 executes a predetermined program loaded on the memory 902, thereby realizing each function of the determination device 1.

[0053] The determination device 1 may be implemented by one computer, or by multiple computers, or may be a virtual machine implemented on a computer.

[0054] The program for the determination device 1 can be stored in a computer-readable recording medium such as a HDD, SSD, USB memory, CD, or DVD. The computer-readable recording medium is, for example, a non-transitory recording medium. The program for the determination device 1 can also be distributed via a communication network.

[0055] REFERENCE SIGNS LIST 1 Determination device 11 Metrics extraction unit 12 Control signal metrics extraction unit 13 Abnormal metrics number calculation unit 14 Recovery determination transition determination unit 15 Recovery determination unit 901 CPU 902 Memory 903 Storage 904 Communication device 905 Input device 906 Output device

Claims

1. A determination device for determining recovery from degradation in quality of a communication service, comprising: a recovery determination transition determination unit that extracts only the degree of abnormality in the traffic volume of co-occurring abnormal metrics and transitions to a recovery determination when the number of co-occurring abnormal metrics showing an abnormality in the increase in traffic volume reaches a quality degradation occurrence reference value, for metrics related to traffic volume after decomposition of the traffic volume along a predetermined classification axis.

2. The determination device according to claim 1, further comprising a recovery determination unit that determines the time when the degree of abnormality of the traffic volume of a predetermined percentage or a predetermined ratio of the number of the extracted abnormal metrics falls below a recovery completion threshold as the recovery completion time.

3. The determination device according to claim 2, wherein the recovery completion threshold is a value calculated using traffic volume for a predetermined period of time in the past from when the number of abnormal metrics reached the quality degradation occurrence reference value.

4. The determination device according to claim 1, further comprising a recovery determination unit that determines the time when the number of the extracted abnormal metrics becomes less than the number of abnormal metrics immediately before the quality degradation occurrence reference value as the recovery completion time.

5. A method for determining recovery from degradation in quality of a communication service, comprising: a determination device extracting only the degree of abnormality in the traffic volume of the co-occurring abnormal metrics, when the number of co-occurring abnormal metrics showing an abnormality in the increase in traffic volume reaches a quality degradation occurrence reference value, for metrics related to traffic volume after decomposition of the traffic volume according to a predetermined classification axis, and transitioning to a recovery determination.

6. A judgment program that causes a computer to function as the judgment device according to claim 1.

Images