Method and device for augmenting alarm data

Abstract

A method for augmenting alarm data is disclosed. The method for augmenting alarm data according to the present invention comprises the steps of: collecting a plurality of alarm data generated from a plurality of network devices and composed of a plurality of items; classifying the plurality of items into a representative value and a variable value; generating augmented alarm data by replacing the representative value of the alarm data with another representative value within a replaceable cluster or by matching a newly generated variable value to the representative value of the alarm data; and generating a label for the augmented alarm data.

Description

Technology Field

[0001] The present invention relates to a method and apparatus for augmenting alarm data that is generated from network equipment and has a table format, for use in training an artificial intelligence model. Background Technology

[0002] For an artificial intelligence model to achieve sufficient performance, a large amount of training data and appropriate labeling are required for training the model. In an AI model designed to detect anomalies in network equipment, labels indicating a normal state will be matched to normal data (data collected during normal conditions), and labels indicating an anomaly will be matched to abnormal data (data collected during abnormal conditions).

[0003] However, anomalies occur less frequently than normal situations, and AI models struggle to achieve sufficient performance when anomaly data is scarce. Therefore, while it is necessary to collect a sufficient amount of anomaly data, gathering enough data takes a long time, and costs are incurred during the labeling process; consequently, the time and expenses involved increase proportionally to the amount of data. Furthermore, in special cases of anomalies, collecting anomaly data itself may be impossible. Thus, data augmentation is a technique that utilizes existing collected data to augment training data for use in training AI models.

[0004] Data augmentation techniques are frequently used because they can augment training data at a low cost. However, there are two problems with utilizing general data augmentation techniques to augment network data generated by network equipment.

[0005] First, general augmentation techniques applied to existing image or audio data cannot be used for network data. Specifically, network data consists of various numerical or categorical data or text such as system logs depending on the network environment, and is not provided in a consistent format like image or audio data. Therefore, it is difficult to apply pixel augmentation techniques generally used for image data, such as rotation, transformation, and translation, or sound augmentation techniques used for audio data, such as speed / volume control, noise addition, and inverse playback. Furthermore, as technology advances and network equipment is added or removed, the ability to respond to dynamically changing network environments is required; consequently, there is a problem in that general methodologies lacking domain knowledge are difficult to apply.

[0006] Secondly, it is difficult to generate appropriate anomalous data using existing network data augmentation techniques. Specifically, referring to Fig. 1a, existing network data augmentation techniques represent previously collected data in a vector space and generate new data using oversampling in the space between the data. That is, this technique is a method that creates new data similar to the characteristics of a cluster by utilizing numerical similarity under the assumption that the data forms a cluster (a specific distribution). However, due to the nature of generating new data based on the characteristics of the cluster after defining the categories and classifications of the data in advance, it is dependent on the characteristics of the previously collected data and has a problem in that it is difficult to generate new data with characteristics that are completely different from the previously collected data. In other words, to improve the performance of an artificial intelligence model, anomalous data must exhibit patterns different from the previously collected data or must not have a consistent pattern; however, there is a problem in that it is difficult to generate data that lacks similarity to the previously collected data using existing network data augmentation techniques, as shown in Fig. 1b. The problem to be solved

[0007] The present invention aims to solve the aforementioned problems, and the objective of the present invention is to provide a method and apparatus for augmenting alarm data that augments alarm data generated from network equipment and having a table format for use in training an artificial intelligence model. means of solving the problem

[0008] A method for augmenting alarm data according to the present invention comprises the steps of: collecting a plurality of alarm data generated from a plurality of network devices and composed of a plurality of items; classifying the plurality of items into a representative value and a variable value; generating augmented alarm data by replacing the representative value of the alarm data with another representative value within a replaceable cluster or by matching a newly generated variable value to the representative value of the alarm data; and generating a label for the augmented alarm data.

[0009] In this case, the step of classifying the plurality of items into representative values ​​and variable values ​​may include the step of setting an item among the plurality of items that can classify the plurality of alarm data into a plurality of clusters as the representative value, and the step of setting an item among the plurality of items that constitutes the content of the alarm as the variable value.

[0010] In this case, the above representative value may be a name assigned to the network equipment by the network operator, an identification code of the network equipment, or a type of network equipment.

[0011] Meanwhile, the above variable value may include at least one of the alarm occurrence time, alarm type, and status value.

[0012] Meanwhile, the step of generating augmented alarm data by replacing the representative value of the above alarm data with another representative value within a replaceable cluster may include the step of generating a plurality of clusters by grouping alarm data having the same representative value, the step of determining similarity between the plurality of clusters, and the step of setting two or more clusters that are similar to each other as replaceable clusters based on the similarity.

[0013] In this case, the step of determining similarity between the plurality of clusters includes the step of determining similarity between the plurality of clusters based on domain knowledge corresponding to the representative value, and the domain knowledge may include at least one of the customer type of the network equipment and the equipment type of the network equipment.

[0014] Meanwhile, the step of determining similarity between the plurality of clusters may include: a step of calculating a first data distribution corresponding to the first cluster using a variable value of alarm data belonging to the first cluster; a step of calculating a second data distribution corresponding to the second cluster using a variable value of alarm data belonging to the second cluster; and a step of determining whether the first cluster and the second cluster are similar by comparing the first data distribution and the second data distribution.

[0015] Meanwhile, the step of generating augmented alarm data by replacing the representative value of the above alarm data with another representative value within the substitutable cluster may further include the step of generating augmented alarm data by maintaining a variable value of the first alarm data belonging to the first cluster within the substitutable cluster and matching the representative value of the second alarm data belonging to the second cluster within the substitutable cluster with the maintained variable value.

[0016] In this case, the step of generating a label for the augmented alarm data may include the step of matching the label of the first alarm data to the generated augmented alarm data to generate a label corresponding to the generated augmented alarm data.

[0017] Meanwhile, the step of generating augmented alarm data by matching a newly generated variable value to a representative value of the above alarm data may include the step of newly generating a variable value based on domain knowledge of a cluster corresponding to the representative value, and the step of generating augmented alarm data by matching the generated variable value to the representative value.

[0018] In this case, the step of newly generating a variable value based on domain knowledge of a cluster corresponding to the representative value may include: a step of determining at least one of a generation range of the variable value and a type of alarm based on the variable value of alarm data within the cluster corresponding to the representative value; and a step of newly generating the variable value using at least one of the generation range and the type of alarm.

[0019] Meanwhile, the step of generating a label for the augmented alarm data may include: a step of matching a label corresponding to the pattern to the augmented alarm data when the augmented alarm data corresponds to a pattern identified from the alarm data within the cluster corresponding to the representative value; and a step of matching a different label to the augmented alarm data when the augmented alarm data does not have a pattern identified from the alarm data within the cluster corresponding to the representative value.

[0020] Meanwhile, the alarm data augmentation device according to the present invention includes a data collection unit that collects a plurality of alarm data generated from a plurality of network devices and composed of a plurality of items, and a control unit that classifies the plurality of items into a representative value and a variable value, generates augmented alarm data by replacing the representative value of the alarm data with another representative value within a replaceable cluster, or by matching a newly generated variable value to the representative value of the alarm data, and generates a label for the augmented alarm data.

[0021] In this case, the control unit may set the item among the plurality of items that can classify the plurality of alarm data into a plurality of clusters as the representative value, and set the item among the plurality of items that constitutes the content of the alarm as the variable value.

[0022] In this case, the above representative value may be a name assigned to the network equipment by the network operator, an identification code of the network equipment, or a type of network equipment.

[0023] Meanwhile, the above variable value may include at least one of the alarm occurrence time, alarm type, and status value.

[0024] Meanwhile, the control unit may group alarm data having the same representative value to create a plurality of clusters, determine the similarity between the plurality of clusters, and, based on the similarity, set two or more clusters that are similar to each other as interchangeable clusters.

[0025] In this case, the control unit determines the similarity between the plurality of clusters based on domain knowledge corresponding to the representative value, and the domain knowledge may include at least one of the customer type of the network equipment and the equipment type of the network equipment.

[0026] Meanwhile, the control unit can calculate a first data distribution corresponding to the first cluster using a variable value of alarm data belonging to the first cluster, calculate a second data distribution corresponding to the second cluster using a variable value of alarm data belonging to the second cluster, and determine whether the first cluster and the second cluster are similar by comparing the first data distribution and the second data distribution.

[0027] Meanwhile, the control unit can generate augmented alarm data by maintaining a variable value of first alarm data belonging to a first cluster within the substitutable cluster and matching a representative value of second alarm data belonging to a second cluster within the substitutable cluster with the maintained variable value. Effects of the invention

[0028] According to the present invention, there is an advantage in that the performance of an artificial intelligence model can be improved by augmenting training data using previously collected data, even for network data in a table format consisting of multiple items, to which general augmentation techniques are difficult to apply. Brief explanation of the drawing

[0029] Figure 1 is a diagram illustrating the problems of conventional data augmentation techniques. FIG. 2 is a block diagram illustrating an alarm data enhancement device according to the present invention. FIG. 3 is a flowchart illustrating a method for augmenting alarm data according to the present invention. FIG. 4 is a diagram illustrating alarm data according to the present invention. Figure 5 is a diagram illustrating clusters in which multiple alarm data are classified. FIG. 6 is a diagram illustrating clusters reconfigured according to similarity according to the present invention. FIG. 7 is a diagram illustrating a method for generating augmented alarm data by replacing a representative value of alarm data with another representative value according to the present invention. FIG. 8 is a diagram illustrating a method for generating a variable value and matching the generated variable value to a representative value according to the present invention. Figure 9 is a diagram illustrating a method for generating labels for augmented alarm data. Specific details for implementing the invention

[0030] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols will be assigned the same reference number, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not inherently possess distinct meanings or roles. Furthermore, in describing embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.

[0031] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0032] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0033] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0034] For convenience of explanation in implementing the present invention, the components may be described in detail; however, these components may be implemented within a single device or module, or a single component may be divided and implemented across multiple devices or modules.

[0035] FIG. 2 is a block diagram illustrating an alarm data enhancement device according to the present invention.

[0036] The alarm data enhancement device (100) (hereinafter referred to as the device (100)) may include a data collection unit (110), a control unit (120), and a memory (130).

[0037] The data collection unit (110) can collect network data. Here, network data may include alarm data generated from network equipment constituting the network.

[0038] Specifically, when the event conditions for generating an alarm are satisfied, the network equipment generates alarm data including the alarm occurrence time, alarm type, status value, etc., and can retain the generated alarm data or transmit it to the network operator's server. In addition, the data collection unit (110) can collect alarm data generated from multiple network equipment.

[0039] Network equipment may generate alarm data by determining that event conditions are satisfied in the event of a failure. However, it is not limited to this, and network equipment may also generate alarm data including information regarding the status of the network or network equipment even in situations where no failure has occurred.

[0040] The data collection unit (110) includes a communication module that provides a communication interface for communicating with other devices, and can collect multiple alarm data generated from multiple network devices from multiple network devices or an operator's server through the communication module.

[0041] The control unit (120) is composed of one or more processors and can control the overall operation of the device (100). Additionally, the control unit (120) can classify multiple items within the alarm data into representative values, dependent values, and variable values, and generate augmented alarm data by replacing the representative values ​​from the alarm data or matching newly generated variable values ​​to the representative values. Additionally, the control unit (120) can generate labels for the generated augmented alarm data.

[0042] The memory (130) can store a program or other instructions for the operation of the device (100). Additionally, the memory (130) can store collected alarm data and augmented alarm data generated based on the collected alarm data. Additionally, the memory (130) can store domain knowledge corresponding to a representative value.

[0043] Meanwhile, the device (100) may include an output unit (not shown) that outputs collected alarm data and generated augmented alarm data. Specifically, the output unit may include a display module and display alarm data and augmented alarm data. Additionally, the output unit may include a communication module and transmit alarm data and augmented alarm data to a training device for an artificial intelligence model or to an operator's server.

[0044] FIG. 3 is a flowchart illustrating a method for augmenting alarm data according to the present invention.

[0045] The method for augmenting alarm data according to the present invention may include the steps of: collecting alarm data generated from a plurality of network devices and composed of a plurality of items (S310); classifying the plurality of items into a representative value and a variable value (S320); generating augmented alarm data by replacing the representative value of the alarm data with another representative value within a replaceable cluster or by matching a newly generated variable value to the representative value of the alarm data (S330); and generating a label for the augmented alarm data (S340).

[0046] FIG. 4 is a diagram illustrating alarm data according to the present invention.

[0047] The data collection unit (110) can collect multiple alarm data generated from multiple network devices (S310). Specifically, the data collection unit (110) can collect multiple alarm data generated from multiple network devices from multiple network devices or from the server of the operating equipment. The alarm data generated from the network devices is data that actually occurred in the network devices, and such alarm data can be used for training an artificial intelligence model and can also be used for generating augmented alarm data.

[0048] Alarm data may be data in the form of a table composed of multiple items. Specifically, in the table of FIG. 4, one row represents one alarm data, and one alarm data may be composed of multiple items.

[0049] In FIG. 4, the device name, device code, installation location, time, and alarm type (Type 1 to Type N) of the network equipment are shown as examples of multiple items.

[0050] The device name of network equipment may be a name assigned to the network equipment by the network operator. For example, if a network operator operates or manages network equipment installed at Bank A, the network operator may assign the name “Bank A” to the network equipment installed at Bank A. Therefore, multiple alarm data generated from the same network equipment may contain the same device name.

[0051] A device code refers to an identification code uniquely assigned to individual network equipment. Therefore, even multiple network devices belonging to the same equipment type may be assigned different identification codes. Additionally, multiple alert data generated from the same network equipment may contain the same device code.

[0052] The installation location may refer to the location, place, or area where network equipment is installed.

[0053] Time may refer to the time at which an alarm occurs. That is, when a network device triggers an alarm because an event condition is satisfied, the network device may generate alarm data that includes the time at which the alarm occurred. The alarm occurrence time may include not only the hour, minute, and second at which the alarm occurred, but also additional information regarding the year, month, day, day of the week, weekday, weekend, etc.

[0054] The type of alarm (Type 1 to Type N) may be information that classifies and categorizes failures or conditions occurring in a network or network equipment. For example, the alarm data may include a first failure code (e.g., power cut) or a second failure code (e.g., inability to communicate).

[0055] Additionally, multiple items constituting the alarm data may further include status values. For example, the magnitude of a signal received from network equipment may be listed in the status value item of the alarm data. Here, the status value may include a level of status. For example, a signal magnitude of 0 to 10 dB may be expressed as Level 1, and a signal magnitude of 10 to 20 dB may be expressed as Level 2.

[0056] Additionally, multiple items constituting the alarm data may include the equipment type of the network equipment. For example, the alarm data may include first equipment type information (e.g., PTS, Packet Transport System), second equipment type information (MSPP, Multi Service Provisioning Platform), third equipment type information (ROADM, Re-configurable Optical Add-Drop Multiplexer), etc., to which the network equipment that triggered the alarm belongs.

[0057] Next, the control unit (120) can classify a plurality of items constituting alarm data into representative values ​​and variable values ​​(S320).

[0058] The control unit (120) can set an item among the multiple items constituting the alarm data that can classify multiple alarm data into multiple clusters as a representative value. Specifically, the representative value provides a criterion for dividing the collected multiple alarm data into multiple clusters, and can distinguish individual clusters without distinguishing individual alarm data. In addition, the representative value can represent a single cluster containing multiple alarm data.

[0059] The representative value may be a name assigned to the network equipment by the network operator, an identification code of the network equipment, or a type of network equipment.

[0060] Regarding the device name (a name assigned to network equipment by a network operator), multiple alarm data may include the same device name "A Bank," another multiple alarm data may include the same device name "Bank B," and yet another multiple alarm data may include the same device name "C Post Office." In this case, "A Bank," "Bank B," and "C Post Office," which belong to the device name (a name assigned to network equipment by a network operator) item, may be selected as representative values.

[0061] Regarding the identification code of network equipment, multiple alarm data may contain the same identification code “B0280,” another multiple alarm data may contain the same identification code “B0048,” and yet another multiple alarm data may contain the same identification code “B6482.” In this case, “B0280,” “B0048,” and “B6482,” which belong to the identification code item of the network equipment, may be selected as representative values.

[0062] Regarding the equipment type of network equipment, multiple alarm data (alarm data generated from a first network equipment and alarm data generated from a second network equipment) may include the same equipment type, such as a first equipment type (e.g., PTS), and another multiple alarm data may include the same equipment type, such as a second equipment type (e.g., MSPP). In this case, the first equipment type (e.g., PTS) and the second equipment type (e.g., MSPP) belonging to the equipment type item may be selected as representative values.

[0063] Additionally, the installation location of network equipment can also be selected as a representative value.

[0064] Meanwhile, among the multiple items constituting the alarm data, only one item may be selected as the representative value. For example, if the multiple items constituting a single alarm data include a name assigned to network equipment by a network operator, an identification code of the network equipment, or a type of network equipment, only one of these items may be selected as the representative value.

[0065] When a representative value is selected, the control unit (120) can select a variable value among a plurality of items constituting the alarm data. Here, a variable value may refer to an item that is not dependent on the representative value and can be changed to increase the data. Additionally, a variable value may be an item constituting the content of the alarm.

[0066] Specifically, the variable value may include at least one of the alarm occurrence time, alarm type, and status value. For example, the alarm occurrence time, alarm type, and status value are items unrelated to the representative value (network equipment name, identification code, equipment type, installation location). Therefore, the variable value can be changed regardless of what the representative value is within the alarm data.

[0067] Furthermore, while the alarm occurrence time, alarm type, and status value may serve as criteria for dividing individual alarm data, they cannot serve as criteria for dividing individual clusters. Therefore, variable values ​​may be defined as items that cannot serve as criteria for dividing individual clusters among multiple items. Additionally, while multiple alarm data generated from the same network equipment may contain the same representative value, multiple alarm data generated from the same network equipment may contain different variable values.

[0068] Additionally, when a representative value is selected, the control unit (120) can select a dependent value among a plurality of items constituting the alarm data. Here, a dependent value is an item dependent on the representative value, and may be an item determined according to the determined representative value once the representative value is determined. Specifically, when the name of a network device is selected as the representative value, the identification code, installation location, and equipment type of the network device may be determined depending on the network device because they are information assigned to the network device. For example, if Bank A is the representative value, the identification code, installation location, and equipment type of the network device installed at Bank A may be dependent values.

[0069] Next, the control unit (120) can generate augmented alarm data by replacing the representative value of the alarm data with another representative value within a replaceable cluster or by matching a newly generated variable value to the representative value of the alarm data (S330). First, a first embodiment of generating augmented alarm data by replacing the representative value of the alarm data with another representative value within a replaceable cluster will be explained with reference to FIGS. 5 to 7.

[0070] Figure 5 is a diagram illustrating clusters in which multiple alarm data are classified. In the following explanation, it is assumed that the device name (a name assigned to network equipment by the network operator) is selected as the representative value.

[0071] The control unit (120) can create multiple clusters by grouping alarm data having the same representative value. For example, referring to FIG. 5, a “Bank A cluster” is illustrated. The “Bank A cluster” is created by grouping multiple alarm data having a representative value of “Bank A”, and thus, the “Bank A cluster” corresponding to the representative value “Bank A” may include multiple alarm data having a representative value of “Bank A”. Additionally, the “Bank B cluster” may include multiple alarm data having a representative value of “Bank B”, and the “Post Office C cluster” may include multiple alarm data having a representative value of “Post Office C”.

[0072] Next, the control unit (120) can determine the similarity between multiple clusters. As a first method for determining the similarity between multiple clusters, the control unit (120) can determine the similarity between multiple clusters based on domain knowledge corresponding to a representative value.

[0073] Here, the domain knowledge corresponding to the representative value may include at least one of the customer type of the network equipment having the representative value and the equipment type of the network equipment. Specifically, the representative value and the customer type corresponding to the representative value may be stored in the database within the memory (130). For example, a customer type named "Bank" may be matched to a device name (a name assigned to the network equipment by the network operator) named "A Bank" and stored in the database. As another example, a customer type named "Bank" may be matched to a device name named "B Bank" and stored in the database.

[0074] In addition, a representative value and a corresponding equipment type may be stored in the database within the memory (130). For example, a network equipment type named “PTS” may be matched to a device name “A Bank” and stored in the database, a network equipment type named “PTS” may be matched to a device name “B Bank” and stored in the database, and a network equipment type named “MSPP” may be matched to a device name “C Post Office” and stored in the database.

[0075] And the control unit (120) can determine the similarity between multiple clusters based on domain knowledge corresponding to a representative value. For example, the control unit (120) can determine that the clusters being compared are similar to each other if at least one of the customer type of the network equipment and the equipment type of the network equipment matches.

[0076] Additionally, customer types may be extracted from representative values. For example, if the first cluster has a representative value of “Bank A” and the second cluster has a representative value of “Bank B”, the control unit (120) can extract a customer type of “Bank” from the representative value of the first cluster and extract a customer type of “Bank” from the representative value of the second cluster, and then determine that the first cluster and the second cluster are similar to each other.

[0077] As a second method for determining similarity between multiple clusters, the control unit (120) can determine similarity between clusters by comparing the distribution of variable values ​​between clusters. When it is impossible to determine similarity using the first method based on domain knowledge, the control unit (120) can determine similarity between clusters by comparing the distribution of variable values ​​between clusters.

[0078] Specifically, the control unit (120) can calculate the similarity between clusters using a mathematical algorithm that measures the similarity between two distributions, such as KL-Divergence. Specifically, the control unit (120) can calculate the first data distribution of alarm data belonging to the first cluster using the variable value of alarm data belonging to the first cluster, and calculate the second data distribution of alarm data belonging to the second cluster using the variable value of alarm data belonging to the second cluster.

[0079] Then, the control unit (120) can compare the first data distribution and the second data distribution to determine whether the first cluster and the second cluster are similar to each other. Specifically, the control unit (120) can calculate a similarity index of the first data distribution and the second data distribution, and if the similarity index of the first data distribution and the second data distribution is smaller than a threshold value, it can determine that the first cluster and the second cluster are similar to each other.

[0080] FIG. 6 is a diagram illustrating clusters reconfigured according to similarity according to the present invention.

[0081] The x-axis of the graph in Fig. 6 represents a day divided into fixed time units (e.g., 1-hour units), and the y-axis represents the frequency. That is, the length of the bar graph shown in Fig. 6 may represent the number (or ratio) of failures that occurred at that time. Additionally, the length of the bar graph shown in Fig. 6 may represent the number (or ratio) of alarm data collected at that time.

[0082] The control unit (120) can set two or more similar clusters as interchangeable clusters. Specifically, the control unit (120) can combine two or more similar clusters to create one interchangeable cluster. For example, if the first cluster (Bank A cluster in FIG. 5) and the second cluster (Bank B cluster in FIG. 5) are determined to be similar to each other, the first interchangeable cluster (Bank in FIG. 6) may include alarm data belonging to the first cluster and alarm data belonging to the second cluster. As another example, if the third cluster (Post Office C cluster in FIG. 5), the fourth cluster (Post Office D cluster in FIG. 5), and the fifth cluster (Post Office E cluster in FIG. 5) are determined to be similar to each other, the second interchangeable cluster (Post Office in FIG. 6) may include alarm data belonging to the fourth cluster, alarm data belonging to the fifth cluster, and alarm data belonging to the sixth cluster. If a specific cluster does not have similar clusters, a replaceable cluster containing only that specific cluster may be established.

[0083] FIG. 7 is a diagram illustrating a method for generating augmented alarm data by replacing a representative value of alarm data with another representative value according to the present invention.

[0084] The control unit (120) can generate augmented alarm data by replacing the representative value of the alarm data with another representative value within the replaceable cluster.

[0085] Specifically, the control unit (120) can generate augmented alarm data by maintaining a variable value of the first alarm data belonging to the first cluster within the replaceable cluster and matching the representative value of the second alarm data belonging to the second cluster within the replaceable cluster with the maintained variable value.

[0086] For example, referring to FIG. 7, it is assumed that within the first substitutable cluster, the first cluster (Bank A cluster) and the second cluster (Bank B cluster) are included. The first cluster (Bank A cluster) may include first alarm data (710). In this case, the first alarm data (710) includes a representative value (711), a dependent value (712), and a variable value (713), and the representative value (711) of the first alarm data is Bank A.

[0087] Additionally, the second cluster (Bank B cluster) may include second alarm data. In this case, the second alarm data includes a representative value, a dependent value, and a variable value, and the representative value of the second alarm data is Bank B.

[0088] In this case, the control unit (120) may maintain the variable value (713) of the first alarm data (710) belonging to the first cluster (Bank A cluster) without changing it. Additionally, the control unit (120) may generate augmented alarm data (720) by matching the representative value (Bank B) of the second alarm data belonging to the second cluster (Bank B cluster) within the replaceable cluster to the maintained variable value (713). Accordingly, the augmented alarm data (720) may include the representative value (721) (Bank B) of the second alarm data and the variable value (713) of the first alarm data.

[0089] In addition, the representative value (B Bank) of the augmented alert data (720) is a value representing the second cluster (B Bank cluster). Therefore, the newly generated augmented alert data (720) can belong to the second cluster.

[0090] In this manner, alarm data can be augmented. For example, it is assumed that the third to fifth clusters are grouped into a second substitutable cluster based on the same customer type, “Post Office.” In this case, the control unit (120) can generate augmented alarm data by matching the representative value (D Post Office) of the fourth alarm data belonging to the fourth cluster (D Post Office cluster) with the variable value of the third alarm data belonging to the third cluster (C Post Office cluster). Accordingly, the augmented alarm data may include the representative value (D Post Office) of the fourth alarm data and the variable value of the third alarm data, and may belong to the fourth cluster (D Post Office cluster). That is, the data from the network equipment of the D Post Office can be augmented by using the alarm data (more specifically, the variable value of the alarm data) generated from the network equipment of the C Post Office. In other words, since the operating pattern of the network equipment installed at Post Office C may also appear in the network equipment installed at Post Office D, new alarm data can be generated in the network equipment at Post Office D using the alarm data generated from the network equipment at Post Office C.

[0091] As another example, assume that the 6th and 7th clusters are grouped into a 3rd substitutable cluster based on the same equipment type called “PTN”. In this case, the control unit (120) can generate augmented alarm data by matching the representative value of the 7th alarm data belonging to the 7th cluster to the variable value of the 6th alarm data belonging to the 6th cluster. That is, a failure occurring in network equipment A having a specific equipment type can also occur in network equipment B having the same equipment type. Therefore, alarm data from network equipment B can be newly generated using alarm data generated from network equipment A.

[0092] Meanwhile, it is assumed that Cluster 8 (G Post Office Cluster) and Cluster 9 (H Court Cluster) exist, and that Cluster 8 (Post Office Cluster) and Cluster 9 (Court Cluster) are not similar to each other and are therefore not grouped into the same interchangeable cluster. In this case, alarm data in Cluster 9 (H Court Cluster) is not augmented using alarm data generated in Cluster 8 (G Post Office Cluster). In other words, similarity between clusters can be used to set the limit for augmenting alarm data.

[0093] Meanwhile, just like alarm data, augmented alarm data is also used for training an artificial intelligence model. Therefore, augmented alarm data must also have the same format as alarm data. Accordingly, the control unit (120) can generate augmented alarm data by matching a representative value of the second alarm data belonging to the second cluster to a variable value of the first alarm data belonging to the first cluster, and additionally matching a dependent value dependent on the representative value of the second alarm data. For example, if the representative value of the second alarm data is Bank B, the control unit (120) can generate augmented alarm data by additionally matching a dependent value of Bank B (identification code, installation location, equipment type of network equipment installed at Bank B). Accordingly, augmented alarm data (720) may include a representative value (721) of the second alarm data, a dependent value (722) of the second alarm data, and a variable value (713) of the first alarm data.

[0094] Next, the control unit (120) can generate a label for the augmented alarm data (S340). Specifically, the control unit (120) can generate a first label (740) corresponding to the augmented alarm data (720) by matching the first label (730) of the first alarm data (710) to the generated augmented alarm data (720).

[0095] More specifically, the first alarm data (710) may already have a first label (730) (e.g., “actual failure”) matched to it, and the first label (730) may be a label created by a person. In this case, the first alarm data (710) and the first label (730) constitute a single training data set, and the first alarm data (710) may be provided as input data to an artificial intelligence model, and the first label (730) may be provided as correct answer data to the artificial intelligence model. And when the artificial intelligence model outputs output data based on the input data, the control unit (120) or other training device can train the artificial intelligence model based on the difference between the output data and the correct answer data.

[0096] Likewise, in order to utilize augmented alarm data for training an artificial intelligence model, a label must also be matched to the augmented alarm data (720). In this case, the control unit (120) may use the first label (730) of the first alarm data (710), which provided a variable value to the augmented alarm data (720), as the first label (740) corresponding to the augmented alarm data (720). That is, the label “actual failure” can also be matched to the augmented alarm data (720).

[0097] In this case, the augmented alarm data (720) and the first label (740) constitute a single training data set, and the augmented alarm data (720) can be provided as input data to the artificial intelligence model, and the first label (740) can be provided as correct data to the artificial intelligence model.

[0098] Meanwhile, labels may be determined according to the purpose of the artificial intelligence model. For example, if the purpose of the artificial intelligence model is to classify whether alarm data was caused by an actual failure of network equipment or by customer operation, the first label may be “Actual Failure,” which is an alarm caused by an actual failure of network equipment, and the second label may be “Customer Operation,” which is an alarm caused by customer operation.

[0099] As another example, if the purpose of an artificial intelligence model is to classify whether the alarm data indicates a failure situation of network equipment or a normal situation of network equipment, the first label may be a “normal situation” indicating that no problem has occurred with the network equipment, and the second label may be a “failure” indicating that a failure has occurred with the network equipment. Additionally, the “failure” may be further subdivided so that the first label indicates a “normal situation,” the second label indicates an “actual failure,” and the third label indicates “customer operation.”

[0100] Next, a second embodiment for generating augmented alarm data by matching a newly created variable value to a representative value of alarm data will be explained with reference to FIGS. 8 and 9.

[0101] FIG. 8 is a diagram illustrating a method for generating a variable value and matching the generated variable value to a representative value according to the present invention.

[0102] The control unit (120) can generate a new variable value to match the representative value of the alarm data. Specifically, the control unit (120) can generate a new variable value based on domain knowledge of the cluster corresponding to the representative value. Here, the domain knowledge may include at least one of the range of generating variable values ​​in the cluster and the type of alarm.

[0103] More specifically, the control unit (120) can determine the generation range of variable values ​​based on domain knowledge of the cluster corresponding to the representative value. For example, if the first item among the variable values ​​represents the hour / minute / second among the time information where the alarm occurred, multiple alarm data within the cluster corresponding to the representative value display an arbitrary “hour” from 0 to 24, an arbitrary “minute” from 0 to 60, and an arbitrary “second” from 0 to 60 in the first item. In this case, the control unit (120) can determine the generation range of the first item among the variable values ​​to be an “hour” from 0 to 24, a “minute” from 0 to 60, and a “second” from 0 to 60. As another example, if the second item among the variable values ​​represents the “day of the week” among the time information where the alarm occurred, multiple alarm data within the cluster corresponding to the representative value display an arbitrary “day of the week” from Monday to Sunday in the second item. In this case, the control unit (120) can determine the generation range of the second item among the variable values ​​to be Monday to Sunday. In this way, the generation range of variable values ​​can be determined for year, month, day, weekday, weekend, etc.

[0104] Additionally, the control unit (120) can determine the generation range of the state value among the variable values ​​based on domain knowledge of the cluster corresponding to the representative value. For example, the third item among the variable values ​​represents the magnitude of the signal, and multiple alarm data within the cluster corresponding to the representative value have a value of -70dB to 70dB in the third item. In this case, the control unit (120) can determine the generation range of the third item among the variable values ​​to be -70dB to 70dB.

[0105] The control unit (120) can determine the type of alarm to be included in the variable value based on domain knowledge of the cluster corresponding to the representative value. For example, if multiple alarm data within the cluster corresponding to the representative value have alarm types of types 1 to 6, the newly created variable value may also include alarm types of types 1 to 6.

[0106] Next, the control unit (120) can generate a new variable value using the generation range and the type of alarm. That is, the control unit (120) can generate a new variable value within the previously determined generation range, and the newly generated variable value can also include the type of alarm that existed in the existing variable value.

[0107] One or more items of a newly generated variable value for a specific cluster may be identical to one or more items of a variable value of multiple alarm data within that specific cluster. Additionally, the generation range of the variable value can also be appropriately set by referencing multiple alarm data within that specific cluster. For example, by utilizing domain knowledge of a specific cluster, it is possible to prevent the generation of unrealistic and meaningless values ​​in the newly generated variable value, such as the “hour” appearing as 26 hours, the “minute” appearing as 85 minutes, or the “month” appearing as 15 months.

[0108] Meanwhile, when a designated range for variable values ​​(at least one of the variable value generation range and alarm type) is determined, the control unit (120) can generate multiple variable values ​​within the designated range using various distribution algorithms. For example, the control unit (120) can select a uniform distribution, a normal distribution, a Gaussian distribution, etc., within the designated range and generate multiple variable values ​​that follow the selected distribution. Additionally, the control unit (120) can generate multiple variable values ​​by reflecting the distribution of previously collected alarm data. In FIG. 8, a set of generated variable values ​​(810) is shown, and the following is explained assuming that the first variable value (813) among the set of variable values ​​(810) is matched to the representative value (821).

[0109] The control unit (120) can generate augmented alarm data (830) by matching the generated first variable value (813) to the representative value (821). Specifically, if the first variable value (813) is generated based on the domain knowledge of the first cluster, the control unit (120) can generate augmented alarm data (830) by matching the first variable value (813) to the representative value (821) corresponding to the first cluster. In this case, the generated augmented alarm data (830) also belongs to the first cluster, and the generated augmented alarm data (830) may include the representative value (821) corresponding to the first cluster and the variable value (813) newly generated by referring to the domain knowledge of the first cluster.

[0110] Similar to alarm data, augmented alarm data is also used for training an artificial intelligence model. Therefore, augmented alarm data must also have the same format as alarm data. Accordingly, the control unit (120) can generate augmented alarm data by matching a first variable value (813) to a representative value (821) corresponding to a first cluster, and additionally matching a dependent value dependent on the representative value (821).

[0111] Figure 9 is a diagram illustrating a method for generating labels for augmented alarm data.

[0112] The x-axis of the graph in Fig. 9 represents a day divided into fixed time units, and the y-axis represents the frequency. That is, the length of the bar graph shown in Fig. 9 may represent the number (or ratio) of failures that occurred at that time. Additionally, the length of the bar graph shown in Fig. 9 may represent the number (or ratio) of alarm data collected at that time.

[0113] The control unit (120) can generate a label for the augmented alarm data (S340). In the first embodiment described in FIGS. 5 to 7, the label of the alarm data with a variable value provided to the augmented alarm data is used as is, whereas in the second embodiment described in FIGS. 8 to 9, the augmented alarm data is generated by newly creating a variable value. Therefore, in the second embodiment, a separate process for determining the label for the augmented alarm data is required.

[0114] The control unit (120) can match a label to the augmented alarm data based on domain knowledge of the cluster to which the augmented alarm data belongs. Here, the cluster to which the augmented alarm data belongs may refer to a cluster that matches the representative value described above, or it may refer to a substitutable cluster in which two or more clusters are grouped together.

[0115] Specifically, when referring to multiple alarm data within the cluster to which the augmented alarm data belongs, variable values ​​may exhibit specific patterns. For example, let us assume that the objective of the artificial intelligence model is to classify whether a failure is a "real failure" caused by actual network equipment failure or a "customer operation" caused by customer manipulation. Referring to the variable values ​​of the alarm data within the Post Office cluster in Fig. 9, failures tend to occur intensively between 6:00 PM and 7:00 PM; this is attributed to Post Office employees turning off the power while leaving work during this period. In other words, this failure is caused by "customer operation" and is not a "real failure."

[0116] In addition, referring to the variable values ​​of the alarm data within the bank cluster in Fig. 9, failures tend to occur intensively between 0:00 and 2:00. This is attributed to the bank server administrator resetting the server between 0:00 and 2:00, and is a failure caused by “customer operation” and not an “actual failure.”

[0117] Accordingly, the control unit (120) can match a label corresponding to the pattern to the augmented alarm data when the augmented alarm data corresponds to a pattern identified from alarm data within a cluster corresponding to a representative value. For example, the pattern identified from alarm data within a post office cluster and additional information about customers is “a failure caused by customer operation occurs between 6 PM and 7 PM.” Therefore, if the augmented alarm data (911) belongs to a post office cluster and the time of occurrence of the alarm within the augmented alarm data (911) is between 6 PM and 7 PM, the control unit (120) can match “customer operation,” which is a label corresponding to the pattern, to the augmented alarm data (911). As another example, the pattern identified from alarm data within a bank cluster and additional information about customers is “a failure caused by customer operation occurs between 0 AM and 2 AM.” And if the augmented alarm data (920) belongs to a bank cluster and the time of occurrence of the alarm within the augmented alarm data (920) is between 0 o'clock and 2 o'clock, the control unit (120) can match “customer operation,” which is a label corresponding to the pattern, to the augmented alarm data (911).

[0118] Meanwhile, if the augmented alarm data does not have a pattern identified from the alarm data within the cluster corresponding to the representative value, the control unit (120) can match a different label to the augmented alarm data. For example, the pattern identified from the alarm data within the bank cluster is “a failure caused by customer operation occurs between 0:00 and 2:00.” And if the augmented alarm data (930) belongs to the bank cluster and the time of occurrence of the alarm within the augmented alarm data (930) is 11:00 AM, the augmented alarm data (930) does not have a pattern identified from the alarm data. In this case, the control unit (120) can match “actual failure,” which is a label different from the label corresponding to the pattern, to the augmented alarm data (930).

[0119] Meanwhile, augmented alert data and labels matched to augmented alert data constitute a single training data set, and the augmented alert data can be provided as input data to an artificial intelligence model, and the labels can be provided as ground truth data to an artificial intelligence model.

[0120] Meanwhile, when the matching of labels for the augmented alarm data is completed, the control unit (120) can integrate the previously collected alarm data and the newly generated augmented alarm data to form a plurality of training sets for training the artificial intelligence model.

[0121] Meanwhile, the previously collected alarm data is actual existing data, and the previously collected alarm data is matched with labels directly labeled by a person. In contrast, the augmented alarm data is generated by the device (100), and the labels matching it are also due to the device (100)'s auto-labeling. Considering this difference in reliability, the control unit (120) or other training device may assign different weights to training using alarm data and training using augmented alarm data.

[0122] For example, the control unit (120) or other training device may provide alarm data as input data to the artificial intelligence model and update the parameters (weights, biases, etc.) of the artificial intelligence model using the difference (LOSS) between the output data output by the artificial intelligence model and the correct answer data (label). In this case, the control unit (120) may apply a first weight to the difference (LOSS) and then update the parameters (weights, biases, etc.) of the artificial intelligence model using the difference (LOSS) to which the first weight has been applied.

[0123] Additionally, the control unit (120) or other training device may provide augmented alarm data as input data to the artificial intelligence model and update the parameters (weights, biases, etc.) of the artificial intelligence model using the difference (LOSS) between the output data output by the artificial intelligence model and the correct data (label). In this case, the control unit (120) may update the parameters (weights, biases, etc.) of the artificial intelligence model after assigning a second weight to the difference (LOSS). Also, the first weight may be greater than the second weight. For example, the first weight may be 1 and the second weight may be 0.7.

[0124] As such, according to the present invention, there is an advantage in that the performance of an artificial intelligence model can be improved by augmenting training data using previously collected data, even for network data in a table format consisting of multiple items, to which general augmentation techniques are difficult to apply.

[0125] In addition, according to the present invention, the limitations of difficult collection of abnormal data can be overcome, and abnormal data can be generated that is not dependent on the characteristics of previously collected network data, thereby resolving the imbalance between normal data and abnormal data.

[0126] In addition, according to the present invention, by classifying representative values ​​and variable values ​​within the data, generating clusters based on representative values, and augmenting the data and performing auto-labeling using the characteristics of the clusters or domain knowledge, there is an advantage of being able to generate highly reliable augmented data while reducing the cost required for data augmentation.

[0127] The foregoing invention may be implemented as computer-readable code on a medium on which a program is recorded. A computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. Additionally, the computer may include a control unit. Accordingly, the above detailed description should not be interpreted restrictively in all respects and should be considered exemplary. The scope of the invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

Claims

Claim 1 A method for augmenting alarm data comprising: a step of collecting multiple alarm data generated from multiple network devices and composed of multiple items; a step of classifying the multiple items into representative values ​​and variable values; a step of generating augmented alarm data by replacing a first representative value of alarm data belonging to a first cluster within a replaceable cluster with a second representative value of another cluster within the replaceable cluster; and a step of generating a label for the augmented alarm data. Claim 2 A method for augmenting alarm data according to claim 1, wherein the step of classifying the plurality of items into a representative value and a variable value comprises: a step of setting an item among the plurality of items that can classify the plurality of alarm data into a plurality of clusters as the representative value; and a step of setting an item among the plurality of items that constitutes the content of the alarm as the variable value. Claim 3 In paragraph 2, the above representative value is a method for augmenting alarm data, which is a name assigned to network equipment by the network operator, an identification code of network equipment, or a type of network equipment. Claim 4 In claim 2, the variable value is a method for augmenting alarm data including at least one of the alarm occurrence time, alarm type, and status value. Claim 5 In claim 2, the step of generating the augmented alarm data comprises: a step of generating a plurality of clusters by grouping alarm data having the same representative value; a step of determining similarity between the plurality of clusters; and a step of setting two or more clusters that are similar to each other as interchangeable clusters based on the similarity. Claim 6 In claim 5, the step of determining similarity between the plurality of clusters includes the step of determining similarity between the plurality of clusters based on domain knowledge corresponding to the representative value; and the domain knowledge includes at least one of the customer type of the network equipment and the equipment type of the network equipment, a method for augmenting alarm data. Claim 7 In claim 5, the step of determining similarity between the plurality of clusters comprises: a step of calculating a first data distribution corresponding to the first cluster using a variable value of alarm data belonging to the first cluster; a step of calculating a second data distribution corresponding to the second cluster using a variable value of alarm data belonging to the second cluster; and a step of determining whether the first cluster and the second cluster are similar by comparing the first data distribution and the second data distribution; a method for augmenting alarm data. Claim 8 In claim 5, the step of generating the augmented alarm data further comprises the step of generating the augmented alarm data by maintaining a variable value of the first alarm data belonging to the first cluster within the substitutable cluster and matching a representative value of the second alarm data belonging to the second cluster within the substitutable cluster with the maintained variable value; a method for augmenting alarm data. Claim 9 In claim 8, the step of generating a label for the augmented alarm data comprises the step of matching the label of the first alarm data to the generated augmented alarm data to generate a label corresponding to the generated augmented alarm data; a method for augmenting alarm data. Claim 10 A method for augmenting alarm data according to claim 2, further comprising the step of generating augmented alarm data by matching a newly generated variable value to a representative value of the alarm data, wherein the step of generating augmented alarm data by matching a newly generated variable value to a representative value of the alarm data comprises: a step of generating a new variable value based on domain knowledge of a cluster corresponding to the representative value; and a step of generating augmented alarm data by matching the generated variable value to the representative value. Claim 11 In claim 10, the step of newly generating a variable value based on domain knowledge of a cluster corresponding to the representative value comprises: a step of determining at least one of a generation range of the variable value and a type of alarm based on the variable value of alarm data within the cluster corresponding to the representative value; and a step of newly generating the variable value using at least one of the generation range and the type of alarm; a method for augmenting alarm data. Claim 12 In claim 10, the step of generating a label for the augmented alarm data comprises: a step of matching a label corresponding to the pattern to the augmented alarm data when the augmented alarm data corresponds to a pattern identified from the cluster alarm data corresponding to the representative value; and a step of matching a different label to the augmented alarm data when the augmented alarm data does not have a pattern identified from the cluster alarm data corresponding to the representative value. Claim 13 An alarm data augmentation device comprising: a data collection unit that collects multiple alarm data generated from multiple network devices and composed of multiple items; and a control unit that classifies the multiple items into representative values ​​and variable values, generates augmented alarm data by replacing a first representative value of alarm data belonging to a first cluster within a replaceable cluster with a second representative value of another cluster within the replaceable cluster, and generates a label for the augmented alarm data. Claim 14 In claim 13, the control unit sets an item among the plurality of items capable of classifying the plurality of alarm data into a plurality of clusters as the representative value, and sets an item among the plurality of items constituting the content of the alarm as the variable value. Claim 15 In Clause 14, the above representative value is a device for augmenting alarm data, which is a name assigned to network equipment by the network operator, an identification code of network equipment, or a type of network equipment. Claim 16 In claim 14, the variable value is an alarm data enhancement device comprising at least one of the alarm occurrence time, alarm type, and status value. Claim 17 In claim 14, the control unit groups alarm data having the same representative value to create a plurality of clusters, determines the similarity between the plurality of clusters, and sets two or more clusters that are similar to each other as interchangeable clusters based on the similarity. Claim 18 In claim 17, the control unit determines the similarity between the plurality of clusters based on domain knowledge corresponding to the representative value, and the domain knowledge includes at least one of the customer type of the network equipment and the equipment type of the network equipment, an alarm data augmentation device. Claim 19 In claim 17, the control unit calculates a first data distribution corresponding to a first cluster using a variable value of alarm data belonging to a first cluster, calculates a second data distribution corresponding to a second cluster using a variable value of alarm data belonging to a second cluster, and determines whether the first cluster and the second cluster are similar by comparing the first data distribution and the second data distribution. Claim 20 In claim 17, the control unit is an alarm data augmentation device that generates augmented alarm data by maintaining a variable value of first alarm data belonging to a first cluster within the substitutable cluster and matching a representative value of second alarm data belonging to a second cluster within the substitutable cluster with the maintained variable value.

Images