Method for detecting anomaly in time series data and computing apparatus performing same

Abstract

A method for detecting an anomaly in time series data, which is performed by a computing apparatus, comprises the steps of: (A) collecting training data classified as normal and abnormal; (B) extracting a time series feature from the collected training data; (C) defining a threshold value for detecting whether or not time series data is abnormal, by inputting the extracted time series feature into a pre-provided classification model; and (D) extracting a time series feature of real-time time series data, and detecting, by using the defined threshold value, whether or not the real-time time series data is abnormal.

Description

Anomaly detection method for time series data and computing device performing the same

[0001] The present disclosure relates to a method for detecting anomalies in time series data and a computing device performing the same, and more specifically, to a method for distinguishing anomalies in time series data using a deep learning algorithm that receives data having a time series nature from equipment and sets a normal range.

[0002] Unless otherwise indicated herein, the materials described in this section are not prior art to the claims of this application, and their inclusion in this section is not intended to be admitted as prior art.

[0003] Anomaly detection algorithms are techniques for identifying or classifying outliers in a data set. An outlier is defined as a data point that deviates from a given data pattern. Algorithms for identifying outliers include statistical algorithms, which use the mean and standard deviation of the data to calculate a score for each data point, machine learning algorithms, and deep learning algorithms.

[0004] Machine learning-based algorithms include Isolation Forest, which divides data into trees to detect outliers with unique patterns, and One-Class SVM (Support Vector Machine), which trains a model using only normal values ​​and detects outliers by determining whether new data belongs to the normal class.

[0005] Additionally, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), a clustering algorithm, clusters data points based on density and considers data in low-density areas as outliers. K-Means detects outliers using the distance from the cluster center.

[0006] Among deep learning-based algorithms, autoencoders use neural networks to reconstruct input data and consider large reconstruction errors as outliers. Furthermore, recurrent neural networks, such as LSTM (Long Short-Term Memory), can be used to detect outliers by learning patterns in time-series data.

[0007] Meanwhile, existing algorithms for identifying outliers suffer from the problem of not considering the time-series nature of the data. Data understanding and domain knowledge are crucial for outlier detection, and in some cases, combining multiple algorithms can be effective.

[0008] [Prior Art Literature]

[0009] (Patent Document 1) 1. Korean Patent Publication No. 10-2023-0011117 (January 20, 2023)

[0010] (Patent Document 2) 2. Korean Patent Publication No. 10-2023-0086461 (June 15, 2023)

[0011] A method for detecting anomalies in time series data according to an embodiment and a computing device for performing the same extracts characteristics of time series data using a deep learning technique and determines whether there is an anomaly in time series data using an SVDD (Support Vector Data Description) model, which is an anomaly detection algorithm.

[0012] In addition, a method for detecting anomalies in time series data according to an embodiment and a computing device performing the same acquire time series data using an LSTM algorithm to acquire characteristics of time series data, and learn the size of a sphere so that normal and abnormal values ​​can be distinguished through the sphere by applying an SVDD model to the acquired time series data. Thereafter, the computing device according to the embodiment tests how far apart data are from the center of the sphere using test data when the entire learning is completed, and records the result. In the embodiment, the computing device distinguishes normal values ​​and abnormal values ​​based on the size of the sphere learned in the experimental phase.

[0013] However, the problems to be solved according to one embodiment are not limited to those mentioned above.

[0014] A method for detecting anomalies in time series data, performed in a computing device according to an embodiment, comprises: (A) a step of collecting learning data classified as normal or abnormal; (B) a step of extracting time series features from the collected learning data; (C) a step of inputting the extracted time series features into a pre-prepared classification model to define a threshold value for detecting anomalies in time series data; and (D) a step of extracting time series features in real-time time series data and detecting anomalies in the real-time time series data using the defined threshold value.

[0015] In addition, the classification model can be prepared through at least one of soft boundary learning, which defines a soft threshold using only normal values ​​of time series features, and hard boundary learning, which defines a hard threshold using normal values ​​and abnormal values ​​of time series features.

[0016] In addition, soft boundary learning can be used to calculate the center point as the average of normal values, set it as the center point, and then learn the distance between the normal value and the center point to define the radius within the normal range as the threshold value.

[0017] In addition, hard boundary learning learns a hard threshold value that distinguishes between normal and abnormal values ​​by learning the distance between normal values ​​and abnormal values ​​and the center point, but can learn a hard threshold value by imposing a penalty on points corresponding to abnormal values ​​in a soft threshold value that uses only normal values.

[0018] In addition, step (D) inputs the time series features of real-time time series data into the classification model to map values ​​corresponding to the time series features, and if the distance between the mapped value and the center point is within a threshold value, it is judged as normal, and if the distance between the mapped value and the center point exceeds the threshold value, it is judged as abnormal.

[0019] In addition, after step (D), a step of performing incremental learning on a classification model using the result of detecting anomalies in real-time time series data as learning data may be further included.

[0020] The method for detecting anomalies in time series data as described above and the method for proposing a computing device for performing the same improve the accuracy and reliability of determining whether there are anomalies in time series data.

[0021] In addition, the method for detecting anomalies in time series data according to an embodiment and the computing device performing the same can be widely applied to markets requiring the determination of abnormalities in products and process systems such as manufacturing, robots, and production.

[0022] The effects of the present invention are not limited to the effects described above, and should be understood to include all effects that can be inferred from the detailed description of the present invention or the composition of the invention described in the claims.

[0023] Figure 1 is a drawing for explaining the process of implementing an anomaly detection model for time series data implemented in an embodiment.

[0024] Figure 2 is a diagram showing an anomaly detection process of time series data according to an embodiment.

[0025] Figure 3 is a block diagram showing a computing device according to an embodiment.

[0026] Figure 4 is a diagram showing a processor configuration according to an embodiment.

[0027] Figure 5 is a diagram showing the LSTM (Long Short Term Memory) model.

[0028] Figure 6 is a diagram for explaining the learning process of an SVDD model according to an embodiment.

[0029] Figure 7 is a drawing for explaining the incremental data learning method used in the embodiment.

[0030] Figure 8 is a diagram showing a time series data anomaly detection process according to an embodiment.

[0031] Figure 9 is a diagram showing a feature extraction process of time series data according to an embodiment.

[0032] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0033] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0034] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0035] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0036] In this specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Furthermore, a single unit may be realized using two or more pieces of hardware, and two or more units may be realized by a single piece of hardware.

[0037] Some of the operations or functions described herein as being performed by a terminal, apparatus, or device may instead be performed by a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described herein as being performed by a server may also be performed by a terminal, apparatus, or device connected to the server.

[0038] Hereinafter, the present invention will be described in detail with reference to the attached drawings.

[0039] Figure 1 is a diagram for explaining the process of implementing an anomaly detection model for time series data implemented in an embodiment.

[0040] Referring to FIG. 1, a computing device that performs a time series data anomaly detection method according to an embodiment implements a deep learning model that performs anomaly detection of time series data. To this end, the computing device for time series data anomaly detection according to the embodiment extracts training data and test data from a database in which time series data is stored. Thereafter, a feature extraction algorithm is used to acquire the characteristics of the time series data to acquire the characteristics of the time series data. In an embodiment, the feature extraction algorithm may include LSTM. The LSTM (Long Short-Term Memory) algorithm is a deep learning algorithm that is a variation of a recurrent neural network (RNN) and is used to identify the characteristics of time series data. In an embodiment, LSTM may be used to acquire the characteristics of time series data to effectively improve the long-term memory problem of the RNN algorithm.

[0041] Thereafter, the computing device applies the acquired time series data to a classification model to learn the size of the sphere so that normal and abnormal values ​​can be distinguished through the sphere. In an embodiment, the classification model may include an SVDD model, etc. The computing device according to the embodiment facilitates distance learning by using the center of a portion of the initial data to set the center value. When learning of the training data and the test data is completed, the computing device tests how far the data are from the center of the sphere using the test data and records the result. During the testing process, normal values ​​and abnormal values ​​are distinguished based on the learned sphere size. In an embodiment, various recurrent neural network algorithms may be used to acquire the characteristics of time series data.

[0042] Figure 2 is a diagram showing an anomaly detection process of time series data according to an embodiment.

[0043] Referring to FIG. 2, the computing device performs an offline training process and an online testing process to perform a time series data anomaly detection method.

[0044] In one embodiment, a computing device collects initial time-series data, classifies it into test data and training data, and determines the size of the training data. If the training data is sufficiently large, exceeding a preset size, it is input into a feature extraction algorithm to extract features. If the training data is insufficient, falling below a predetermined size, the training data is increased. In one embodiment, the training data can be increased using a data augmentation algorithm.

[0045] After extracting features using a feature extraction algorithm, normal and outlier values ​​of time series features are input for classification learning used as a classification model. In an embodiment, the classification model may include SVDD. In an embodiment, the classification model performs soft boundary learning and hard boundary learning. In an embodiment, soft boundary learning is performed by inputting normal values, and hard boundary learning is performed by learning both normal values ​​and outliers. In an embodiment, a center point is calculated using normal values, and the calculated center point is used for soft boundary learning and hard boundary learning. Thereafter, a computing device updates the boundary values ​​of the classification model based on the results of soft boundary learning and hard boundary learning.

[0046] In an embodiment, a computing device collects time-series data for online testing and extracts features using a feature extraction algorithm. At this time, the computing device can input classified test data for offline learning and extract features from the data using a feature extraction algorithm. Thereafter, the computing device collects the updated results of the classification model boundary value, calculates the classification model boundary value, and compares the calculated boundary value with the distance from the data and the center point to determine an outlier. If the distance between the center point and the data is less than the boundary value, the computing device determines the value as normal, and if the distance between the center point and the data is greater than the boundary value, the computing device determines the value as an outlier.

[0047] FIG. 3 is a block diagram showing a computing device according to an embodiment.

[0048] In the embodiment, the computing device (100) is a system that provides services to other computers or devices in a computer network or stores and manages data. The configuration of the computing device (100) illustrated in FIG. 2 is merely a simplified example.

[0049] The communication unit (110) may be configured regardless of the communication mode, such as wired or wireless, and may be configured with various communication networks, such as a personal area network (PAN) and a wide area network (WAN). In addition, the communication unit (110) may operate based on the well-known World Wide Web (WWW), and may also utilize a wireless transmission technology used for short-distance communication, such as infrared (IrDA: Infrared Data Association) or Bluetooth. For example, the communication unit (110) may be responsible for transmitting and receiving data required to perform a technique according to an embodiment of the present disclosure.

[0050] The memory (120) may refer to any type of storage medium. For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk. Such a memory (120) may also constitute a database as illustrated in FIG. 1.

[0051] The memory (120) can store at least one instruction that can be executed by the processor (130). In addition, the memory (120) can store any type of information generated or determined by the processor (130) and any type of information received by the server (200). For example, the memory (120) stores RM data and RM protocols according to the user, as will be described later. In addition, the memory (120) stores various types of modules, instruction sets, and models.

[0052] The processor (130) may perform technical features according to embodiments of the present disclosure, which will be described later, by executing at least one instruction stored in the memory (120). In one embodiment, the processor (130) may be configured with at least one core and may include a processor for data analysis and / or processing, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), or a tensor processing unit (TPU) of a computer device.

[0053] Fig. 4 is a diagram showing a processor configuration according to an embodiment.

[0054] Referring to FIG. 4, a processor according to an embodiment may be configured to include a collection unit (121), a preprocessing unit (122), a learning unit (123), a judgment unit (124), a feedback unit (125), and a data generation unit (126). The term 'unit' used in this specification should be interpreted to include software, hardware, or a combination thereof, depending on the context in which the term is used. For example, software may be machine language, firmware, embedded code, and application software. As another example, hardware may be a circuit, a processor, a computer, an integrated circuit, an integrated circuit core, a sensor, a MEMS (Micro-Electro-Mechanical System), a passive device, or a combination thereof.

[0055] The collection unit (121) collects a series of data required for training a model for detecting time-series data anomalies. In an embodiment, the collection unit (121) collects training data classified as normal or abnormal. For example, the collection unit (121) collects training data and test data as a training data set for the model. To this end, the collection unit (121) can classify and extract training data and test data from a database storing time-series data.

[0056] The preprocessing unit (122) preprocesses the collected training data to remove biased or discriminatory data from the collected artificial intelligence learning data set. In an embodiment, the preprocessing unit (122) preprocesses the collected training data set and processes it into a form suitable for artificial intelligence model learning. For example, the preprocessing unit (122) may perform processes such as noise removal, outlier removal, and missing value processing. In addition, the preprocessing unit (122) may normalize data, remove outliers, or adjust the scale of data through data preprocessing to prevent the model from learning unnecessary patterns.

[0057] The learning unit (123) trains a deep learning neural network with the collected training data set to implement a deep learning model for time series data anomaly detection. The learning unit (123) according to an embodiment may store a time series data anomaly detection model (1). Each module or model may be in the form of an application executable by the processor (130). In an embodiment, a classification model for time series data anomaly detection may be implemented as an SVDD model. In addition, the classification model may be prepared through at least one of soft boundary learning, which defines a soft threshold value using only normal values ​​of time series features, and hard boundary learning, which defines a hard threshold value using normal values ​​and abnormal values ​​of time series features.

[0058] The learning unit (123) extracts time series features from the collected learning data. Thereafter, the extracted time series features are input into a pre-prepared classification model to define a threshold for detecting anomalies in the time series data. Thereafter, the learning unit (120) extracts time series features of the real-time time series data and uses deep learning to detect anomalies in the real-time time series data using the defined threshold. In an embodiment, the learning unit (123) can train a classification model for anomaly detection using initial time series data. In an embodiment, the classification model extracts time series features of the real-time time series data and detects anomalies in the real-time time series data using the defined threshold.

[0059] In addition, the learning unit (123) secures a normal value through data augmentation when the training data is below a certain capacity. In an embodiment, if the training data is image data, the training data is augmented through rotation, flipping, resizing, cropping, color adjustment, noise addition, etc. If the training data is text data, the training data is augmented through synonyms and antonyms, random word deletion, sentence structure change, etc. In addition, if the training data is voice data, in an embodiment, the training data can be augmented through methods such as speed adjustment, background noise addition, and time distortion.

[0060] In addition, the learning unit (123) extracts features of the learning data set by utilizing the time series feature extraction deep learning model of the feature extraction algorithms to build a classification model. The LSTM (Long Short Term Memory) model, which is exemplarily employed as a feature extraction algorithm in the embodiment, is a neural network that can model sequential information as the connection between units has a circular structure, and extracts features of multiple time series data. The LSTM illustrated in Fig. 5 is a deep learning model that solves the problems of the existing Recurrent Neural Network and is capable of extracting time series features of long-term memory, and is used to extract time series features for anomaly detection of time series data.

[0061] In the embodiment, the training data set includes training data and test data, each of which is separated from the initial time-series data. Furthermore, in the embodiment, the learning unit (123) learns a classification model for anomaly detection using time-series features acquired through a feature extraction algorithm. Furthermore, in the embodiment, the classification model performs hard boundary learning and soft boundary learning, and calculates anomaly detection thresholds based on the learning results.

[0062] In an embodiment, the classification model may be, for example, an SVDD model. This SVDD (Support Vector Data Description) model is an unsupervised learning model that identifies normal regions of data. The SVDD model detects a single hyperplane representing a class of data. This hyperplane indicates the normal region where data of that class are located. The SVDD model detects anomalies in time-series data by determining that all data points are abnormal the farther they are from this hyperplane.

[0063] In this embodiment, the classification model sets the center of the normal value and then uses the remaining data to learn the distance from the center value to define the radius value within the normal range. In this embodiment, the classification model generates a data-adaptive center point and optimizes data classification.

[0064] Figure 6 is a diagram for explaining the learning process of a classification model according to an embodiment.

[0065] Referring to Figure 6, the classification model learns the radius (R), which is the distance between the features of time series data and the center point (C), to learn a boundary (1) that can effectively distinguish between normal and abnormal data. In the embodiment, rather than using a center (Center) that is unrelated to the data, the classification model can quickly and accurately learn data through adaptive center point generation that calculates the center point as the average of the normal values ​​of the time series feature data.

[0066] In addition, in the embodiment, the classification model can simultaneously perform hard boundary learning and soft boundary learning to optimize data classification. In the embodiment, hard boundary learning is to learn only the normal value of time series data. Soft boundary learning is to learn the threshold value of the normal range by using hard threshold, normal value, and outlier together. In the embodiment, the classification model is trained to be optimal for data classification by using hard boundary learning and soft boundary learning methods simultaneously. In the embodiment, the outlier (10) is time series data (t n ) can be predicted and identified through feature analysis.

[0067] Below, soft boundary learning according to an embodiment is described.

[0068] Soft boundary learning according to the embodiment calculates the center point as the average of normal values, sets it as the center point, and then learns the distance between the normal value and the center point to define the radius within the normal range as the threshold.

[0069] In the embodiment, the classification model performs boundary learning through mathematical expression 1, which is a boundary learning function.

[0070] Mathematical formula 1

[0071]

[0072] In the embodiment, the classification model sets the average of the time series features obtained from the normal values ​​as the center C. Then, using only the normal values, the boundary learning method learns the distance R (radius) between the time series data features and C, thereby learning a threshold that includes the normal values. In the embodiment, for threshold learning, common features of normal values ​​are extracted and each data point is mapped close to the center (c) of the sphere.

[0073] For example, the classification model uses the average of the time series features obtained from the normal values ​​as the center value, extracts common features of the normal values, maps each data point around the center of the sphere, obtains the distance (R) between the extracted features and the center value, and learns and defines a threshold value that includes the normal values ​​based on the distance.

[0074] Below, a hard boundary learning method according to an embodiment is described.

[0075] Hard boundary learning according to the embodiment learns a hard threshold value that distinguishes between normal and abnormal values ​​by learning the distance between normal values ​​and abnormal values ​​and the center point, and learns a hard threshold value by imposing a penalty on points corresponding to abnormal values ​​in a soft threshold value that uses only normal values.

[0076] In the embodiment, the classification model performs boundary learning through mathematical expression 2, which is a hard boundary learning function.

[0077] Mathematical formula 2

[0078]

[0079] Boundary learning learns the distance R (radius) between the features of time series data and the center of the sphere (c) and learns a hard boundary threshold that can effectively distinguish between normal values ​​and outliers.

[0080] In the embodiment, boundary learning is a learning method that uses both normal values ​​and outliers, and enables optimal boundary learning by imposing a penalty on points of outliers in boundaries that use only normal values.

[0081] Additionally, the learning unit (123) trains a classification model that detects outliers in time series data through a deep learning method that gradually learns data. Fig. 7 is a diagram illustrating the incremental data learning method used in the embodiment.

[0082] Referring to FIG. 7, the learning unit (123) according to the embodiment utilizes a data-based performance enhancement technique. In the embodiment, the learning unit (123) learns new types of outliers generated while performing anomaly detection with new data, and periodically updates the threshold value of the classification model based on outliers that change over time and shape.

[0083] Additionally, the learning unit (123) retrains the algorithm through periodic updates. For example, when the anomaly detection performance drops below a certain level or a product production cycle ends, the learning unit (123) utilizes newly detected anomaly data as training data for the classification model. In an embodiment, the learning unit (123) can enhance the classification model by retraining the threshold value in the classification model.

[0084] A model in this specification may refer to any form of computer program that operates based on a network function, an artificial neural network, and / or a neural network. Throughout this specification, the terms model, neural network, network function, and neural network may be used interchangeably. A neural network is a network in which one or more nodes are interconnected through one or more links to form input node and output node relationships within the neural network. The characteristics of a neural network can be determined based on the number of nodes and links within the neural network, the correlation between the nodes and links, and the weight value assigned to each link. A neural network may be composed of a set of one or more nodes. A subset of the nodes constituting the neural network may constitute a layer.

[0085] A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. A deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder, a generative adversarial network (GAN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a Q network, a U network, a Siamese network, a generative adversarial network (GAN), a transformer, and the like. The description of the above-described deep neural network is merely an example, and the present disclosure is not limited thereto.

[0086] Neural networks can learn through at least one of the following methods: supervised learning, unsupervised learning, semi-supervised learning, self-supervised learning, or reinforcement learning. Neural network learning can be the process of applying knowledge to the neural network to perform a specific action.

[0087] Neural networks can be trained to minimize output errors. This process involves repeatedly inputting training data into the neural network, calculating the neural network output and target error for the training data, and backpropagating the neural network error from the output layer to the input layer to update the weights of each node in the neural network to reduce the error. In supervised learning, labeled data is used for each training data, while unsupervised learning uses unlabeled data. The amount of change in the connection weights of each updated node can be determined by the learning rate. The neural network's calculation of input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate can vary depending on the number of iterations in the neural network's training cycle. Additionally, to prevent overfitting, methods such as increasing the learning data, regularization, dropout that disables some nodes, and batch normalization layers can be applied.

[0088] The judgment unit (124) determines whether the time series data is abnormal based on the radius (R), which is the threshold value of the classification model, which is one of the learned classification models. In the embodiment, the judgment unit (124) determines that the distance between the center of the sphere and the data is a normal value if it is within the learned range, and determines that the distance is an abnormal value if it is a value exceeding the range. In addition, the judgment unit (124) determines that the distance between the center of the sphere and the data is an abnormal value if it is greater than the threshold value (R), and determines that the distance is a normal value if it is less than the threshold value.

[0089] The feedback unit (125) evaluates the learned artificial neural network model and deep learning model. In an embodiment, the feedback unit (125) can evaluate the artificial neural network model through at least one of accuracy, precision, and recall. Accuracy is an index that measures how well the results predicted by the artificial neural network model match the actual results. Precision is an index that measures the ratio of actual positives among the results predicted as positive. Recall is an index that measures the ratio of actual positives predicted by the model as positives. In an embodiment, the feedback unit (125) can calculate the accuracy, precision, and recall of the artificial neural network model, and evaluate the artificial neural network model based on at least one of the calculated indexes.

[0090] In an embodiment, the feedback unit (125) can measure the accuracy of the artificial neural network model using an evaluation dataset. The evaluation dataset consists of data that the model did not use for training and is used to objectively evaluate the model's performance. In an embodiment, the feedback unit (125) executes the artificial neural network model using the evaluation dataset and compares the predicted value of the artificial neural network model for each input data with the actual correct answer value of the corresponding data. Thereafter, the accuracy of the model's predictions can be measured based on the comparison results. For example, the accuracy in the feedback unit (125) can be calculated as the ratio of data correctly predicted by the model among the entire data.

[0091] In addition, the feedback unit (125) can calculate the F1 score, which is an index indicating the balance of precision and recall, which is an index calculated as the harmonic mean of precision and recall, evaluate the artificial neural network model based on the calculated F1 score, generate an AUC-ROC curve, which is an index that visualizes the performance of the classification model in a graph, and evaluate the artificial neural network model based on the generated AUC-ROC curve. In an embodiment, the feedback unit (125) can evaluate that the performance of the model is better as the area under the ROC curve (AUC) is closer to 1.

[0092] In addition, the feedback unit (125) can evaluate the interpretability of the artificial neural network model. In an embodiment, the feedback unit (125) evaluates the interpretability of the artificial neural network model through SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) methods. SHAP (SHapley Additive exPlanations) is a library that provides an interpretation of the results predicted by the model, and the feedback unit (125) extracts SHAP values ​​from the library. In an embodiment, the feedback unit (125) can predict how much the characteristic information input to the model influenced the model prediction through the extraction of SHAP values.

[0093] The Local Interpretable Model-agnostic Explanations (LIME) method is a method for explaining model predictions for individual samples. In one embodiment, the feedback unit (125) uses the LIME method to approximate the sample as an interpretable model and calculate the importance of each characteristic. Furthermore, the feedback unit (125) can estimate the influence of each characteristic variable by analyzing the model's internal weights and bias values.

[0094] The feedback unit (125) performs improvement work when the fairness of the artificial neural network model is low or shows discrimination. In an embodiment, the feedback unit (125) collects additional data representing a specific group when a certain level of data for that group is insufficient and performs a data preprocessing process. In an embodiment, the feedback unit (125) performs a data preprocessing process including data normalization, outlier removal, and data scaling to prevent the model from learning unnecessary patterns. In addition, in an embodiment, the feedback unit (125) can add specific conditions to the model learning algorithm to prevent discrimination or ensure fairness.

[0095] In the embodiment, the feedback unit (125) evaluates the performance of the model by comparing the model's predicted results with the actual results through confusion matrix analysis to ensure fairness. The confusion matrix is ​​a matrix that evaluates the classification performance of the model in supervised learning. The confusion matrix displays the classification results by comparing the model's predicted results with the actual results. In the embodiment, the feedback unit (125) can evaluate the performance of the model by calculating the accuracy and misclassification rate for each class through confusion matrix analysis.

[0096] Additionally, in the embodiment, the feedback unit (125) enables the distribution of data to be confirmed through visual analysis of learning data. For example, in the case of image data, image samples for each class can be visualized to evaluate the diversity and fairness of the data.

[0097] In addition, the feedback unit (125) verifies the fairness and diversity of the learning data and improves the artificial neural network model through fairness verification and evaluation index calculation. In an embodiment, fairness verification is to check whether the artificial neural network model shows discrimination for specific data attributes with respect to the learning information. In an embodiment, the feedback unit (125) can compare the number of samples for each attribute or evaluate the classification performance for each attribute to check whether discrimination for a specific attribute is present.

[0098] Additionally, the feedback unit (125) calculates various indicators to evaluate the performance of the artificial neural network model. For example, model performance can be evaluated by calculating indicators such as accuracy, precision, recall, and F1-score. At this time, indicators for each class can be calculated to evaluate the fairness and diversity of the model.

[0099] In addition, the feedback unit (125) collects feedback on problems that occur when the artificial neural network model is used in an actual environment, and continuously improves the artificial neural network model by reflecting the collected feedback in the artificial neural network model.

[0100] Below, a method for detecting anomalies in time series data is sequentially described. Since the operation (function) of the method for detecting anomalies in time series data according to the embodiment is essentially the same as the function of a computing device that performs the method for detecting anomalies in time series data, any description overlapping with that in FIGS. 1 to 8 will be omitted.

[0101] Fig. 8 is a diagram illustrating a time series data anomaly detection process according to an embodiment.

[0102] Referring to Figure 8, in step S110, features of time series data are extracted using deep learning in a computing device. In step S120, an SVDD model, a classification model for anomaly detection, is trained. In step S130, the trained SVDD value is used to determine whether the data is abnormal. For example, in step S130, if the distance between the center of the sphere and the data is smaller than the radius (R), which is the trained classification model threshold, the value is determined to be normal, and if it is larger, the value is determined to be abnormal.

[0103] Figure 9 is a diagram showing a model learning process according to an embodiment.

[0104] Referring to Figure 9, in step S121, the average of some initial data is calculated, and in step S122, the center value is calculated using the calculated average. Thereafter, step S130 is entered to perform anomaly detection of time series data.

[0105] The method for detecting anomalies in time series data as described above and the method for proposing a computing device for performing the same improve the accuracy and reliability of determining whether there are anomalies in time series data.

[0106] In addition, the method for detecting anomalies in time series data according to an embodiment and the computing device performing the same can be widely applied to markets requiring the determination of abnormalities in products and process systems such as manufacturing, robots, and production.

[0107] The disclosed content is merely an example, and various modifications and implementations can be made by a person skilled in the art without departing from the gist of the claims claimed in the patent, so the scope of protection of the disclosed content is not limited to the specific embodiments described above.

Claims

1. A method for detecting anomalies in time series data performed on a computing device, (A) A step of collecting learning data classified as normal or abnormal; (B) a step of extracting time series features from the collected learning data; (C) a step of inputting the extracted time series features into a pre-prepared classification model to define a threshold value for detecting anomalies in time series data; and (D) A method for detecting anomalies in time series data, comprising: a step of extracting time series features of real-time time series data and detecting anomalies in the real-time time series data using the defined threshold value; 2. In the first paragraph, the classification model A method for detecting anomalies in time series data, characterized in that the method comprises at least one of soft boundary learning, which defines a soft threshold value using only normal values ​​of time series features, and hard boundary learning, which defines a hard threshold value using normal values ​​and abnormal values ​​of the time series features.

3. In paragraph 2, The above soft boundary learning A method for detecting anomalies in time series data, wherein the center point is calculated as the average of the above normal values, the center point is set as the center point, and the distance between the above normal values ​​and the center point is learned to define the radius within the normal range as the threshold value.

4. In paragraph 2, The above hard boundary learning A method for detecting anomalies in time series data, characterized in that a hard threshold value is learned for distinguishing between normal and abnormal values ​​by learning the distance between the normal values ​​and abnormal values ​​and the center point, and the hard threshold value is learned by imposing a penalty on points corresponding to the abnormal values ​​in a soft threshold value that uses only the normal values.

5. In paragraph 1, Step (D) above, The time series features of the above real-time time series data are input into the above classification model and the values ​​corresponding to the time series features are mapped. If the distance between the mapped value and the center point is within the threshold value, it is judged as normal. An anomaly detection method for time series data, wherein if the distance between the mapped value and the center point exceeds the threshold value, it is determined to be an anomaly.

6. In paragraph 1, After step (D) above, A method for detecting anomalies in time series data, further comprising the step of performing incremental learning on the classification model by using the results of detecting anomalies in the real-time time series data as the learning data.

7. In a computing device for anomaly detection of time series data, a memory storing at least one instruction; and Includes a processor, By executing at least one instruction of the above by the processor, Collect learning data that has been judged to be normal or abnormal, and extract time series features from the collected learning data. A computing device for anomaly detection of time series data, which inputs the extracted time series features into a pre-arranged classification model to define a threshold value for detecting anomalies in time series data, extracts time series features of real-time time series data, and detects anomalies in the real-time time series data using the defined threshold value.

8. In paragraph 7, the classification model is A computing device characterized in that it is provided through at least one method among soft boundary learning that defines a soft threshold using only normal values ​​of time series features, and hard boundary learning that defines a hard threshold using normal values ​​and abnormal values ​​of the time series features.

9. In paragraph 8, The above soft boundary learning A computing device that calculates a center point using the average value of the above normal values ​​and sets it as the center point, and then learns the distance between the above normal values ​​and the center point and defines a radius within the normal range as the threshold value.

10. In paragraph 8, The above hard boundary learning A computing device characterized in that it learns a hard threshold value that distinguishes between normal and abnormal values ​​by learning the distance between the normal values ​​and abnormal values ​​and the center point, and learns the hard threshold value by imposing a penalty on points corresponding to the abnormal values ​​in a soft threshold value that uses only the normal values.

11. In paragraph 7, The above computing device, The time series features of the above real-time time series data are input into the above classification model and the values ​​corresponding to the time series features are mapped. If the distance between the mapped value and the center point is within the threshold value, it is judged as normal. A computing device that determines an abnormality if the distance between the mapped value and the center point exceeds the threshold value.

12. In paragraph 7, the computing device A computing device characterized by performing incremental learning on a classification model using the results of detecting anomalies in real-time time series data as learning data.

Images