Machine learning-based anomaly detection device and method

Abstract

A computing device for performing anomaly detection according to the present disclosure includes a memory including at least one memory bank which is a logic area; and a processor configured to train a neural network using at least one functional module, to generate a reduced feature extraction module, to generate the memory bank based on reduced feature data generated from the reduced feature extraction module, and to determine whether test data is normal or not using the generated reduced feature extraction module and memory bank.

Description

[Technical Field]

[0001] The present disclosure relates to an anomaly detection apparatus and method based on machine learning, and more particularly to an anomaly detection technique that utilizes machine learning based on deep learning. [Background technology]

[0002] Anomaly detection is defined as a technique for detecting abnormal data that can be distinguished from normal data. A frequent problem with anomaly detection techniques using deep learning classification models is the lack of abnormal data. Training a deep learning classification model requires a large amount of normal and abnormal data, but since most naturally available data is normal data, deep learning classification models have difficulty learning abnormal data.

[0003] To solve these problems, various techniques such as data augmentation and self-supervised learning have been studied. As part of this research, there has been ongoing research into technologies that can detect abnormal data by learning only from normal data. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton, "Simple Framework for Contrastive Learning of Visual Representations"ICML 2020 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure aims to realize an anomaly detection module based on machine learning that can detect abnormal data using only normal data.

[0006] Another objective of the present disclosure is to realize an anomaly detection module through data amplification and supervised learning in order to solve the problem of a lack of abnormality data for training an anomaly detection module that detects anomalies in manufacturing objects during the manufacturing process.

[0007] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0008] In order to achieve the above-mentioned technical problem, one aspect of the present disclosure provides a computing device for performing anomaly detection, comprising: a memory including at least one memory bank which is a logic domain; and a processor configured to train a neural network using at least one functional module, to generate a reduced feature extraction module, to generate the memory bank based on reduced feature data generated from the reduced feature extraction module, and to determine whether inspection data is normal or not using the generated reduced feature extraction module and the memory bank. As an operation of generating the reduced feature extraction module, the processor may train a feature extraction network using one or more first training data, generate one or more feature patch data from one or more second training data using the trained feature extraction network, train a dimensionality reduction network using the one or more feature patch data, and generate the reduced feature data from the feature patch data using the trained dimensionality reduction network.

[0009] Additionally, the feature extraction network of the present disclosure can be trained to determine the distance or similarity between two or more training data.

[0010] In addition, the operation of the preset scheme according to the present disclosure may include generating an abnormal class sample having a class different from one or more data included in the single-class dataset, or adding dummy data to the single-class dataset.

[0011] Additionally, the feature extraction network according to the present disclosure can be additionally trained to infer how the abnormal class samples were generated.

[0012] Furthermore, the feature extraction network according to the present disclosure may include one or more feature map generation layers, the one or more feature map generation layers generating at least one feature map, the feature patch data being generated from the at least one feature map, and the at least one feature map including at least one highly abstracted feature map.

[0013] Furthermore, the at least one feature map according to the present disclosure may include feature maps at all feature map generation layers of the feature extraction network.

[0014] In addition, the dimension reduction network according to the present disclosure can make the reduced feature data imply position information of the feature patch data in the training data or the feature map generated by the feature extraction network.

[0015] Additionally, the dimension reduction network according to the present disclosure can be trained by learning the distance or similarity between the one or more feature patch data.

[0016] Furthermore, the dimension reduction network according to the present disclosure can be additionally trained to infer position information of the feature patch data in the training data or the feature map generated by the feature extraction network based on the reduced feature data.

[0017] Additionally, generating a memory bank from the reduced feature data according to the present disclosure may include classifying one or more of the reduced feature data into one or more subsets.

[0018] In addition, determining whether test data is normal using the reduced feature extraction module generated by the present disclosure can involve generating reduced feature data for the test data using the reduced feature extraction module, using the reduced feature data to identify a nearest subset of reduced feature data for the test data from one or more subsets included in the memory bank, calculating an abnormality score using the nearest subset and the reduced feature data, and detecting whether the test data is abnormal based on the abnormality score.

[0019] In addition, to achieve the above-mentioned technical objectives, an anomaly detection method based on machine learning performed by a computing device according to one aspect of the present disclosure includes the steps of generating a reduced feature extraction module, generating a memory bank based on reduced feature data generated from the reduced feature extraction module, and determining whether test data is normal or not using the generated reduced feature extraction module and the memory bank, wherein the step of generating the reduced feature extraction module includes the steps of training a feature extraction network using one or more first training data, generating one or more feature patch data from one or more second training data using the trained feature extraction network, and generating one or more reduced feature data from the one or more feature patch data using a dimensionality reduction network.

[0020] In addition, to achieve the above-mentioned technical problem, a computer program stored in a computer-readable storage medium according to one aspect of the present disclosure causes a processor to perform the following operations for detecting anomalies when executed thereon, the operations including: generating a reduced feature extraction module; generating a memory bank based on reduced feature data generated from the reduced feature extraction module; and determining whether test data is normal or not using the generated reduced feature extraction module and the memory bank. The operation of generating the reduced feature extraction module includes: training a feature extraction network using one or more first training data; generating one or more feature patch data from one or more second training data using the trained feature extraction network; and generating one or more reduced feature data from the one or more feature patch data using a dimensionality reduction network.

[0021] Additionally, a computer-readable recording medium may be provided that records a computer program for executing a method for realizing the present disclosure. [Effects of the Invention]

[0022] According to the above-described means for solving the problems of the present disclosure, abnormal data can be detected using only normal data. Therefore, an effective learning process can be constructed while reducing data construction costs. Furthermore, a feature extraction network adapted to the data type can be trained using only normal data, which can be expected to result in more accurate abnormal value detection than conventional techniques. In addition, the outlier detection module takes into account more diverse information than conventional techniques, which can be expected to result in more accurate abnormal value detection than conventional techniques.

[0023] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a conceptual diagram illustrating an overview of a system that implements an anomaly detection method based on machine learning according to the present disclosure. [Figure 2] FIG. 1 illustrates the overall flow of a machine learning-based anomaly detection method according to the present disclosure. [Figure 3] FIG. 1 is a block diagram illustrating a machine learning based anomaly detection device according to the present disclosure. [Figure 4] FIG. 1 is a conceptual diagram illustrating one embodiment of a method for generating one or more training data according to the present disclosure. [Figure 5] FIG. 1 is an illustrative diagram illustrating one embodiment of a feature extraction network according to the present disclosure. [Figure 6] FIG. 1 is a conceptual diagram illustrating one embodiment of one or more feature patch data according to the present disclosure. [Figure 7] FIG. 1 illustrates one embodiment of a dimensionality reduction network according to the present disclosure. [Figure 8] FIG. 1 is a conceptual diagram illustrating one embodiment of a memory bank generation method according to the present disclosure. [Figure 9] 1 is a flowchart illustrating an embodiment of a method for determining whether test data is normal or not according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0025] The same reference numerals refer to the same components throughout this disclosure. This disclosure does not describe all elements of the embodiments, etc., and general content in the technical field to which the disclosure belongs or content that overlaps between the embodiments, etc. will be omitted. The terms "unit, module, component, block" used in the specification can be realized by software or hardware, and depending on the embodiment, multiple "units, modules, components, blocks" may be realized by one component, or one "unit, module, component, block" may include multiple components.

[0026] Throughout this specification, when a part is said to be "coupled" to another part, this includes not only direct coupling but also indirect coupling, and indirect coupling includes coupling via a wireless communication network.

[0027] The terms "first," "second," etc. are used to distinguish one component from another, and the components are not limited to the terms described above.

[0028] Hereinafter, the principles of operation and embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0029] Anomaly detection is defined as a technique for detecting abnormal data that can be distinguished from normal data. Anomaly detection techniques have made great strides with the use of deep learning classification models. A common problem with anomaly detection techniques using deep learning classification models is the lack of abnormal data. Training a deep learning classification model requires a large amount of normal and abnormal data. However, since most naturally available data is normal, deep learning classification models face difficulties in learning abnormal data. This problem can be particularly exacerbated when applying anomaly detection techniques based on deep learning classification models in the industrial sector. The primary issue is the scarcity of abnormal data due to the nature of mass production processes. Furthermore, anomaly detection in the industrial sector faces the problem of abnormal data exhaustion because defects that constitute abnormal data vary widely, from minor issues like scratches to major issues like missing parts.

[0030] To address these issues, various techniques, such as data augmentation and self-supervised learning, have been studied. As part of this research, research is ongoing into technologies that can detect abnormal data by learning only from normal data. In this specification, the term "device according to the present disclosure" includes all types of devices that can perform computations and provide results to users. For example, the device according to the present disclosure may include all or any one of a computer, a server device, and a portable terminal.

[0031] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0032] The server device is a server that communicates with external devices and processes information, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0033] The portable terminal is, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0034] The anomaly detection device based on machine learning according to the present disclosure may be realized by including at least one of the above-mentioned computer, server device, and portable terminal. For example, the anomaly detection device based on machine learning may be one of the computer, server device, and portable terminal, or may be realized as a system in which the server device performs the prediction method in the form of a web service and provides the service result to the computer and / or portable terminal.

[0035] In this disclosure, a reduced feature extraction module refers to a module that receives at least one input data and generates reduced feature data. The input data of the reduced feature extraction module may be, but is not limited to, image data. The reduced feature extraction module may be composed of one or more neural networks, and output data of one neural network may serve as input data of another neural network. The one or more different networks constituting the reduced feature extraction module may have different structures, but this is not necessarily the case. Furthermore, the one or more different neural networks constituting the reduced feature extraction module may be trained using different data sets and different techniques. The one or more different neural networks constituting the reduced feature extraction module may not be trained simultaneously. The input data of the reduced feature extraction module may be training data or test data. The training data may be data for training some or all of the neural networks constituting the reduced feature extraction module. The input data of the reduced feature extraction module may be a dataset containing one or more homogeneous or heterogeneous data.

[0036] In the present disclosure, the feature extraction network may be one of the networks constituting the reduced feature extraction module. The feature extraction network may refer to a neural network that generates one or more corresponding feature data from input data. The feature extraction network may be expressed as a structure in which layers performing pre-specified operations are stacked. In the present disclosure, the pre-specified operation performed in each layer of the feature extraction network may be a convolution operation, and each layer may generate one or more feature maps according to the result of the convolution operation. As will be described later with reference to FIG. 6, the feature maps generated according to the result of the convolution operation in each layer may be expressed as three-dimensional tensors. However, the above-described content regarding the convolution operation of the feature extraction network and the generated data is merely an example of realizing a feature extraction network, and the structure and function of the feature extraction network are not limited thereto.

[0037] The dimension reduction network according to the present disclosure may be a neural network that receives input feature data and generates reduced feature data corresponding to the input data. The dimension reduction network may be one or more neural networks constituting a reduced feature extraction module. The dimension reduction network may have a different structure from the feature extraction network and may be trained using a different dataset and technique than the feature extraction network. The dimension reduction network may be composed of one or more different neural networks, each of which may have a different loss function and may influence each other. The dimension reduction network may also be trained using a combination of the loss functions of the one or more neural networks that constitute it. In particular, the dimension reduction network according to the present disclosure may be composed of one or more neural networks that measure the similarity between two or more data. In this case, the dimension reduction network may use only some of the multiple neural networks in an exploitation step after training is completed. The reduced feature data generated by the dimension reduction network may have reduced dimensions compared to the input data. For example, if the input data is a 2,048-dimensional vector, the dimension reduction network can generate a 1,024-dimensional vector or a 512-dimensional vector corresponding to the 2,048-dimensional vector. The above is merely an example of the configuration and function of the dimension reduction network, and the dimension reduction network is not limited to the above.

[0038] The feature patch data according to the present disclosure is data generated by a feature extraction network and may refer to a part or all of the feature map generated in each layer of the feature extraction network. The feature patch data may vary in size depending on a preset patch size. The feature patch data may also be a slice of the feature map generated in each layer of the feature extraction network. This will be described in detail later with reference to FIG. 6.

[0039] The reduced feature data according to the present disclosure may be data generated by inputting the above-described feature patch data into the above-described dimension reduction network. The reduced feature data may be data in which loss of information contained in the above-described feature patch data is minimized. In particular, the reduced feature data may be constructed to include position information of the feature patch data in the original input data. The reduced feature data may have a format different from that of the above-described feature patch data. The above content is merely an example of realizing the reduced feature data, and interpretation of the reduced feature data should not be limited to this content.

[0040] The memory bank in the present disclosure may be a collection of one or more reduced feature data. Once the memory bank is constructed, it can be used to detect whether or not the test data is abnormal. In one embodiment, the anomaly detection device in the present disclosure can determine whether or not the test data is an abnormal value by comparing the reduced feature data of the test data with reduced feature data previously stored in the memory bank. In the present disclosure, the reduced feature data stored in the memory bank can include reduced feature data generated during training of the reduced feature extraction module. Alternatively, in the present disclosure, the memory bank can include only reduced feature data of normal data. In yet another embodiment, the memory bank can include all of the reduced feature data generated during training of the reduced feature extraction module. In another embodiment, the anomaly detection device in the present disclosure can generate a reduced feature data subset that best explains the entire reduced feature data generated during training and store it in the memory bank for computational efficiency and reduced inference time. This will be described in detail below with reference to FIG. 8.

[0041] The training data and test data according to the present disclosure are input data for the reduced feature extraction module and can be used to train the reduced feature extraction module or to verify whether test data is abnormal using the reduced feature extraction module. In the present disclosure, "training data" or "test data" can also refer to a training data set or a test data set, which are a collection of one or more training data or test data. In the present disclosure, one or more neural networks constituting the reduced feature extraction module may not be trained simultaneously. That is, the feature extraction network and the dimensionality reduction network may be trained using different training data at different times. The training data according to the present disclosure may include only normal data or may include modified data or dummy data generated using a data augmentation technique. This will be described in more detail below with reference to FIG. 4.

[0042] FIG. 1 is a conceptual diagram showing an overview of a system that realizes an anomaly detection method based on machine learning according to the present disclosure.

[0043] In one embodiment, the anomaly detection device 100 according to the present disclosure may be realized by hardware and software for performing an anomaly detection method. In this disclosure, the expression "training the anomaly detection device 100" may mean repeatedly inputting data into the anomaly detection device 100 and changing the internal hardware or software of the anomaly detection device 100 from an initial state to reach a preset level according to a pre-designed method. In another embodiment, the anomaly detection device 100 according to the present disclosure may receive a request from the client 200 and, in response to the received request, may be trained using data received from the external input device 310 or data pre-stored in the database 320. In yet another embodiment, the anomaly detection device 100 according to the present disclosure may receive a request from the client 200 and, in response to the received request, detect whether the data input from the external input device 310 or the data stored in the database 320 contains an anomaly, store the result in the database 320, and transmit the result to the client 200.

[0044] The anomaly detection device 100 according to the present disclosure may be a device capable of determining whether input data is normal or abnormal. In one embodiment, the anomaly detection device 100 may perform part or all of the anomaly detection method according to the present disclosure. In another embodiment, the anomaly detection device 100 may include part or all of dedicated hardware for performing the anomaly detection method according to the present disclosure. In yet another embodiment, the anomaly detection device 100 may be a general-purpose computing device, such as a commercial server computer, capable of executing software for performing the anomaly detection method according to the present disclosure. The above description is merely an example of the physical form of the anomaly detection device 100, and the method for realizing the anomaly detection device 100 should not be considered limited thereto.

[0045] In the present disclosure, the anomaly detection device 100 may perform an anomaly detection task in response to a request from a client 200. In one embodiment, the client 200 according to the present disclosure may be a device that can send an anomaly detection task execution request to the anomaly detection device 100 using the illustrated network, receive the execution result, and provide it to a user. Specifically, the client 200 may include a general-purpose computing device such as a desktop or laptop, a mobile device such as a smartphone or tablet, or the like, or may be a dedicated device designed to perform the anomaly detection method according to the present disclosure.

[0046] In the present disclosure, the external input device 310 may be a module that converts information outside the system into a format that can be transmitted to the anomaly detection device 100, and specifically may include an image sensor module including a camera module, an audio sensor module, an infrared sensor module, etc.

[0047] The database 320 according to the present disclosure may store data related to the anomaly detection device 100. In one embodiment, the database 320 may include training data for training the anomaly detection device 100, inspection data for detecting anomalies using the anomaly detection device 100, etc. In another embodiment, the database 320 may store anomaly detection results performed by the anomaly detection device 100. The database 320 may receive and store data transmitted from not only the anomaly detection device 100 but also external modules in the system, including the client 200 and the external input device 310, via a network. In yet another embodiment, the database 320 may store one or more software modules usable by the anomaly detection device 100.

[0048] FIG. 2 is a block diagram illustrating a machine learning-based anomaly detection device according to the present disclosure.

[0049] The input / output unit 300 according to the present disclosure may include an input unit 121 and a communication unit 122. Among the components, the communication unit may include one or more components that enable communication with an external device, and may include, for example, at least one of a wired communication module, a wireless communication module, and a short-range communication module.

[0050] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Digital Visual Interface (DVI), recommended standard 232 (RS-232), power line communication, or plain old telephone service (POTS).

[0051] The wireless communication module may include a Wi-Fi module, a WiBro (Wireless Broadband) module, and other wireless communication modules that support various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G.

[0052] The wireless communication module may include a wireless communication interface having an antenna and a transmitter for transmitting a mobile communication signal, and may further include a conversion module for modulating a digital control signal output from the controller via the wireless communication interface into an analog wireless signal under the control of the controller.

[0053] The wireless communication module may include a wireless communication interface having an antenna and a receiver for receiving a signal, and may further include a conversion module for demodulating an analog wireless signal received via the wireless communication interface into a digital control signal.

[0054] The short-range communication module is for short-range communication and can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0055] The input unit is for inputting image information (or signals), audio information (or signals), data, or information input by a user, and may include at least one of at least one camera, at least one microphone, and a user input unit. The audio data or image data collected by the input unit may be analyzed and processed according to a user's control command.

[0056] The camera processes image frames, such as still images or videos, acquired by the image sensor in a shooting mode. The processed image frames can be displayed on a display unit (or on the screen of the anomaly detection device or client of the present disclosure) or stored in a memory.

[0057] On the other hand, when there are multiple cameras, they can be arranged in a matrix structure, and thus, multiple image information having various angles or focuses can be input through the cameras in the matrix structure, and the cameras can also be arranged in a stereo structure to obtain left and right images to realize a three-dimensional stereoscopic image.

[0058] The interface unit (not shown) serves as a passageway for various types of external devices connected to the device. Such an interface unit may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with a SIM (Single Identification Module), an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. The device can perform appropriate control related to the external device connected to the interface unit.

[0059] The memory unit 131 can store data supporting various functions of the device and programs for the operation of the control unit, can store input / output data (e.g., music files, still images, videos, etc.), can store a plurality of application programs (or applications) run by the device, and data and commands for the operation of the device. At least some of these application programs can be downloaded from an external server via wireless communication.

[0060] The memory 121 may include at least one type of storage medium selected from the group consisting of flash memory, hard disk, solid state disk (SSD), silicon disk drive (SDD), micro multimedia card, card-type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk. The memory may also be a database separate from the device but connected by wire or wirelessly.

[0061] The control unit may be implemented as a memory that stores data for an algorithm or a program that reproduces the algorithm for controlling the operation of components within the device, and at least one processor that performs the above-described operation using the data stored in the memory. In this case, the memory and the processor may be implemented as separate chips. Alternatively, the memory and the processor may be implemented as a single chip. In the present disclosure, the control unit may include one or more processors, such as a first processor 111 and a second processor 112. The first processor 111 and the second processor 112 may be implemented as different chips. The first processor 111 and the second processor 112 may be different types of processors. As an example, the first processor 111 may be implemented as a CPU, and the second processor 112 may be implemented as a GPU, but this is not limited thereto.

[0062] In addition, the control unit can control any one or a combination of the components discussed above in order to realize various embodiments of the present disclosure, etc., described in Figures 2 to 9 below, on this device.

[0063] At least one component may be added or removed depending on the performance of the components shown in Fig. 2. Furthermore, it will be readily understood by those skilled in the art that the relative positions of the components may be changed depending on the performance or structure of the system.

[0064] Meanwhile, each component shown in FIG. 2 represents software and / or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0065] 3 is a diagram showing the overall flow of the anomaly detection method based on machine learning according to the present disclosure. In one embodiment, the anomaly detection method according to the present disclosure can be roughly divided into two parts. One part can include a step of generating a reduced feature extraction module and generating a memory bank based on reduced feature data generated from the reduced feature extraction module (S100 to S400). The other part can include a step of determining whether the test data is normal or not using the generated reduced feature extraction module and memory bank (S500).

[0066] The step of generating a reduced feature extraction module using the anomaly detection method may begin with the step of constructing a training dataset (S100). In one embodiment, the training dataset may include one or more homogenous data. For example, the training dataset may include one or more image data, audio data, text data, etc. Of course, the training dataset may also include heterogeneous data. However, for convenience of explanation, it is assumed below that the training dataset includes one or more homogenous data. In one embodiment, the step of constructing the training dataset (S100) may be implemented by including one or more normal data in the training dataset. In one embodiment, the training dataset may be constructed using only normal data. Furthermore, in another embodiment, the step of constructing the training dataset (S100) may be implemented by including one or more normal data and data obtained by partially modifying such normal data in the training dataset. Specifically, the partial modification applied to the normal data may generate abnormal samples from the normal data, i.e., negative samples. In yet another embodiment, the step of constructing a training dataset (S100) can be realized by including one or more normal data and dummy data unrelated to such normal data in the training dataset.

[0067] Labeling of data included in the training dataset is not necessarily required. That is, each training data may be unlabeled. If the training data is not labeled, the feature extraction network and the dimensionality reduction network may be trained using unsupervised learning or self-supervised learning. The steps of constructing the training dataset are described in detail in FIG. 4. As described above, when constructing the training dataset, even if abnormal data cannot be obtained, data for training the feature extraction network can be obtained, thereby overcoming the difficulty of obtaining training data. Furthermore, in the present disclosure, it is possible to not label the training data, thereby reducing the cost of data labeling.

[0068] The anomaly detection method according to the present disclosure may train a feature extraction network (S200) to generate a reduced feature extraction module. As described above in the present disclosure, the feature extraction network may be one of the neural networks constituting the reduced feature extraction module. In one embodiment, the feature extraction network may extract features from one or more training data input in step S100 to generate feature patch data. The feature extraction network may have one or more repeated layers performing predetermined operations. In one embodiment, one or more layers constituting the feature extraction network may include a convolution layer performing a convolution operation. In some embodiments of the present disclosure, the feature patch data may be one or more combinations of feature maps extracted from one or more layers of the feature extraction network. This will be described in detail below with reference to FIG. 6.

[0069] Various methods can be used to train the feature extraction network, such as supervised learning, unsupervised learning, and self-supervised learning. In one embodiment, the feature extraction network can be trained to determine the similarity between one or more input data (i.e., training data or test data). For example, the anomaly detection device 100 according to the present disclosure can train the feature extraction network by determining the similarity between one or more outputs of the feature extraction network for multiple training data.

[0070] In this disclosure, similarity may refer to a norm that can measure the similarity between two or more objects, and may be expressed, for example, by measuring the spatial distance between two or more objects (e.g., vectors). Specifically, various metric learning methods, such as contrastive learning, may be employed to train the feature extraction network. Specifically, such contrastive learning may apply weak transformations to train different pairs of data transformed so as not to fall outside the range of normal data, so that the distance between their outputs as the same class is close, while the distance between the outputs of abnormal data transformed from normal data and normal data is wide. Additionally, the feature extraction network may be trained to infer the type of transformation applied to an abnormal sample in a training dataset. For example, the feature extraction network may be trained to determine whether the transformation applied to the training data was a cutout, a rotation, or noise addition, using the output data of the feature extraction network.

[0071] As described above, when training a feature extraction network, the present invention can train the feature extraction network to suit the task while preventing the feature extraction network from overfitting to the data, compared to the prior art. Furthermore, since labeling of the training data is not required, the time and cost required for constructing a dataset can be reduced.

[0072] The dimension reduction network can be trained using feature patch data generated through the feature extraction network of the present disclosure. In one embodiment, the feature patch data for training the dimension reduction network may be generated solely from normal data. The dimension reduction network can generate reduced patch data by reducing the dimension of the received feature patch data (S300). The reduced feature data can be data that closely represents the feature patch data. In one embodiment, the dimension reduction network can have one or more sub-neural networks. Specifically, the dimension reduction network can be trained using one or more sub-neural networks that determine the similarity between two or more input data (i.e., feature patch networks). Furthermore, the dimension reduction network can be trained to infer the position of input feature patch data on the training data. When the dimension reduction network is trained to determine the similarity between two or more input data, loss of information contained in the feature patch data can be minimized. This will be described in detail below with reference to FIG. 7.

[0073] In the anomaly detection method according to the present disclosure, once the training of the feature extraction network (S200) and the training of the dimensionality reduction network are completed, a memory bank can be constructed using a reduced feature extraction module including the feature extraction network (S500). To construct the memory bank, reduced feature data of one or more data can be generated using the reduced feature extraction module, and the generated reduced feature data can be included in the memory bank. In one embodiment, the reduced feature data used to generate the memory bank can be generated using only normal data. In one embodiment, only a portion of the generated reduced feature data can be selected to generate the memory bank (S400). That is, one or more subsets that can well explain the entire set of reduced feature data generated using the reduced feature extraction module can be selected and stored in the memory bank. A detailed description of this will be provided below with reference to FIG. 8.

[0074] The anomaly detection method according to the present disclosure may include determining whether the test data is normal using the generated reduced feature extraction module and memory bank (S600). Specifically, the anomaly detection method according to the present disclosure may extract reduced feature data from the test data using the trained reduced feature extraction module, and select a reduced feature data subset from the memory bank that is closest to the reduced feature data of the test data. Thereafter, the anomaly detection method may determine whether the input data is normal by calculating an anomaly score between the closest reduced feature data subset and the reduced feature data of the input data. This will be described in detail below with reference to FIG. 9.

[0075] According to the above-described means for solving the problems of the present disclosure, abnormal data can be detected using only normal data. Therefore, an effective learning process can be constructed while reducing data construction costs. Furthermore, even using only normal data, a feature extraction network adapted to the data type can be trained, and more accurate abnormal value detection results can be expected compared to conventional techniques. In addition, the outlier detection module takes into account more diverse information than conventional techniques, and more accurate abnormal value detection results can be expected compared to conventional techniques.

[0076] 4 is a conceptual diagram illustrating one embodiment of a method for generating one or more training data sets according to the present disclosure. In the present disclosure, a training data set 400 may include only one or more normal data sets 400a, or may include the normal data sets 400a and their modified data sets as training data. For convenience, in the present disclosure, only the normal data sets 400a are referred to as normal data. However, whether data sets are normal or not should be determined based on the degree of modification, and therefore should not be limited to the disclosure of FIG. 4.

[0077] In one embodiment, the training data may be generated by performing a predetermined operation on one or more normal data 400a. For example, the training data may include normal data, data obtained by modifying the normal data, or dummy data. For example, the modification applied to the normal data may generate abnormal data. However, modification of normal data does not necessarily result in abnormal data. In the present disclosure, data obtained by modifying normal data may still be normal data. Furthermore, the present disclosure addresses both supervised and unsupervised learning. Here, abnormal data refers to data that has been modified and labeled as abnormal data, or data that can be classified into a different subset from normal data as a result of training an anomaly detection module. As a specific example, abnormal data may refer to modified data that belongs to a different cluster from normal data as a result of unsupervised learning.

[0078] Additionally, in the present disclosure, dummy data may be data unrelated to the normal data. As shown in FIG. 4, if the normal data is a puppy image 400a, the dummy data may be cat data, turtle data, box data, etc. In the present disclosure, a feature extraction network or a dimensionality reduction network may be based on a training dataset constructed by adding one or more dummy data, such as those described above, to one or more normal datasets. Specifically, training of a feature extraction network using dummy data according to the present disclosure may involve learning a problem of classifying a non-defective dataset into one class and dummy data into another class. In one embodiment, training of a feature extraction network using dummy data may utilize contrastive learning. In this case, images belonging to the same class may be classified into positive pairs, and images belonging to different classes may be classified into negative pairs, and learning may be performed using a contrastive loss function. In yet another embodiment, a feature extraction network trained using a dummy dataset may be trained to infer which class the dummy data was added to by adding a classification layer. As shown in FIG. 4, in this disclosure, the transformation method for creating abnormal data may be crop and resize, cut out, etc., but as mentioned above, abnormal data should not be limited to meaning transformed data generated by the crop and resize or cut out method.

[0079] FIG. 5 is an explanatory diagram showing one embodiment of a feature extraction network 500 according to the present disclosure. The feature extraction network 500 according to the present disclosure may include one or more feature map generation layers 500a and 500b. The one or more feature map generation layers may include a convolution layer that performs a convolution operation and a pooling layer that performs a pooling operation on the result of the convolution operation. In one embodiment, the feature map generation layer 500a and the feature map generation layer 500b may have different parameters. Therefore, the feature maps generated by the feature map generation layer 500a and the feature map generation layer 500b may have different resolutions.

[0080] Each of the feature map generation layers 500a and 500b of the feature extraction network 500 according to the present disclosure can generate at least one feature map, preferably one feature map each. Furthermore, in the anomaly detection method according to the present disclosure, feature patch data can be generated using the feature map data of each of the feature map generation layers 500a and 500b included in the feature extraction network 500. In one embodiment, the feature patch data can be composed of feature map data from one feature map generation layer. In another embodiment, the feature patch data can be generated by combining feature map data from two or more feature map generation layers. Specifically, the combination of feature map data can be performed by concatenation of feature map data. In some embodiments of the present disclosure, the feature map data used to generate the feature patch data can include a feature map generated from a high-abstraction layer (i.e., a task of summarizing key content or functions from a large amount of data or complex material). A high-abstraction layer can refer to a layer that generates a feature map that contains key information contained in data rather than the specific features of the image itself. In other words, a high-abstraction layer may refer to a layer in which the feature map generated from that layer contains little information about the specific features of the image itself, or contains more core information of the data than the specific features of the image itself. In some embodiments, a high-abstraction layer may refer to four or more feature map generation layers. In yet other embodiments, the feature map data for generating the feature patch data may include not only feature map data from the high-abstraction feature map generation layer, but also lower-abstraction feature maps generated from one or more lower-abstraction feature map generation layers. Furthermore, the feature map data for generating the feature patch data may include feature maps from all of the feature map generation layers.

[0081] In the prior art, a general feature extraction network such as ImageNet was used to train a feature extraction network without using data suited to the task. Therefore, since a highly abstracted feature map extracts features biased toward ImageNet rather than extracting features suited to the task, there was a problem in that highly abstracted feature map data could not be used to solve the task. Considering that data at a high abstraction level generally plays an important role in anomaly detection problems in performing anomaly detection on an overall data basis, by utilizing a highly abstracted feature map in the present disclosure, it is expected that performance in anomaly detection problems can be improved.

[0082] FIG. 6 is a conceptual diagram illustrating one embodiment of one or more feature patch data 600b according to the present disclosure. In the present disclosure, the feature map concatenated data 600a may be data obtained by concatenating one or more feature maps generated in one or more feature map generation layers. Furthermore, in the present disclosure, the feature patch data 600b may be a feature slice including feature values ​​of a specific point in the feature map concatenated data 600a and its surrounding areas, and may be a part of the feature map concatenated data 600a. The feature map concatenated data 600a and the feature patch data 600b have an inclusive relationship. Therefore, the following description of the feature patch data can be applied to the feature map concatenated data 600a as well. Therefore, for convenience of explanation, the following description will be based on the feature patch data 600b.

[0083] In the present disclosure, the feature map concatenation data 600a and the feature patch data 600 may be expressed as a three-dimensional tensor. For example, the feature patch data 600 may be generated from one or more feature maps generated from one feature map generation layer. In this case, the feature patch data 600 may be a three-dimensional tensor having the width, height, and depth of the feature map, which is the number of channels of the feature map. In another example, if the feature patch data 600 is generated from feature maps in two or more feature map generation layers, the depth of the feature patch data may be doubled by the number of feature map generation layers. In this case, the width and height values ​​of the feature maps generated from different feature map generation layers may be different. In this case, the anomaly detection device according to the present disclosure may adjust the resolutions of the feature maps in different generation layers to match by enlarging a relatively small feature map or shrinking a relatively large feature map using interpolation. When the feature patch data is generated using feature map data from two or more different layers, a dimension reduction network can simultaneously learn high-level abstraction features from low-level abstraction features, thereby improving the learning effect.

[0084] 7 is a diagram illustrating one embodiment of a dimension reduction network 700 according to the present disclosure. The dimension reduction network 700 according to the present disclosure can be trained by training one or more sub-networks. As described above, the feature patch data for training the dimension reduction network 700 can include only data generated from normal data. In one embodiment, the dimension reduction network can be trained as a whole by individually training one or more of a first embedding network 700a, a second embedding network 700b, a third embedding network 700c, and a location inference network 700d.

[0085] In one embodiment, supervised learning, unsupervised learning, and self-supervised learning can be used to train the dimension reduction network 700. In particular, the dimension reduction network 700 can be trained to learn the similarity between two or more data when training one or more sub-networks, and specifically, various metric learning methods can be used. In one embodiment, the first embedding network 700a, the second embedding network 700b, and the third embedding network 700c can learn the similarity between two or more input data. In some embodiments of the present disclosure, the first embedding network 700a and the second embedding network 700b can have the same structure, and the training results or parameter changes of the second embedding network 700b can affect the first embedding network 700a. In particular, in the present disclosure, the first embedding network 700a, the second embedding network 700b, or the third embedding network 700c can generate reduced feature data by reducing the dimension of feature patch data and embedding it. Hereinafter, without loss of generality, data embedded by the first embedding network 700a, the second embedding network 700b, or the third embedding network 700c will be referred to as embedding data or embedding vector, and such embedding data may refer to reduced feature data whose dimension is reduced compared to the feature patch data input by the first embedding network 700a, the second embedding network 700b, or the third embedding network 700c.

[0086] As a specific example, the first embedding network 700a may calculate the similarity between embeddings of two or more input data by combining pairwise similarity and contextual similarity. Pairwise similarity considers the similarity between two embedding vectors in the embedding space, while contextual similarity considers the degree of overlap between nearby embedding vectors of the two embedding vectors. In a specific example, the combination between the pairwise similarity map and contextual similarity may refer to the average between the two similarities. In unsupervised learning, it is known that pairwise similarity may produce inaccurate results. Therefore, in addition to considering the similarity between two embedding vectors through pairwise similarity, similarity learning through pairwise similarity can be supplemented by considering the degree of overlap between neighboring vectors of the two embedding vectors.

[0087] In one embodiment according to the present disclosure, the second embedding network 700b and the third embedding network 700c may derive the similarity between two embedding vectors using Euclidean distance. In one embodiment, the second embedding network 700b and the third embedding network 700c may calculate a relaxed contrastive loss, which constitutes part of the loss function of the dimension reduction network 700, based on the derived Euclidean distance and the similarity value generated from the first embedding network 700a. The relaxed contrastive loss may train the dimension reduction network by shortening the distance between the two embedding vectors when the two embedding vectors embedded by the second embedding network 700b and the third embedding network 700c are similar, and by increasing the distance between the two embedding vectors when the two embedding vectors are dissimilar, but by not increasing the distance between the two embedding vectors by more than a predetermined distance even when the two embedding vectors are dissimilar.

[0088] In yet another embodiment, based on the Euclidean distance derived by the second embedding network 700b and the Euclidean distance derived by the third embedding network 700c, the Kullback-Leibler divergence between the second embedding network 700b and the third embedding network 700c can be used as part of the loss function for training the dimensionality reduction network 700. The Kullback-Leibler divergence represents the difference when the third embedding network 700c is used instead of the second embedding network 700b to represent input feature patch data, and can serve to train the third embedding network 700c to embed reduced feature data in a manner similar to the second embedding network 700b.

[0089] The dimension reduction network 700 may further include a location inference network 700d. In some embodiments of the present disclosure, the location inference network 700d may receive embedding data of feature patch data as input and be trained to infer the location of the feature patch data or embedding data in the input image data (i.e., data input to the feature extraction network) or feature map. In one embodiment, the location inference network 700d may receive embedding data of the third embedding network 700c as input and be trained to infer the location of the feature patch data in the input image data or feature map. In yet other embodiments, the location inference network 700d may utilize the embedding data of the first embedding network 700a or the second embedding network 700b. In this case, a loss function obtained by measuring the mean squared error (MSE) between the position of the reduced patch data inferred by the position inference network 700d and the actual position of the reduced patch data or the distance between the two positions, normalizing it with a Gaussian kernel, and converting it to a negative log scale can be used as part or all of the loss function for training the dimension reduction network 700. By using this loss function, the dimension reduction network 700 can be trained to store position information in the feature map of the reduced patch data when embedding the reduced patch data into reduced feature data. In one embodiment, the loss function for training the dimension reduction network 700 can be configured by using part or all of a relaxed contrast loss, Kullback-Leibler divergence, and the MSE error or the negative log scale conversion loss normalized with a Gaussian kernel.

[0090] Generally, when a pooling technique is used to reduce the dimension of input data, various information contained in the input data may be lost. As in the present disclosure, by training the input data using a metric learning technique and simultaneously reducing the dimension of the data while adding the task of inferring the location of the input data to the dimension reduction network, the loss of location information of the input data can be minimized. In an anomaly detection problem, if location information can be embedded in the embedding data of the input data as in the present disclosure, the location of an anomaly can be more effectively inferred. In particular, as described above, when the location inference network 700d is used, the dimension reduction network 700 can be trained so that the embedding data for the feature patch data of the first embedding network 700a and the second embedding network 700b or the embedding data for the feature patch data of the third embedding network 700c implies location information in the input data feature map of the input patch data.

[0091] In one embodiment, once the training of the dimension reduction network 700 is complete, only a portion of the dimension reduction network can be used to generate a memory bank or to infer whether or not the inspection data is abnormal. In one embodiment, after the training of the dimension reduction network 700 is complete, only the third embedding network 700c can be used to generate (embedding) reduced feature data. That is, in this case, the dimension reduction network 700 can generate reduced feature data from feature patch data using only the third embedding network 700c. Therefore, the amount of calculation of the dimension reduction network 700 can be reduced, and an improvement in calculation speed can be expected.

[0092] 8 is a conceptual diagram illustrating one embodiment of a memory bank generation method according to the present disclosure. In this disclosure, a memory bank may refer to a data set including one or more feature patch data or one or more reduced feature data. In one embodiment, the reduced feature data stored in the memory bank may include only data starting from normal data.

[0093] In one embodiment, the memory bank may contain one or more reduced feature data generated from the dimension reduction network. In another embodiment, the memory bank may classify a portion of the reduced feature data generated from the dimension reduction network into one or more subsets and store the resulting data in the memory bank. Here, the one or more reduced feature data classified into one or more subsets can well explain the overall reduced feature data generated from the dimension reduction network. Specifically, being able to well explain the overall reduced feature data may mean that the distribution of the reduced feature data classified into one or more subsets is similar to that of the overall reduced feature data, or that the accuracy of an anomaly detection task using test data is not significantly different when the memory bank is composed of one or more subset data compared to when the memory bank is composed of the overall reduced feature data. Specifically, classifying the portion of the reduced feature data into one or more subsets may involve selecting reduced feature data included in each of the subsets using a minimax algorithm.

[0094] In this way, when only a portion of the total reduced feature data is used in the memory bank, the amount of calculation required for anomaly detection using test data can be significantly reduced, which has the advantage of significantly shortening the inference time.

[0095] 9 is a flowchart illustrating an embodiment of a method for determining whether inspection data is normal according to the present disclosure. In the present disclosure, determining whether inspection data is normal using a generated reduced feature extraction module may be performed by capturing an image of a product with a vision inspection camera (S601), extracting feature patch data for the inspection data using a trained feature extraction network in the reduced feature extraction module (S602), inputting the generated feature patch data into a dimension reduction network in the reduced feature extraction module to generate reduced feature data for the inspection data (S603), detecting a nearest subset from among the generated reduced feature data and one or more subsets pre-stored in a memory bank (S604), calculating an anomaly score using the nearest subset and the reduced feature data (S605), and determining whether the anomaly score is within a predetermined criterion (i.e., a threshold) (S606), thereby detecting whether the inspection data is abnormal (S606 and S607). In one embodiment, detecting the closest subset from among the reduced feature data and one or more subsets may be performed using a k-Nearest Neighbor (k-NN) algorithm. Once the closest subset is extracted, the maximum distance between the reduced feature data included in the subset and the reduced feature data of the inspection data may be determined as the number of anomalies for detecting whether or not there is an anomaly. Since defects generally occur in a portion of a product rather than the entire product, it may be more effective to measure defects based on the maximum distance. In another embodiment, when determining the number of anomalies, the maximum distance between the reduced feature data included in the subset and the reduced feature data of the inspection data may be normalized using distance information from neighboring subsets. Normalizing the number of anomalies may correct for differences in distance between subsets, thereby enabling more accurate measurement of the number of anomalies.

[0096] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0097] Computer-readable recording media include all types of recording media that store instructions that can be decoded by a computer, such as ROM (Read Only Memory), RAM (Random Access Memory), magnetic tape, magnetic disk, flash memory, and optical data storage devices.

[0098] The disclosed embodiments have been described above with reference to the accompanying drawings. Those skilled in the art will understand that the present disclosure may be embodied in forms different from the disclosed embodiments without changing the technical concept or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting. [Industrial Applicability]

[0099] The present invention has industrial applicability because it provides a method for a computing device including an anomaly detection module to train the anomaly detection module to detect abnormalities and normalities in a manufacturing process.

Claims

1. 1. A computing device for performing anomaly detection, comprising: a memory including at least one memory bank that is a logical region; a processor configured to train a neural network using at least one functional module, generating a reduced feature extraction module, generating the memory bank based on reduced feature data generated from the reduced feature extraction module, and determining whether the inspection data is normal or not using the generated reduced feature extraction module and the memory bank; Equipped with The processor: The operation of generating the reduced feature extraction module includes: training a feature extraction network using one or more first training data; generating one or more feature patch data from one or more second training data using the trained feature extraction network; training a dimensionality reduction network using the one or more feature patch data; A computing device that utilizes the trained dimensionality reduction network to generate the reduced feature data from the feature patch data.

2. The computing device of claim 1 , wherein the feature extraction network is trained to determine distance or similarity between two or more data.

3. The one or more first training data The computing device of claim 2 , further comprising modified or dummy data generated based on one or more normal data.

4. The computing device of claim 3 , wherein the feature extraction network is additionally trained to infer the manner in which the transformed data was transformed or the class of the dummy data.

5. the feature extraction network comprises one or more feature map generation layers; the one or more feature map generation layers generate at least one feature map; the feature patch data is generated from the at least one feature map; The computing device of claim 1 , wherein the at least one feature map includes at least one or more highly abstracted feature maps.

6. The at least one feature map comprises: The computing device of claim 5 , including feature maps at all feature map generation layers of the feature extraction network.

7. The one or more second training data 6. The computing device of claim 5, including only normal data.

8. The dimensionality reduction network The computing device of claim 1 , wherein the computing device is trained by learning distances or similarities between the one or more feature patch data.

9. The dimensionality reduction network The computing device of claim 8 , further trained to infer position information of the feature patch data in the second training data or position information in a feature map generated by the feature extraction network based on the reduced feature data.

10. The dimensionality reduction network The computing device of claim 8 , wherein the reduced feature data implies position information of the feature patch data in the second training data or position information in a feature map generated by the feature extraction network.

11. Generating a memory bank from the reduced feature data includes: The computing device of claim 1 , further comprising classifying one or more of the reduced feature data into one or more subsets.

12. The memory bank The computing device of claim 11 , including only reduced feature data for normal data.

13. Determining whether the inspection data is normal or not using the generated reduced feature extraction module is generating reduced feature data for the inspection data using the reduced feature extraction module; using the reduced feature data to identify a closest subset of reduced feature data to the test data from one or more subsets contained in the memory bank; Calculating an anomaly score using the nearest neighbor subset and the reduced feature data; The computing device according to claim 1 , wherein the presence or absence of an abnormality in the inspection data is detected based on the number of abnormal points.

14. 1. A machine learning based anomaly detection method performed by a computing device, comprising: generating a reduced feature extraction module; generating a memory bank based on the reduced feature data generated from the reduced feature extraction module; determining whether the test data is normal or not using the generated reduced feature extraction module and memory bank; Including, The step of generating the reduced feature extraction module comprises: training a feature extraction network using one or more first training data; generating one or more feature patch data from one or more second training data using the trained feature extraction network; generating one or more of the reduced feature data from the one or more feature patch data using a dimension reduction network; 1. A machine learning-based anomaly detection method comprising:

15. A computer program stored on a computer-readable storage medium, the computer program causing a processor to perform the following operations for performing anomaly detection, the operations comprising: generating a reduced feature extraction module; generating a memory bank based on the reduced feature data generated from the reduced feature extraction module; and The method includes determining whether the test data is normal or not using the generated reduced feature extraction module and memory bank; The operation of generating the reduced feature extraction module includes: training a feature extraction network using one or more first training data; generating one or more feature patch data from one or more second training data using the trained feature extraction network; and A computer program stored on a computer-readable storage medium, comprising: generating one or more of the reduced feature data from the one or more feature patch data using a dimensionality reduction network.

Images