Anomaly detection method and device, and learning method thereof

Abstract

A learning method performed by an anomaly detection device according to an embodiment comprises the steps of: obtaining a target feature from a pre-trained network by inputting an image into the pre-trained network; obtaining a local feature by sampling a normal feature on the basis of a local grid obtained by reducing the dimension of a local area of the target feature; obtaining a global feature by sampling the normal feature on the basis of a global grid obtained by reducing the dimension of the entire target feature; and updating the pre-trained network on the basis of a result of combining the local feature and the global feature.

Description

Anomaly detection method and device, and its learning method

[0001] The present invention relates to an anomaly detection method, a device for performing the same, and a learning method performed by the anomaly detection device.

[0002] This study is related to the research project "Development of Design Technology for a Deep Learning-Based Bidirectional Collaborative Neuromorphic Image Sensor System" (No. 2410000237), which was supported by the Sungkyunkwan University Industry-Academic Cooperation Foundation and funded by the Ministry of Trade, Industry and Energy (Government) in 2024.

[0003] This study was conducted in connection with the Artificial Intelligence Graduate School Support Project (No. 2710008628) supported by the Sungkyunkwan University Industry-Academic Cooperation Foundation and funded by the Ministry of Science and ICT (Government) in 2024.

[0004] This study is related to the ICT Talent Development Project (NO. 2710007880) supported by the Sungkyunkwan University Industry-Academic Cooperation Foundation and funded by the Ministry of Science and ICT (Government) in 2024.

[0005] This study is related to the research project "Development of Core Technology and Human Resources Training of Artificial Intelligence System Semiconductors for Smart Mobility" (No. 2710008712), which was conducted with the support of the Inha University Industry-Academic Cooperation Foundation and funding from the Ministry of Science and ICT (Government) in 2024.

[0006] This study is related to the Artificial Intelligence Innovation Hub Research and Development project (NO. 2710007402) supported by the Korea University Industry-Academic Cooperation Foundation and funded by the Ministry of Science and ICT (Government) in 2024.

[0007] This study is related to the research project Development of a Deepfake Detection Enhancement, Creation Suppression, and Distribution Prevention Platform for Responding to Malicious Altered Content (NO. 2710008048), which was conducted with funding from the Ministry of Science and ICT (Government) in 2024 and support from the Sungkyunkwan University Industry-Academic Cooperation Foundation.

[0008] This study is related to the 3D Cognitive Neural Codec (NO. 2710004315) project, which was supported by the Sungkyunkwan University Industry-Academic Cooperation Foundation with funding from the Ministry of Science and ICT (Government) in 2024.

[0009] For reference, this application claims priority to Korean Patent Application No. 10-2023-0195695, filed on December 28, 2023. The entire contents of that application, which serves as the basis for this priority claim, are incorporated herein by reference.

[0010] Conventional techniques train separate models for each type of object to be detected for anomaly detection. However, this presents significant challenges in learning the unique distribution of each object within a single model. Separate models must be trained for each object type, with the training stage structured so that each model learns only a single distribution. This approach, in practical situations, can be costly, requiring repeated training and management of multiple models.

[0011] Furthermore, conventional anomaly detection suffers from weak or overgeneralization in discrete feature spaces. For example, the input can be reconstructed by referencing a single feature in the feature space that is most similar to the input feature, or by referencing all learned features in the feature space. The former method, which refers to a single feature, can misdetect abnormalities when unseen normal features are input during the training phase. The latter method, on the other hand, can fail to detect anomalies by reconstructing the input using the weighted sum of all features in the feature space, even including anomalies within the input.

[0012] (Prior art literature)

[0013] (Patent Document 1) Republic of Korea Patent Publication No. 2545772, published on August 31, 2022.

[0014] According to one embodiment, there are provided an anomaly detection method for updating a training network for anomaly detection through local feature sampling and global feature sampling in a continuous feature space, a device for performing the method, and a learning method performed by the anomaly detection device based on the results learned in this manner.

[0015] The problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0016] A learning method performed by an anomaly detection device according to a first aspect includes: inputting an image into a pre-training network to obtain target features from the pre-training network; sampling normal features based on a local grid obtained through dimension reduction in a local region of the target features to obtain local features; sampling normal features based on a global grid obtained through dimension reduction in the entirety of the target features to obtain global features; and updating the pre-training network based on a result of combining the local features and the global features.

[0017] The computer program of the non-transitory computer-readable recording medium storing the computer program according to the second viewpoint includes instructions for causing the processor to perform the learning method when executed by the processor.

[0018] An anomaly detection device according to a third aspect includes a memory storing at least one command; and a processor, wherein the processor performs anomaly detection on the image based on a result of combining local features and global features sampled from target features acquired from an image through a pre-trained network by executing the at least one command, wherein the pre-trained network is learned by a learning method of inputting the image to the pre-trained network to acquire the target features from the pre-trained network, sampling normal features based on a local grid obtained by dimension reduction for a local region of the target features to acquire the local features, sampling normal features based on a global grid obtained by dimension reduction for the entirety of the target features to acquire the global features, and updating the pre-trained network based on a result of combining the local features and the global features.

[0019] According to a fourth aspect, an anomaly detection method performed by an anomaly detection device is learned by a learning method including: performing anomaly detection on an image based on a result of combining local features and global features sampled from target features acquired from an image through a pre-trained network; and wherein the pre-trained network comprises: a step of inputting the image to the pre-trained network to acquire the target features from the pre-trained network; a step of sampling normal features based on a local grid obtained by dimension reduction for a local region of the target features to acquire the local features; a step of sampling normal features based on a global grid obtained by dimension reduction for the entirety of the target features to acquire the global features; and a step of updating the pre-trained network based on a result of combining the local features and the global features.

[0020] The computer program of the non-transitory computer-readable recording medium storing the computer program according to the fifth viewpoint includes instructions for causing the processor to perform the above-described anomaly detection method when executed by the processor.

[0021] The computer program of the non-transitory computer-readable recording medium storing the computer program according to the second viewpoint includes instructions for causing the processor to perform the anomaly detection method when executed by the processor.

[0022] In one embodiment, a training network for anomaly detection is learned by updating the training network through local feature sampling and global feature sampling in a continuous feature space, and anomaly detection is performed based on the results learned in this way.

[0023] This minimizes the cost incurred in realistic applications by enabling learning of normal features for all types of detected objects with a single model, and solves the problem of weak generalization or overgeneralization when using an ideal feature space by storing normal features in a continuous feature space and flexibly referencing only features similar to the input, thereby improving performance.

[0024] FIG. 1 is a configuration diagram of a learning method for anomaly detection and an anomaly detection device capable of performing the anomaly detection method according to one embodiment of the present invention.

[0025] FIG. 2 is a flowchart for explaining a learning method and anomaly detection method performed by an anomaly detection device according to one embodiment of the present invention.

[0026] FIG. 3 is a conceptual diagram for explaining a learning method and anomaly detection method performed by an anomaly detection device according to one embodiment of the present invention.

[0027] Figure 4 is a conceptual diagram illustrating a sampling process performed during a learning method according to one embodiment of the present invention.

[0028] FIG. 5 and FIG. 6 are conceptual diagrams for explaining a jittering process performed during a learning method according to one embodiment of the present invention.

[0029] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

[0030] The terms used in this specification will be briefly explained, and the present invention will be described in detail.

[0031] The terms used in this invention have been selected from widely used, current terms, taking into account the functions of the invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the invention.

[0032] When a part of a specification is said to 'include' a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

[0033] Also, the term 'part' used in the specification means a software or hardware component such as an FPGA or ASIC, and the 'part' performs certain functions. However, the 'part' is not limited to software or hardware. The 'part' may be configured to reside on an addressable storage medium or may be configured to play one or more processors. Thus, as an example, the 'part' includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

[0034] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily practice them. Furthermore, in order to clearly explain the present invention, portions irrelevant to the description are omitted in the drawings.

[0035] FIG. 1 is a configuration diagram of an anomaly detection device (100) capable of performing a learning method for anomaly detection and an anomaly detection method according to one embodiment of the present invention.

[0036] Referring to FIG. 1, an anomaly detection device (100) according to one embodiment includes a memory (110) and a processor (120) in which an anomaly detection model (111) is stored and mounted, and may further include an input unit (130) and / or an output unit (140).

[0037] The memory (110) is equipped with an anomaly detection model (111) including instructions that can be executed by the processor (120), and can further store various types of information for executing the anomaly detection model (111). The anomaly detection model (111) can enable the processor (120) to perform a learning method and / or anomaly detection method for anomaly detection according to one embodiment by executing instructions by the processor (120).

[0038] The processor (120) can load an anomaly detection model (111) by executing instructions stored in the memory (110), and perform learning and anomaly detection for anomaly detection according to the anomaly detection model (111). This processor (120) can be composed of one or more processors. For example, one or more processors can be a general-purpose processor such as a central processing unit (CPU), a digital signal processor (DSP), a graphics-only processor such as a graphics processing unit (GPU), a vision processing unit (VPU), or an artificial intelligence-only processor such as a neural processing unit (NPU).

[0039] When learning for anomaly detection by an anomaly detection model (111), the processor (120) inputs an image into a pre-training network to obtain target features from the pre-training network, samples normal features based on a local grid obtained through dimension reduction for a local region of the target features to obtain local features, samples normal features based on a global grid obtained through dimension reduction for the entire target features to obtain global features, and updates the pre-training network based on a result of combining the local features and the global features. In addition, the processor (120) can perform anomaly detection on an input image based on a result of combining the local features and the global features sampled from the target features obtained through the pre-training network from the image during anomaly detection by the learned anomaly detection model (111).

[0040] In learning and detecting such anomalies, the processor (120) can acquire target features by combining intermediate layer feature maps of the pre-trained network. When acquiring local features, the processor (120) can vectorize each pixel of the target features and use it as coordinates. In addition, the processor (120) can add noise to the coordinate values ​​before performing sampling. In addition, when acquiring global features, the processor (120) can vectorize the entire target features and use it as coordinates. In addition, when updating the pre-trained network, the processor (120) can minimize the difference between the result of combining the local and global features and the reconstructed features generated based on the target features and the re-input features. In addition, when updating the pre-trained network, the processor (120) can calculate the difference between the reconstructed features and the re-input features, and replace at least some areas of the reconstructed features that show a difference less than a preset threshold with the corresponding areas of the re-input features.

[0041] The input unit (130) can receive various information required for the processor (120) to execute the anomaly detection model (111). Such information can be input in real time, or, if input in advance, can be stored in the memory (110). For example, the input unit (130) can receive training images and anomaly detection target images.

[0042] The output unit (140) can output various processing data generated as the processor (120) executes the anomaly detection model (111). For example, the output unit (140) can include a data interface capable of outputting various processing data to external peripheral devices, a communication module capable of transmitting various processing data through a communication channel, etc.

[0043] FIG. 2 is a flowchart for explaining a learning method and anomaly detection method performed by an anomaly detection device according to an embodiment of the present invention, FIG. 3 is a conceptual diagram for explaining a learning method and anomaly detection method performed by an anomaly detection device according to an embodiment of the present invention, FIG. 4 is a conceptual diagram showing a sampling process performed during a learning method according to an embodiment of the present invention, and FIGS. 5 and 6 are conceptual diagrams for explaining a jittering process performed during a learning method according to an embodiment of the present invention.

[0044] Hereinafter, with reference to FIGS. 1 to 6, the learning method and the abnormality detection method performed by the abnormality detection device according to one embodiment of the present invention will be examined in detail.

[0045] First, the processor (120) of the anomaly detection device (100) loads the anomaly detection model (111) by executing a command stored in the memory (110), and starts learning for anomaly detection according to the anomaly detection model (111).

[0046] In learning for such anomaly detection, an image is input into a pre-trained network (301) to obtain target features (303). At this time, the intermediate layer feature maps of the pre-trained network (301) can be combined (302) to obtain the target features (303) (S210). Since the pre-trained network (301) is a model tuned to perform well the dataset used during pre-training and the purpose of the pre-training stage, feature maps of layers that are too deep contain a large amount of bias learned during pre-training and are therefore unsuitable for use in anomaly detection. Therefore, the intermediate layer feature maps of the pre-trained network (301) are extracted and combined to obtain target features (303) that are the target of learning. These target features (303) are used in all subsequent stages, and serve as the target of training as well as the target of reconstruction during the training stage.

[0047] Then, the local features are obtained by sampling (304) the normal features based on the local grid obtained by dimension reduction in the local area of ​​the target feature (303). Here, when obtaining the local features, each pixel of the target feature can be vectorized and used as coordinates (S220).

[0048] In a grid structure, sampling is performed using interpolation when a grid and coordinate values ​​are given. For example, in a one-dimensional grid such as Fig. 4, if a vertical line-shaped grid and an input coordinate value of 0.25 are given, the result of interpolating the data stored at 0 on the grid and the data stored at 0.5 is sampled. This can be extended to 2D and 3D grids in the future, and the data stored in the grid can also store various types of data such as scalars, vectors, and tensors. Since the grid structure allows sampling even between values ​​stored in the data, the normal features stored in the grid can be considered to exist in a continuous space.

[0049] To sample local features from a grid, each local region of the extracted feature map must be converted into coordinates. Therefore, a dimensionality reduction method is used to reduce the dimension of each region of the extracted feature map to match the grid dimension. For example, when using a 1D grid, each region is reduced to a 1D scalar value, and when using a 3D grid, each region is reduced to a 3D vector. For example, methods that can be used for dimensionality reduction include methods that utilize convolutional networks, global average pooling (GAP), and multi-layer perceptrons (MLP). Afterwards, the normal features for each pixel in the grid are sampled using the converted coordinate values ​​to obtain local features.

[0050] Here, the processor (120) can add noise to the coordinate values ​​before performing sampling. Before performing sampling on the grid using vectorized coordinates, jittering can be performed to add noise to the coordinate values ​​in order to update a wider grid area with a single value. Fig. 5 shows a case where jittering is not performed, and Fig. 6 shows a case where jittering is performed. By adding noise to the coordinate values, even if the input values ​​are the same, the added noise changes for each training interaction (iteration), so that grid values ​​that were not previously involved can be updated, which improves generalization performance.

[0051] Next, the processor (120) obtains global features by sampling normal features (305) based on the global grid obtained by dimension reduction of the entire target features (303). Here, when obtaining global features, the processor (120) can vectorize the entire target features and use them as coordinates (S230).

[0052] Thereafter, the processor (120) performs a process (306) of combining the local features obtained in step S220 and the global features obtained in step S230, thereby obtaining a combined result (307) of the local features and global features to be used for updating the pre-trained network (301). For example, concatenation and convolution layers can be used as a combining method.

[0053] And, the processor (120) updates the pre-trained network (301) through feature refinement using the combined result (307) of the local and global features. Here, the difference between the reconstructed feature (308) generated based on the combined result (307) of the local and global features and the target feature (303) and the re-input feature (303) can be minimized. And, the processor (120) can calculate the difference between the reconstructed feature (308) and the re-input feature (303), and replace at least some area of ​​the reconstructed feature (308) showing a difference less than a preset threshold with the corresponding area of ​​the re-input feature (303). At this time, the difference measurement result (309) and the 1-difference measurement result (310) can be used as a mask (S240).

[0054] After performing the learning process of steps S210 to S240, the anomaly detection device (100) can perform anomaly detection on the input image. The processor (120) outputs the final anomaly detection result (311) by identifying the difference between the reconstructed feature (308) and the input feature (303) when detecting anomalies using the learned anomaly detection model (111). Here, a mean squared error, etc. can be used as a method for measuring the difference (S250).

[0055] Table 1 below shows the quantitative evaluation results for the industrial anomaly detection benchmark dataset (MVTec AD).

[0056] Training a separate model for each object type Training a single model DetectLocalizeDetectLocalizeUS87.793.974.581.8PaDiM95.597.484.289.5MKD87.890.781.984.9DRAEM98.097.388.187.2UniAD96.696.696.596.8PatchCore99.098.197.697.1The present invention99.497.999.397.8

[0057] In Table 1, each performance was evaluated using the Area Under the Curve (AUROC) score, with higher scores indicating higher performance. As shown in Table 1, the method proposed in this invention outperforms other conventional techniques in both "individual model training" and "single model training."

[0058] As described above, according to one embodiment of the present invention, a training network for anomaly detection is updated through local and global feature sampling in a continuous feature space, and anomaly detection is performed based on the results learned in this way. This enables learning of normal features for all types of detected objects with a single model, thereby minimizing costs in practical applications. In addition, by storing normal features in a continuous feature space, only features similar to the input are dynamically referenced, thereby resolving the problem of weak or overgeneralization when using an ideal feature space, and consequently improving performance.

[0059] Meanwhile, each step included in the learning method performed by the anomaly detection device according to the above-described embodiment can be implemented as a computer program recorded on a recording medium including commands for causing a processor to perform these steps.

[0060] In addition, each step included in the anomaly detection method performed by the anomaly detection device according to the above-described embodiment can be implemented in a computer-readable recording medium having recorded thereon a computer program including instructions for causing a processor to perform these steps.

[0061] The combination of each step of each flowchart attached to the present invention may be performed by computer program instructions. These computer program instructions may be installed in a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment create a means for performing the functions described in each step of the flowchart. These computer program instructions may also be stored in a computer-usable or computer-readable recording medium that can direct a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-usable or computer-readable recording medium can also produce a manufactured article that includes an instruction means for performing the functions described in each step of the flowchart. Since the computer program instructions can also be installed on a computer or other programmable data processing device, a series of operational steps can be performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform can also provide steps for performing the functions described in each step of the flowchart.

[0062] Additionally, each step may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative embodiments, the functions described in the steps may occur out of order. For example, two steps depicted in succession may actually be performed substantially concurrently, or the steps may sometimes be performed in reverse order, depending on the corresponding function.

[0063] The above description is merely an illustrative illustration of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims

1. A learning method performed by an anomaly detection device, A step of inputting an image into a pre-training network and obtaining target features from the pre-training network; A step of obtaining local features by sampling normal features based on a local grid obtained by dimension reduction of a local area of ​​the above target features; A step of obtaining global features by sampling normal features based on a global grid obtained by dimension reduction of the entire target features; and A step of updating the pre-trained network based on the result of combining the local features and the global features; How to learn.

2. In paragraph 1, The step of obtaining the above target features is to obtain the target features by combining the intermediate layer feature maps of the above pre-trained network. How to learn.

3. In paragraph 1, The step of obtaining the above local features is to vectorize each pixel of the target feature and use it as a coordinate. How to learn.

4. In paragraph 3, Adding noise to the values ​​of the above coordinates before performing the above sampling. How to learn.

5. In paragraph 1, The step of obtaining the above global features is to vectorize the entire target features and use them as coordinates. How to learn.

6. In paragraph 1, The above updating step is to minimize the difference between the reconstructed features generated based on the combined result and the target features and the re-input features. How to learn.

7. In paragraph 6, The above updating step calculates the difference between the reconstructed feature and the re-input feature, and replaces at least some area of ​​the reconstructed feature that shows a difference less than a preset threshold with the corresponding area of ​​the re-input feature. How to learn.

8. A non-transitory computer-readable recording medium storing a computer program, The above computer program, when executed by a processor, A learning method for anomaly detection, which comprises: inputting an image into a pre-training network to obtain target features from the pre-training network; sampling normal features based on a local grid obtained through dimension reduction for a local region of the target features to obtain local features; sampling normal features based on a global grid obtained through dimension reduction for the entirety of the target features to obtain global features; and updating the pre-training network based on a result of combining the local features and the global features, the learning method comprising instructions for causing the processor to perform the learning method. A computer-readable storage medium storing a computer program.

9. Memory storing at least one instruction; and Processor; including; The processor performs anomaly detection on the image based on a result of combining local features and global features sampled from target features obtained through a pre-trained network from the image by executing the at least one command, The above pre-training network is learned by a learning method that inputs the image to the pre-training network to obtain the target feature from the pre-training network, samples normal features based on a local grid obtained by dimension reduction for a local region of the target feature to obtain the local feature, samples normal features based on a global grid obtained by dimension reduction for the entire target feature to obtain the global feature, and updates the pre-training network based on a result of combining the local feature and the global feature. Anomaly detection device.

10. An anomaly detection method performed by an anomaly detection device, The above anomaly detection method performs anomaly detection on the image based on the result of combining local features and global features sampled from target features obtained through a pre-trained network from the image, and The above pre-trained network is, A step of inputting the above image into the above pre-training network and obtaining the target feature from the above pre-training network; A step of obtaining the local features by sampling normal features based on a local grid obtained by dimension reduction in a local area of ​​the target features; A step of obtaining the global features by sampling normal features based on a global grid obtained by dimension reduction of the entire target features; and A learning method including a step of updating the pre-trained network based on the result of combining the local features and the global features; Anomaly detection methods.

11. In Article 10, The step of obtaining the above target features is to obtain the target features by combining the intermediate layer feature maps of the above pre-trained network. Anomaly detection methods.

12. In paragraph 10, The step of obtaining the above local features is to vectorize each pixel of the target feature and use it as a coordinate. Anomaly detection methods.

13. In paragraph 12, Adding noise to the values ​​of the above coordinates before performing the above sampling. Anomaly detection methods.

14. In paragraph 10, The step of obtaining the above global features is to vectorize the entire target features and use them as coordinates. Anomaly detection methods.

15. In paragraph 10, The above updating step is to minimize the difference between the reconstructed features generated based on the combined result and the target features and the re-input features. Anomaly detection methods.

16. In paragraph 15, The above updating step calculates the difference between the reconstructed feature and the re-input feature, and replaces at least some area of ​​the reconstructed feature that shows a difference less than a preset threshold with the corresponding area of ​​the re-input feature. Anomaly detection methods.

17. A non-transitory computer-readable recording medium storing a computer program, The above computer program, when executed by a processor, Includes instructions for causing the processor to perform anomaly detection on the image based on the result of combining local features and global features sampled from target features obtained through a pre-trained network from the image, The above pre-training network is learned by a learning method that inputs the image to the pre-training network to obtain the target feature from the pre-training network, samples normal features based on a local grid obtained by dimension reduction for a local region of the target feature to obtain the local feature, samples normal features based on a global grid obtained by dimension reduction for the entire target feature to obtain the global feature, and updates the pre-training network based on a result of combining the local feature and the global feature. A non-transitory computer-readable storage medium storing a computer program.

Images