Knowledge distillation-based anomaly detection learning method and system therefor

Abstract

A knowledge distillation-based anomaly detection learning method according to an embodiment of the present invention comprises the steps of: acquiring a plurality of test sample images; calculating an anomaly score that is data obtained by digitizing a probability of a defect per patch feature for each of the plurality of test sample images; determining, as a defective candidate image, a test sample image in which the anomaly score satisfies a predetermined first condition; detecting, as a defective candidate patch feature, a patch feature in which the anomaly score satisfies a predetermined second condition, from among patch features of the defective candidate image; training a first adapter that is in charge of training in the current period, on the basis of the defective candidate patch feature; and updating a memory bank storing training information about a predetermined outlier detection model in a past period, on the basis of the training, wherein the first adapter is a network that performs the training in the current period, which reflects new information according to the plurality of test sample images while maintaining the training information in the past period at a certain level.

Description

Knowledge distillation-based anomaly detection learning method and system

[0001] The present invention relates to a knowledge distillation-based anomaly detection learning method and system. More specifically, it relates to a knowledge distillation-based anomaly detection learning method and system that performs adaptive update learning that reflects newly accumulated data while preserving previously learned information.

[0002] Anomaly detection refers to the process of identifying abnormal patterns, outliers, and / or exception values ​​from given data.

[0003] In other words, anomaly detection can be a process of detecting components that deviate from the attributes of normal data.

[0004] Systems implementing such anomaly detection are actively used in various application fields where the identification of abnormal patterns is important, such as process monitoring, security intrusion detection, fraud identification, and / or medical diagnosis.

[0005] Conventional deep learning models that perform anomaly detection as described above primarily learn using good product data and operate by distinguishing between good and defective products.

[0006] While model training and process application occur rapidly during the initial setup phase, problems arise where model performance degrades over time as data distribution changes due to shifts in production lines or inspection environments.

[0007] This results in problems such as failure to accurately detect defects or false positives where good products are mistakenly identified as defective.

[0008] To address this problem, conventional methods in this technical field have attempted to retrain existing models; however, this has the disadvantage that previously learned information may be lost while incorporating new data.

[0009] If previously learned information is lost during the retraining process, the overall performance of the model may degrade.

[0010] Therefore, there is a growing need for sustainable anomaly detection models that can continuously improve performance by incorporating new data while maintaining existing learned information.

[0011] Meanwhile, conventional anomaly detection models face a problem in that they are limited in the performance of task processing for anomaly detection when the data required for training the model is relatively scarce or diverse and insufficient—such as when it is difficult to collect abnormal data containing specific defects, when labeled data is limited, or when attempting to train a large amount of unlabeled data.

[0012] Furthermore, while image-based anomaly detection is actively performed in the field of vision inspection, there is a problem in that the data processing and computational costs are inefficient because these images are high-dimensional data, and using all data from the entire image in a single batch to detect anomalies is not efficient.

[0013] In addition, in most cases, outliers are observed as abnormal patterns appearing in various sizes and shapes within small parts of an image; however, conventional methods have a problem in that they have very low discriminability regarding such local patterns across the entire image.

[0014] Therefore, it is also necessary to develop new technologies that can further improve the accuracy and efficiency of anomaly detection even under limited environmental conditions, while simultaneously enhancing task processing performance accordingly.

[0015] One embodiment of the present invention is devised to solve the problems of the prior art as described above, and aims to provide a knowledge distillation-based anomaly detection learning method and system that performs adaptive update learning that reflects newly accumulated data while preserving previously learned information.

[0016] In addition, one embodiment of the present invention aims to provide a knowledge distillation-based anomaly detection learning method and system that performs adaptive update learning, which automatically detects a data distribution that changes over time and performs relearning (update) reflecting it.

[0017] In addition, one embodiment of the present invention aims to provide a knowledge distillation-based anomaly detection learning method and system that performs patch feature-based learning on a predetermined pretrained model based on an image data set of anomaly detection targets.

[0018] At this time, one embodiment of the present invention aims to provide a knowledge distillation-based anomaly detection learning method and system that performs patch feature-based learning to reduce the variance of mutually similar patch features and increase the difference between mutually heterogeneous patch features.

[0019] However, the technical problems that the present invention and the embodiments of the present invention aim to solve are not limited to the technical problems described above, and other technical problems may exist.

[0020] A knowledge distillation-based anomaly detection learning method according to an embodiment of the present invention is a method in which a computing system including memory and a processor performs knowledge distillation-based anomaly detection learning, comprising: a step of acquiring a plurality of test sample images; a step of calculating an anomaly score, which is data quantifying the probability of defects for each of the plurality of test sample images by patch feature; a step of determining a test sample image in which the anomaly score satisfies a predetermined first condition as a defect candidate image; a step of detecting a patch feature among the defect candidate images in which the anomaly score satisfies a predetermined second condition as a defect candidate patch feature; a step of performing learning of a first adapter responsible for learning of the current cycle based on the defect candidate patch feature; and a step of updating a memory bank that stores learning information of a past cycle of a predetermined anomaly detection model based on the learning, wherein the first adapter is a network that performs learning of the current cycle that reflects new information according to the plurality of test sample images while maintaining the learning information of the past cycle at a certain level.

[0021] In another aspect, the step of acquiring the plurality of test sample images includes the step of collecting test sample images accumulated over a predetermined period.

[0022] In another aspect, the step of calculating the anomaly score includes the step of calculating a representative anomaly score for each of the plurality of test sample images based on the anomaly scores for each of the plurality of test sample images, wherein the representative anomaly score is an anomaly score having a maximum value among the anomaly scores.

[0023] In another aspect, the step of determining the defective candidate image includes the step of determining the test sample image as the defective candidate image in which the anomaly representative score satisfies the predetermined first condition.

[0024] In another aspect, the step of performing the first adapter training comprises a first loss function which is a loss function that optimizes the distance between patch features based on a predetermined similarity between patch features, a second loss function which is a loss function that optimizes the distance between patch features based on a predetermined classification class for each patch feature, and a third loss function which is a loss function that optimizes the learning similarity between patch features corresponding to the same patch.

[0025] In another aspect, the step of performing the first adapter learning further comprises the step of forming a batch that includes, in a predetermined ratio, at least some of the defective candidate patch features according to the plurality of test sample images and the remaining patch features excluding the defective candidate patch features, which are good candidate patch features, and the step of performing the first adapter learning based on the formed batch.

[0026] In another aspect, the step of performing the first adapter training based on the first loss function includes the step of performing the first adapter training based on pairwise similarity, which is the similarity between the embedding vectors of each of the patch features, and contextual similarity, which is the similarity between the nearest neighbor patches of each of the patch features.

[0027] In another aspect, the step of performing the first adapter training based on the first loss function further includes the step of performing the first adapter training based on a Euclidean distance that specifies the straight-line distance between the patch features.

[0028] In another aspect, the step of performing the first adapter learning based on the second loss function includes the step of performing the first adapter learning to optimize the distinction between a defective class corresponding to a defective candidate patch feature in the batch and a good class corresponding to a good candidate patch feature in the batch.

[0029] In another aspect, the step of performing the first adapter learning based on the third loss function includes the step of performing the first adapter learning based on the cosine similarity between the patch feature learned in the past period and the patch feature learned in the current period for a predetermined first patch.

[0030] In another aspect, a knowledge distillation-based anomaly detection learning method according to one embodiment of the present invention further includes the step of using the learning information of the current cycle learned through the first adapter as the learning information of the past cycle when learning the future cycle.

[0031] In another aspect, a knowledge distillation-based anomaly detection learning method according to one embodiment of the present invention further comprises the step of performing third network learning based on the learning information of the past cycle and the output of the first adapter, wherein the third network is a network that minimizes the difference between a patch feature according to the learning information of the past cycle and a patch feature according to the output of the first adapter for a predetermined first patch.

[0032] In another aspect, the step of performing the third network learning includes the step of minimizing the difference between the patch features based on the L1 loss function.

[0033] In another aspect, the step of updating the memory bank includes the step of updating the memory bank based on coreset sampling.

[0034] In another aspect, a knowledge distillation-based anomaly detection learning method according to one embodiment of the present invention further includes the step of determining whether to perform knowledge distillation-based anomaly detection learning based on anomaly scores for each of the plurality of test sample images.

[0035] Meanwhile, a knowledge distillation-based anomaly detection learning system according to one embodiment of the present invention comprises at least one memory; The apparatus comprises at least one processor that reads at least one application stored in the memory and performs knowledge distillation-based anomaly detection learning; wherein the instructions of the processor include the steps of acquiring a plurality of test sample images, calculating an anomaly score which is data quantifying the probability of defects for each patch feature of each of the plurality of test sample images, determining a test sample image in which the anomaly score satisfies a predetermined first condition as a defect candidate image, detecting a patch feature among the patch features of the defect candidate image in which the anomaly score satisfies a predetermined second condition as a defect candidate patch feature, performing first adapter learning responsible for learning in the current cycle based on the defect candidate patch feature, and updating a memory bank that stores learning information of a past cycle of a predetermined anomaly detection model based on the learning; wherein the first adapter is a network that performs learning in the current cycle while maintaining the learning information of the past cycle at a certain level and reflecting new information according to the plurality of test sample images.

[0036] The knowledge distillation-based anomaly detection learning method and system according to one embodiment of the present invention performs adaptive update learning that preserves previously learned information while reflecting newly accumulated data, thereby ensuring a connection between existing learning information and new learning information. This ensures that the model can provide a robust model that stably maintains anomaly detection accuracy and consistency even in a data environment that changes over time and continuously improves its performance.

[0037] In addition, the knowledge distillation-based anomaly detection learning method and system according to one embodiment of the present invention automatically detects data distributions that change over time and performs retraining (updates) reflecting them, thereby enabling the model to adapt quickly and efficiently to new data environments, which reduces resource requirements for maintenance and simultaneously facilitates the improvement of anomaly detection performance.

[0038] In addition, the knowledge distillation-based anomaly detection learning method and system according to one embodiment of the present invention has the effect of providing an anomaly detection model that further improves task processing performance and quality for anomaly detection by performing patch feature-based learning on a predetermined pretrained model based on an image data set of anomaly detection targets, thereby performing more efficient data processing.

[0039] In addition, the knowledge distillation-based anomaly detection learning method and system according to one embodiment of the present invention has the effect of improving the accuracy and efficiency of anomaly detection even in a limited learning environment, and simultaneously enhancing task processing performance accordingly, by performing patch feature-based learning that reduces the variance of mutually similar patch features and increases the difference between mutually heterogeneous patch features.

[0040] However, the effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects can be clearly understood from the description below.

[0041] FIG. 1 illustrates an example of a block diagram of a computing system implementing an anomaly detection learning service according to one embodiment of the present invention.

[0042] FIG. 2 illustrates an example of a block diagram of a computing device implementing an anomaly detection learning service according to one embodiment of the present invention.

[0043] FIG. 3 illustrates an example of a block diagram in another aspect of a computing device implementing an anomaly detection learning service according to one embodiment of the present invention.

[0044] FIG. 4 is an example of a block flowchart for explaining an outlier detection model according to one embodiment of the present invention.

[0045] FIG. 5 is a flowchart illustrating a patch feature learning method for anomaly detection according to an embodiment of the present invention.

[0046] FIG. 6 is a flowchart illustrating a patch feature-based feature representation learning method according to an embodiment of the present invention.

[0047] FIG. 7 is an example of a drawing showing an example of measuring similarity between patch features according to an embodiment of the present invention.

[0048] FIG. 8 is an example of a drawing showing an example of the application of a ReConPatch Process according to an embodiment of the present invention.

[0049] FIG. 9 is a flowchart illustrating a knowledge distillation-based anomaly detection learning method according to one embodiment of the present invention.

[0050] FIG. 10 is an example of a drawing for explaining data shift detection according to an embodiment of the present invention.

[0051] FIG. 11 is a conceptual diagram illustrating an adaptive update adapter learning method according to one embodiment of the present invention.

[0052] FIG. 12 is a conceptual diagram illustrating a third network learning method according to an embodiment of the present invention.

[0053] FIG. 13 is a conceptual diagram illustrating a memory bank update method according to one embodiment of the present invention.

[0054] FIG. 14 is a conceptual diagram illustrating an anomaly detection inference process using a learned outlier detection model according to one embodiment of the present invention.

[0055] The present invention is capable of various modifications and may have various embodiments; therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described below in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various forms. In the following embodiments, terms such as "first," "second," etc., are used not in a limiting sense but for the purpose of distinguishing one component from another. Furthermore, singular expressions include plural expressions unless the context clearly indicates otherwise. Also, terms such as "include" or "have" mean that the features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added. Additionally, in the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily depicted for convenience of explanation, so the present invention is not necessarily limited to what is depicted.

[0056] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0057]

[0058] [Exemplary system providing an anomaly detection learning service]

[0059] Hereinafter, an exemplary system for implementing a knowledge distillation-based anomaly detection learning service (hereinafter referred to as the anomaly detection learning service) that performs adaptive update learning reflecting newly accumulated data while preserving previously learned information will be described in detail with reference to the attached drawings.

[0060] FIG. 1 illustrates an example of a block diagram of a computing system implementing an anomaly detection learning service according to one embodiment of the present invention.

[0061] Referring to FIG. 1, a computing system (1000) implementing an anomaly detection learning service of the present invention includes a user computing device (110), a server computing system (130), and a training computing system (150), and the devices can communicate through a network (170).

[0062] A patch feature learning method for anomaly detection according to an embodiment of the present invention may be implemented and provided locally by a user computing device (110), implemented and provided in the form of a web service by a server computing system (130) communicating with the user computing device (110), or implemented and provided by the user computing device (110) and the server computing system (130) in conjunction with each other.

[0063] In this embodiment, the user computing device (110) and / or the server computing system (130) can train a machine learning model (120 and / or 140) through interaction with a training computing system (150) that is communicatedly connected via a network (170). The training computing system (150) may be separate from the server computing system (130) or may be part of the server computing system (130).

[0064] And at this time, the artificial intelligence model (in the embodiment, an anomaly detection model, etc.) can be 1) trained directly locally by the user computing device (110), 2) trained by the server computing system (130) and the user computing device (110) interacting with each other through the network (170), and 3) trained by a separate training computing system (150) using various training and learning techniques. The artificial intelligence model trained by the training computing system (150) may also be implemented by transmitting it to the user computing device (110) and / or the server computing system (130) through the network (170) to provide / update it.

[0065] In some embodiments, the training computing system (150) may be part of the server computing system (130) or part of the user computing device (110).

[0066] The user computing device (110) may include all other types of computing devices, such as a smartphone, a mobile phone, a digital broadcasting device, a PDA (personal digital assistants), a PMP (portable multimedia player), a desktop, a wearable device, an embedded computing device and / or a tablet PC.

[0067] The user computing device (110) includes at least one processor (111) and memory (112). Here, the processor (111) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions, or a plurality of electrically connected processors.

[0068] The memory (112) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof, and may include web storage of a server that performs memory storage functions on the internet. This memory (112) may store data (113) and instructions (114) necessary for the at least one processor (111) to perform functional operations such as training an artificial intelligence model or performing outlier detection through the artificial intelligence model.

[0069] In one embodiment, the user computing device (110) can store at least one machine learning model (120).

[0070] Specifically, the machine learning model (120) may be various machine learning models such as multiple neural networks (e.g., deep neural networks) or other types of machine learning models including non-linear models and / or linear models, and may be composed of a combination of these.

[0071] In this case, the neural network may include at least one of feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks and / or other types of neural networks.

[0072] In one embodiment, a user computing device (110) receives at least one machine learning model (120) from a server computing system (130) through a network (170), stores it in memory (112), and then executes the stored machine learning model (120) by a processor (111) to perform outlier detection, etc.

[0073] In another embodiment, the server computing system (130) may include at least one machine learning model (140) and perform operations through the machine learning model (140), and may provide an anomaly detection learning service to the user by communicating with the user computing device (110) and related data.

[0074] For example, a user computing device (110) can perform an anomaly detection learning service in such a way that a server computing system (130) provides an output for the user's input using a machine learning model (140) via the web.

[0075] Additionally, the artificial intelligence model can be implemented in such a way that at least some of the machine learning models (120 and / or 140) are executed on a user computing device (110) and the rest are executed on a server computing system (130).

[0076] Additionally, the user computing device (110) may include at least one input component (121) for detecting user input. For example, the user input component (121) may include a touch sensor (e.g., a touch screen and / or a touch pad, etc.) for detecting a touch of a user input medium (e.g., a finger or a stylus), an image sensor for detecting user motion input, a microphone for detecting user voice input, a button, a mouse and / or a keyboard, etc. Additionally, the user input component (121) may include an interface and an external controller when receiving input to an external controller (e.g., a mouse and / or a keyboard, etc.) through an interface.

[0077] A server computing system (130) includes at least one processor (131) and memory (132). Here, the processor (131) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions, or a plurality of electrically connected processors.

[0078] And the memory (132) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory device, magnetic disk, etc. and combinations thereof. This memory (132) may store data (133) and instructions (134) necessary for the processor (131) to perform functional operations, such as training an artificial intelligence model or performing outlier detection through the artificial intelligence model.

[0079] In one embodiment, the server computing system (130) may be implemented to include at least one computing device. For example, the server computing system (130) may be implemented to operate a plurality of computing devices according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. Additionally, the server computing system (130) may include a plurality of computing devices connected to a network (170).

[0080] Additionally, the server computing system (130) may store at least one machine learning model (140). For example, the server computing system (130) may include a neural network and / or other multi-layer non-linear model as the machine learning model (140). Exemplary neural networks may include a feed-forward neural network, a deep neural network, a recurrent neural network, and a convolutional neural network.

[0081] The training computing system (150) includes at least one processor (151) and memory (152). Here, the processor (151) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions, or a plurality of electrically connected processors.

[0082] And the memory (152) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory device, magnetic disk, etc. and combinations thereof. This memory (152) may store data (153) and instructions (154) necessary for the processor (151) to perform learning of an artificial intelligence model, etc.

[0083] For example, the training computing system (150) may include a model trainer (160) that trains a machine learning model (120 and / or 140) stored in a user computing device (110) and / or a server computing system (130) using various training or learning techniques, such as back propagation of error (according to the framework illustrated in FIG. 3).

[0084] For example, such a model trainer (160) can perform updates to one or more parameters of a machine learning model (120 and / or 140) in a backpropagation manner based on a defined loss function.

[0085] In some embodiments, performing backpropagation of the error may include performing truncated backpropagation through time. The model trainer (160) may perform a number of generalization techniques (e.g., weight devaluation, dropout and / or knowledge distillation, etc.) to improve the generalization ability of the machine learning model (120 and / or 140) being trained.

[0086] In particular, the model trainer (160) can train a machine learning model (120 and / or 140) based on a series of training data (161). Here, the training data (161) may include data of different forms, such as images, audio samples and / or text, for example. Examples of image types that may be used may include video frames, LiDAR point clouds, X-ray images, computed tomography scans, hyperspectral images and / or various other forms of images.

[0087] These training data (161) may be provided by a user computing device (110) and / or a server computing system (130). When the training computing device trains a machine learning model (120 and / or 140) on specific data of the user computing device (110), the machine learning model (120 and / or 140) may be characterized as a personalized model.

[0088] And the model trainer (160) includes computer logic that is utilized to provide the desired function.

[0089] Additionally, the model trainer (160) may be implemented as hardware, firmware, and / or software that controls a general-purpose processor. In one embodiment, the model trainer (160) may include a program file stored in a storage device, be loaded into memory (152), and be executed by one or more processors (151). In another embodiment, the model trainer (160) includes one or more sets of computer-executable data (153) and instructions (154) stored in a tangible computer-readable storage medium, such as a RAM hard disk or an optical or magnetic medium.

[0090] Network (170) includes, but is not limited to, 3GPP (3rd Generation Partnership Project) network, LTE (Long Term Evolution) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth network, satellite broadcasting network, analog broadcasting network and / or DMB (Digital Multimedia Broadcasting) network.

[0091] Generally, communication through the network (170) can be performed using any type of wired and / or wireless connection through various communication protocols (e.g., TCP / IP, HTTP, SMTP and / or FTP, etc.), encodings or formats (e.g., HTML and / or XML, etc.), and / or protection schemes (e.g., VPN, Secure HTTP and / or SSL, etc.).

[0092] FIG. 2 illustrates an example of a block diagram of a computing device implementing an anomaly detection learning service according to one embodiment of the present invention.

[0093] Including FIG. 2, the computing device (100) included in the user computing device (110), server computing system (130), and training computing system (150) includes a plurality of applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, the applications may include an image processing application (e.g., Detection, Classification and / or Segmentation, etc.), a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application and / or a chat-bot application, etc.

[0094] In an embodiment, the computing device (100) may include a model trainer (160) for training an artificial intelligence model, and by storing and operating the trained artificial intelligence model, it may provide output data according to a predetermined input data (e.g., image data).

[0095] Each application of the computing device (100) can communicate with a number of other components of the computing device (100), such as, for example, at least one sensor, a context manager, a device state component, and / or additional components. In one embodiment, each application can communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0096] FIG. 3 illustrates an example of a block diagram in another aspect of a computing device (100) implementing an anomaly detection learning service according to one embodiment of the present invention.

[0097] Referring to FIG. 3, the computing device (300) includes a plurality of applications (e.g., Application 1 to Application N). Each application can communicate with a central intelligence layer. For example, applications may include an image processing application, a text messaging application, an email application, a dictation application, a virtual keyboard application and / or a browser application, etc. In one embodiment, each application can communicate with the central intelligence layer (and a model stored therein) using an API (e.g., a common API across all applications).

[0098] The central intelligence layer may include a number of machine learning models. For example, as illustrated in FIG. 3, at least some of the machine learning models may be provided for each application and managed by the central intelligence layer. In other embodiments, two or more applications may share a single machine learning model. For example, in some embodiments, the central intelligence layer may provide a single model for all applications. In some embodiments, the central intelligence layer may be included within the operating system of the computing device (300) or otherwise implemented.

[0099] The central intelligence layer can communicate with the central device data layer. The central device data layer may be a centralized data store for the computing device (300). As illustrated in FIG. 3, the central device data layer can communicate with a number of other components of the computing device (300), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some embodiments, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0100] The technology described herein may refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information transmitted to or from said systems. It will be recognized that the inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, division of tasks, and functionality between and from components. For example, the processes described herein may be implemented using a single device or component or multiple devices or components operating in combination. Databases and applications may be implemented in a single system or in a distributed system across multiple systems. Distributed components may operate sequentially or in parallel.

[0101]

[0102] [Outlier Detection Model (ODM)]

[0103] FIG. 4 is an example of a block flowchart for explaining an outlier detection model (ODM) according to one embodiment of the present invention.

[0104] Referring to FIG. 4, an anomaly detection model (ODM) according to an embodiment of the present invention may mean an image deep learning model that performs anomaly detection based on a predetermined input image and classifies and / or recognizes the image based thereon.

[0105] For reference, anomaly detection here may refer to the process of identifying abnormal patterns, outliers, and / or exception values ​​from specific data.

[0106] In other words, anomaly detection can be a process of detecting components that deviate from the attributes of normal data.

[0107] As an example, anomaly detection can be implemented based on a method such as grouping predetermined data into clusters and considering points separated from said clusters as outliers.

[0108] Accordingly, in the embodiment, the outlier detection model (ODM) can determine whether a given input image contains specific abnormal attributes and classify and / or recognize the image based on the determination result.

[0109] Specifically, in the embodiment, the outlier detection model (ODM) may include a first network (TN: Teacher Network) and a second network (SN: Student Network).

[0110] More specifically, the first network (TN) according to the embodiment may mean a neural network that calculates similarity between predetermined features (patch features in the embodiment).

[0111] In an embodiment, this first network (TN) is a first feature representation layer ( ) and the first spatial projection layer ( It may include ).

[0112] Here, the feature representation layer according to the embodiment may refer to a layer that reconstructs (adjusts) a feature in a direction that improves the performance of the feature representation according to a predetermined feature (a patch feature in the embodiment).

[0113] To this end, the feature representation layer can be trained to extract meaningful features from a predetermined feature (patch feature in the example) with higher accuracy and to reconstruct (adjust) the corresponding feature based on this.

[0114] In addition, the spatial projection layer according to the embodiment may mean a layer that projects a feature representation according to a predetermined feature (in the embodiment, a patch feature) into a predetermined feature representation space.

[0115] In the embodiments, this spatial projection layer can be trained to project a feature representation according to a predetermined feature (in the embodiments, a patch feature) onto a feature representation space where the goal of model training can be applied more effectively.

[0116] Meanwhile, the second network (SN) according to the embodiment may mean a neural network that implements feature representation learning.

[0117] For reference, feature representation learning can refer to the process in which a deep learning model automatically detects and learns useful features from given data.

[0118] Through feature representation learning, deep learning models can effectively encode useful information contained in data to generate meaningful features that can be used in various deep learning tasks, and can understand the complex structure and patterns of data and perform more accurate predictions based on this.

[0119] In an embodiment, the second network (SN) implementing the above-mentioned feature representation learning is a second feature representation layer ( ) and the second spatial projection layer ( It may include ).

[0120] At this time, the first feature expression layer according to the embodiment ( ) and second feature representation layer( ) and the first spatial projection layer ( ) and the second spatial projection layer ( Since the purpose of this is to distinguish between the feature representation layer and spatial projection layer included in the first network (TN) and the feature representation layer and spatial projection layer included in the second network (SN), the description of the feature representation layer and spatial projection layer of the second network (SN) follows the description of the feature representation layer and spatial projection layer of the first network (TN) described above.

[0121] In addition, a more detailed description of the first network (TN) and the second network (SN) according to the embodiment will be provided in the patch feature learning method for anomaly detection described later.

[0122] On the other hand, in the embodiment, the outlier detection model (ODM) can perform various functional operations required for an anomaly detection learning service by linking with a model pre-trained to perform concept learning (hereinafter, pre-trained model).

[0123] Here, the pre-trained model according to the embodiment may be an image deep-learning model pre-trained to perform concept learning based on a predetermined training data set (e.g., a predetermined natural image data set and / or a predetermined normal image data set, etc.).

[0124] For reference, concept learning can refer to the process of inferring general rules, concepts, or patterns from given data and classifying them.

[0125] That is, in the embodiment, the pre-trained model may be an image deep learning model that takes a predetermined image as input data, learns common features of objects or patterns within the input image, groups image components having similar features based on this, and supports classifying or recognizing a specific image based on this.

[0126] Specifically, the pre-trained model can 1) extract a feature map for a given input image.

[0127] Specifically, the pre-trained model can automatically extract a feature map based on raw pixel data of an input image using a specified image deep learning neural network (e.g., a Convolutional Neural Network (CNN)).

[0128] These feature maps can represent various visual attributes of the image, such as edges, color, and / or texture.

[0129] In addition, the pre-trained model can perform clustering based on 2) the feature space.

[0130] Specifically, the pre-trained model can classify the input image and / or objects within the input image into groups with similar features based on the feature map extracted as above.

[0131] As an example, a pre-trained model can classify the extracted feature map according to the feature space using a predetermined clustering algorithm (e.g., K-means, DBSCAN and / or hierarchical clustering algorithm, etc.) or a dimensionality reduction algorithm (e.g., t-SNE and / or UMAP, etc.).

[0132] In addition, the pre-trained model can 3) assign a label to each cluster.

[0133] In other words, the pre-trained model can define a concept (hypothesis) representing each cluster and set it as a label for that cluster.

[0134] At this time, the pre-trained model can learn the features of images belonging to a specific concept or category by manually or semi-automatically assigning labels to each cluster.

[0135] In addition, the pre-trained model can 4) verify and adjust the assigned hypothesis.

[0136] Specifically, the pre-trained model can validate the initially defined concept (clustered feature group) as described above and adjust the hypothesis if necessary.

[0137] At this time, the pre-trained model can perform a process of detecting and improving misclustered data using new image data.

[0138] In addition, the pre-trained model can repeat the process described above. 5)

[0139] In other words, the pre-trained model can be trained to continuously improve clustered features and related concepts through new image data and additional feedback, and to classify or recognize images more accurately.

[0140] In this case, in the embodiment, the pre-training model may be directly included in the outlier detection model (ODM) or implemented as a separate device and / or server from the outlier detection model (ODM).

[0141] In the following description, it is explained that the pre-trained model is implemented by being included in the outlier detection model (ODM), but is not limited thereto.

[0142] In addition, in FIG. 4, the outlier detection model (ODM) is described as including the components as described above in order to prevent the features according to the embodiment of the present invention from being obscured; however, it is obvious to a person skilled in the art that, depending on the embodiment, other general-purpose components may be included in addition to the components shown in FIG. 4, or some components shown in FIG. 4 may be omitted.

[0143]

[0144] [Patch Feature Training Method for Anomaly Detection]

[0145] Hereinafter, a patch feature learning method for anomaly detection, in which a computing system (1000) according to an embodiment of the present invention performs patch feature-based learning on a predetermined pretrained model based on an image data set for anomaly detection targets, is described in detail.

[0146] A patch feature learning method for anomaly detection of a computing system (1000) according to an embodiment of the present invention can improve the performance and quality of various anomaly detection-based services by using an anomaly detection model (ODM) trained according to an embodiment of the present invention.

[0147] At this time, the patch feature learning method for anomaly detection of a computing system (1000) according to an embodiment of the present invention can effectively provide an anomaly detection model (ODM) with improved performance by performing patch feature-based learning that reduces the variance of mutually similar patch features and increases the difference between mutually heterogeneous patch features.

[0148] Hereinafter, a patch feature learning method for anomaly detection according to an embodiment of the present invention will be described in more detail with reference to the attached drawings.

[0149] FIG. 5 is a flowchart illustrating a patch feature learning method for anomaly detection according to an embodiment of the present invention.

[0150] Referring to FIGS. 4 and 5, a patch feature learning method for anomaly detection according to an embodiment of the present invention may include: a step of acquiring a feature map based on a pretrained model (S101); a step of extracting a plurality of patch features based on the acquired feature map (S103); a step of performing feature representation learning based on the extracted plurality of patch features (S105); a step of acquiring a reconpatch feature based on the performed feature representation learning (S107); a step of performing coreset sampling based on the acquired reconpatch feature (S109); a step of acquiring a test sample image (S111); a step of acquiring a reconpatch feature based on the acquired test sample image (S113); and a step of performing anomaly detection based on the acquired reconpatch feature (S115). there is.

[0151] Specifically, a computing system (1000) according to an embodiment of the present invention can acquire a feature map based on a pretrained model. (S101)

[0152] In detail, in an embodiment, the computing system (1000) can obtain a feature map according to an image data set (hereinafter, target image data set) for a predetermined outlier detection target through a model pre-trained to perform concept learning (i.e., pre-trained model).

[0153] Here, in other words, the pre-trained model according to the embodiment may be an image deep-learning model pre-trained to perform concept learning based on a predetermined training data set (e.g., a predetermined natural image data set and / or a predetermined normal image data set, etc.).

[0154] In other words, in the embodiment, the computing system (1000) can obtain a feature map based on a predetermined target image data set (i.e., an image data set including a plurality of images of a predetermined outlier detection target) in conjunction with a pre-training model as described above.

[0155] In detail, in an embodiment, the computing system (1000) can input a target image data set (e.g., an image data set including a plurality of images of a predetermined electronic circuit element, etc.) into a pre-training model.

[0156] Then, the above-mentioned pre-trained model can output a feature map according to the input target image data set and provide it to the computing system (1000).

[0157] Thus, the computing system (1000) can obtain a feature map according to the target image data set.

[0158] In addition, in the embodiment, the computing system (1000) can extract a plurality of patch features based on the acquired feature map. (S103)

[0159] Here, the patch feature according to the embodiment may mean a feature extracted from a patch representing a small part of a predetermined image.

[0160] Specifically, the patch may be a rectangular area representing a specific part within a given image. Such a patch can be viewed as a subset containing some of the information of the entire image, and may primarily include local information or texture information.

[0161] In addition, the above feature is feature information extracted from a predetermined image or patch, and can summarize or express important attributes of the image (e.g., pattern, texture, color and / or shape, etc.).

[0162] Therefore, the above patch feature may be data representing local attributes within a given image on a patch basis.

[0163] Specifically, in the embodiment, the computing system (1000) can extract a plurality of patch features based on the feature map obtained as above.

[0164] More specifically, as an example, the computing system (1000) may divide the target training image into predetermined patch size units before inputting the target training image (hereinafter, target training image) included in the target image data set into the aforementioned pre-training model.

[0165] And the computing system (1000) can input each divided patch into a pre-trained model to obtain a feature map corresponding to each patch.

[0166] In other words, the computing system (1000) can obtain a feature map for each patch by dividing the target training image into units of a predetermined patch size and inputting it into a pre-trained model.

[0167] At this time, according to the embodiment, the computing system (1000) can perform coreset sampling on the acquired patch-specific feature map.

[0168] Here, for reference, core set sampling refers to a method for efficiently processing large datasets, which can be described as a process of extracting a set of representative samples that preserve the statistical characteristics or structure of the original dataset as much as possible while reducing the size of the dataset.

[0169] In an example, the computing system (1000) can perform core set sampling using an approximation algorithm that selects some samples that can represent the entire data set while maintaining the characteristics of the original data set within a predetermined error range by considering the distribution of the given data.

[0170] Alternatively, the computing system (1000) may perform core set sampling using an importance sampling method that assigns sampling probabilities based on the importance of each given data point and prioritizes selecting data points with high importance.

[0171] Thus, the computing system (1000) can obtain multiple patch features on which core set sampling has been performed.

[0172] Meanwhile, in another embodiment, the computing system (1000) can obtain a feature map (hereinafter, entire feature map) for the entire area of ​​the target training image.

[0173] And the computing system (1000) can divide the entire acquired feature map into units of a predetermined patch size.

[0174] Thus, the computing system (1000) can extract multiple patch features from the target training image.

[0175] Thus, in the embodiment, the computing system (1000) can extract a plurality of patch features according to the target training image in at least one of the methods described above.

[0176] At this time, according to the embodiment, the computing system (1000) can extract each patch feature by aggregating surrounding feature vectors within a specific patch size.

[0177] Alternatively, according to an embodiment, the computing system (1000) may use the pixel value itself within each patch as a feature.

[0178] Alternatively, according to an embodiment, the computing system (1000) may use a statistical summary of pixel values ​​within each patch (e.g., mean, variance and / or histogram, etc.) as a feature.

[0179] Alternatively, according to an embodiment, the computing system (1000) may analyze the texture pattern within each patch and use it as a feature. For example, the computing system (1000) may extract texture-based patch features using a Gabor filter, LBP (Local Binary Patterns) and / or HOG (Histogram of Oriented Gradients) techniques, etc.

[0180] Alternatively, according to an embodiment, the computing system (1000) can automatically learn and extract high-dimensional features within each patch using a deep learning algorithm such as a convolutional neural network (CNN) and use them as features.

[0181] In this way, in the embodiment, the computing system (1000) extracts features at the patch level and supports outlier detection using them, thereby increasing processing efficiency in the data learning and analysis process and simultaneously detecting abnormal local patterns that mainly appear in small parts of the image in greater detail.

[0182] Additionally, in the embodiment, the computing system (1000) can perform Feature Representation Learning based on a plurality of extracted patch features. (S105)

[0183] Here, in other words, feature representation learning may mean a process in which a deep learning model (in the example, an outlier detection model (ODM)) automatically detects and learns useful features from given data.

[0184] Specifically, in an embodiment, the computing system (1000) can learn the feature representation of a plurality of patch features extracted as above based on an outlier detection model (ODM) according to an embodiment of the present invention.

[0185] That is, the computing system (1000) can perform feature representation learning for an anomaly detection model (ODM) that performs anomaly detection based on the extracted plurality of patch features.

[0186] In other words, anomaly detection can refer to a process of identifying abnormal patterns, outliers, and / or exception values ​​from specific data—that is, a process of detecting components that deviate from the attributes of normal data.

[0187] Accordingly, in the embodiment, the computing system (1000) can determine whether a predetermined input image includes a specific abnormal attribute, and, based on the determination result, perform feature representation learning (e.g., concept learning) for classifying and / or recognizing the image based on the above-described plurality of patch features.

[0188] In this case, in the embodiment, the computing system (1000) can perform the feature representation learning described above based on a semi-supervised learning method.

[0189] In other words, the computing system (1000) can build an anomaly detection model (ODM) that implements semi-supervised anomaly detection based on semi-supervised learning.

[0190] For reference, semi-supervised learning can refer to a deep learning method that trains a model using not only labeled data (i.e., supervised data) but also unlabeled data (i.e., unsupervised data).

[0191] Generally, when collecting base data to build an anomaly detection system, it is difficult to obtain a sufficient amount of abnormal data for smooth learning (e.g., image data capturing the abnormal state of an anomaly detection target), and consequently, limitations may arise in anomaly detection learning aimed at recognizing abnormal states (anomalies) of various shapes with high accuracy.

[0192] Accordingly, in an embodiment of the present invention, a pre-trained model (i.e., a pre-trained model) is constructed primarily using normal data (e.g., image data capturing the normal state of an anomaly detection target, etc.), and a semi-supervised learning-based anomaly detection is implemented by performing anomaly detection based on pseudo-labels using the said pre-trained model.

[0193] Here, for reference, a pseudo-label may refer to a label predicted by a model trained on unlabeled data.

[0194] These pseudo-labels can be used primarily when model training is performed using limited labeled data or by utilizing a large amount of additional unlabeled data.

[0195] Through this, in the embodiment, the computing system (1000) can easily achieve model training and performance improvement for building an outlier detection process even when labeled data is relatively rare or diverse and limited.

[0196] More specifically, in an embodiment, the computing system (1000) can perform feature representation learning based on a plurality of patch features based on a first network (TN) and a second network (SN) of an outlier detection model (ODM).

[0197] FIG. 6 is a flowchart illustrating a patch feature-based feature representation learning method according to an embodiment of the present invention.

[0198] Specifically, referring to FIG. 6, in an embodiment, the computing system (1000) can project any patch feature pair onto a predetermined feature representation space. (S201)

[0199] Specifically, in an embodiment, the computing system (1000) has any first patch feature ( ) and 2nd patch feature( A patch feature pair (hereinafter referred to as the first patch feature pair) consisting of a pair can be projected into a feature representation space.

[0200] In an embodiment, the computing system (1000) has a first patch feature (projected into a feature representation space) Express ) as in [Equation 1 (a)] below, and the second patch feature projected into the feature representation space ( ) can be expressed as [Equation 1 (b)] below.

[0201] [Mathematical Equation 1 - (a)]

[0202]

[0203] [Mathematical Equation 1 - (b)]

[0204]

[0205] Additionally, in the embodiment, the computing system (1000) can calculate pairwise similarity based on patch feature pairs projected into a feature representation space. (S203)

[0206] Here, pairwise similarity according to the embodiment refers to a first patch feature included in any patch feature pair ( ) and 2nd patch feature( It can mean data that measures similarity between ).

[0207] That is, in the embodiment, the computing system (1000) includes a first patch feature ( ) and the second patch feature( Pairwise similarity, which indicates the similarity between ), can be measured.

[0208] At this time, as an example, the computing system (1000) can calculate the pairwise similarity described above according to [Equation 2] below.

[0209] [Mathematical Formula 2]

[0210]

[0211] FIG. 7 is an example of a drawing showing an example of measuring similarity between patch features according to an embodiment of the present invention.

[0212] However, referring to FIG. 7, a first patch feature included in any patch feature pair ( ) and 2nd patch feature( When measuring similarity only in relation to ), the pairwise similarity is the same, but the discrimination accuracy may be reduced in cases where the first feature and the second feature must be classified into different labels as in Fig. 7 (a) (i.e., they must be further apart from each other to get closer to the correct label) and cases where the first feature and the second feature must be classified into the same label as in Fig. 7 (b) (i.e., they must be closer to each other to get closer to the correct label).

[0213] In other words, when measuring only pairwise similarity, the first patch feature ( K nearest neighbors for ) and 2nd patch feature ( K nearest neighbors for ) The accuracy of label prediction may be reduced by performing it without considering mutual similarity in group relationships that include [the element].

[0214] Thus, in the embodiment, the computing system (1000) can calculate contextual similarity based on patch feature pairs projected into a feature representation space. (S205)

[0215] Here, context similarity according to the embodiment refers to a first patch feature included in any patch feature pair ( K nearest neighbors for ) and second patch feature ( K nearest neighbors for ) It can refer to data that measures bidirectional similarity between them.

[0216] In this case, in the embodiment, the bidirectional similarity is a first patch feature ( K nearest neighbors for ) Features and second patch features ( K nearest neighbors for ) It can be calculated based on the average similarity between features.

[0217] In detail, in an embodiment, the computing system (1000) can calculate the context similarity described above according to the K-Nearest Neighbors (K-NN) algorithm, which performs prediction based on the distance between data points as in [Equation 3] below, and [Equation 4].

[0218] [Mathematical Formula 3]

[0219]

[0220] [Mathematical Formula 4]

[0221]

[0222]

[0223] That is, in the embodiment, the computing system (1000) includes a first patch feature ( ) and 2nd patch feature( It is possible to calculate contextual similarity, which is considered to have higher contextual similarity the more closely the nearest neighbors are shared.

[0224] Thus, the computing system (1000) has a first patch feature ( ) and the second patch feature( It is possible to learn feature representations in group relationships including ) and reflect them in the pseudo-label prediction process.

[0225] That is, the computing system (1000) is a first patch feature ( ) and the second patch feature( By extracting the K closest feature samples of ), calculating contextual similarity to measure how many samples form an intersection, and using this together with pairwise similarity to train an outlier detection model (ODM), the trained outlier detection model (ODM) can be enabled to extract higher quality features.

[0226] Thus, the computing system (1000) can more accurately determine cases where the pairwise similarity between the first feature and the second feature is the same but the first feature and the second feature must be classified into different labels (i.e., cases where they need to be further apart from each other to get closer to the correct label) and cases where the first feature and the second feature must be classified into the same label (i.e., cases where they need to be closer to each other to get closer to the correct label), and reflect this in pseudo-label prediction. Therefore, the computing system (1000) can directly improve the task processing quality and performance of an anomaly detection model (ODM) that performs semi-supervised learning-based anomaly detection.

[0227] Returning to the example, the computing system (1000) can also calculate an integrated similarity based on the pairwise similarity and contextual similarity calculated as above. (S207)

[0228] Here, the integrated similarity according to the embodiment refers to a first patch feature included in any patch feature pair ( ) and 2nd patch feature( It can mean data that combines pairwise similarity and contextual similarity.

[0229] Specifically, in an embodiment, the computing system (1000) includes a first patch feature (included in the first patch feature pair according to [Equation 5] as follows) ) and the second patch feature( Pairwise similarity and contextual similarity between ) can be linearly combined.

[0230] [Mathematical Formula 5]

[0231]

[0232]

[0233] At this time, the integrated similarity according to the embodiment is, ' It can be defined as a linear combination of two similarities that satisfy '

[0234] Additionally, in the embodiment, the computing system (1000) can train the second network (SN) of the outlier detection model (ODM) based on the calculated integrated similarity. (S209)

[0235] That is, in the embodiment, the computing system (1000) can perform second network (SN) learning that implements feature representation learning using integrated similarity.

[0236] Specifically, in an embodiment, the computing system (1000) mitigates the integrated similarity calculated for a first patch feature pair and the contrast loss ( The second network (SN) can be trained by applying it to ).

[0237] Here, the loss relative to mitigation according to the embodiment ( ) is equal to the following [Mathematical Formula 6].

[0238] [Mathematical Formula 6]

[0239]

[0240] Here, 'z' in [Equation 6] is ' It is an embedding vector inferred by ', 'N' is the number of mini-batch, i.e., patch instances, 'm' is the reppelling margin, and ' ' can be a parameter that determines the weights of the attraction and repulsion loss terms.

[0241] FIG. 8 is an example of a drawing showing an example of the application of a ReConPatch Process according to an embodiment of the present invention.

[0242] At this time, referring to FIG. 8, in the embodiment, the computing system (1000) obtains a first patch feature ( ) and the second patch feature( If ) is determined to be a patch feature pair (hereinafter, positive feature pair) that should be classified into different labels, the first patch feature ( ) and the second patch feature( The second feature representation layer of the second network (SN) (to map ) while separating them from each other in the feature representation space It is possible to train the embedding function.

[0243] On the other hand, in the embodiment, the computing system (1000) has a first patch feature ( ) and the second patch feature( If ) is determined to be a patch feature pair (hereinafter, voice feature pair) that must be classified with the same label, the first patch feature ( ) and the second patch feature( The second feature representation layer of the second network (SN) (( It is possible to train the ), embedding function).

[0244] Thus, in the embodiment, the computing system (1000) has a second feature representation layer ( It can train )

[0245] That is, the computing system (1000) is a second feature representation layer ( ) can be trained to extract arbitrary patch features in a form closer to the correct pseudo-label.

[0246] Thus, in the embodiment, the computing system (1000) has a second feature representation layer ( By training the model to more accurately extract meaningful features from arbitrary patch features, the feature representation performance of the outlier detection model (ODM) itself can be directly improved, and at the same time, the processing quality of various tasks based thereon (such as anomaly detection in the example) can also be enhanced.

[0247] In addition, in the embodiment, the computing system (1000) that trains the second network (SN) based on integrated similarity can train the first network (TN) of the outlier detection model (ODM).

[0248] Specifically, in an embodiment, the computing system (1000) calculates the parameters of the second network (SN) according to the following [Equation 7] using an Exponential Moving Average (EMA) method ( Data according to ) parameters of the first network (TN) Data based on ) can be gradually distilled.

[0249] [Mathematical Formula 7]

[0250]

[0251] That is, in the embodiment, the computing system (1000) can perform learning of the first network (TN) of the outlier detection model (ODM) by gradually distilling the information learned in the second network (SN) to the first network (TN) according to the above-described [Equation 7].

[0252] At this time, in the embodiment, the computing system (1000) can perform the first network (TN) learning described above by further applying an update rate control variable, which is a variable that controls the speed of information distillation.

[0253] Thus, in the embodiment, the computing system (1000) can implement a plurality of patch feature-based feature representation learning based on the first network (TN) and the second network (SN) of the outlier detection model (ODM).

[0254] In this way, in the embodiment, the computing system (1000) can perform a process (in the embodiment, a ReConPatch Process) to build discriminative features for outlier detection by distilling the main features of the dataset for outlier detection targets in a pre-trained model based on semi-supervised learning as described above.

[0255] Thus, the computing system (1000) can build a high-performance outlier detection model (ODM) trained to classify corresponding features more accurately into correct pseudo-labels at a predetermined patch level.

[0256] Accordingly, the computing system (1000) can directly and effectively improve the processing performance and quality of various tasks (such as anomaly detection in the example) using the above-mentioned outlier detection model (ODM).

[0257] Returning to FIG. 5, in addition to the embodiment, the computing system (1000) can obtain a ReConPatch Feature based on the performed feature representation learning. (S107)

[0258] Here, the reconstructed feature according to the embodiment may refer to a patched feature output through a feature representation layer (hereinafter referred to as the reconstructed layer) on which the feature representation learning described above has been performed.

[0259] That is, in the embodiment, the reconstructed feature is the second feature representation layer of the second network (SN) where feature representation learning is performed ( It may be a patch feature output based on ).

[0260] Specifically, in the embodiment, the computing system (1000) can acquire reconstruct features for each target training image included in the aforementioned target image data set in conjunction with the reconstruct layer.

[0261] In other words, in the embodiment, the computing system (1000) can obtain a reconstructed feature dataset (hereinafter, reconstructed training dataset) in which the variance of similar patch features is reduced and the difference of heterogeneous patch features is increased for a plurality of patch features extracted from each target training image.

[0262] Additionally, in the embodiment, the computing system (1000) can perform Coreset Sampling based on the acquired reconstructed features. (S109)

[0263] Here, in other words, core set sampling according to the embodiment may refer to a process of extracting a set of representative samples that preserves the statistical characteristics or structure of the original data set as much as possible while reducing the size of the data set, as one of the methods for efficiently processing a large-scale data set.

[0264] In detail, as an example, the computing system (1000) can perform core set sampling using an approximation algorithm that selects some samples that can represent the entire reconstructed learning data set while maintaining the characteristics of the original reconstructed learning data set within a predetermined error range, by considering the distribution of the acquired reconstructed learning data set.

[0265] In another embodiment, the computing system (1000) can perform core set sampling using an importance sampling method that assigns sampling probabilities based on the importance of each acquired reconstructed feature data and preferentially selects data points with high importance.

[0266] Thus, the computing system (1000) can obtain a reconstructed learning data set (hereinafter, reconstructed sampling data set) on which core set sampling has been performed.

[0267] In addition, in the embodiment, the computing system (1000) can store and manage the acquired reconstructed sampling data set on a predetermined database.

[0268] Through this, the computing system (1000) can detect outliers with high accuracy while reducing data processing costs.

[0269] Additionally, in the embodiment, the computing system (1000) can acquire a test sample image. (S111)

[0270] Here, the test sample image according to the embodiment may refer to an image for which the presence or absence of anomalies is to be detected, that is, image data capturing the target for anomaly detection.

[0271] In detail, in an embodiment, the computing system (1000) can obtain a test sample image as described above based on a predetermined user input and / or interaction with an external server.

[0272] Additionally, in the embodiment, the computing system (1000) can acquire a reconstruct feature according to the acquired test sample image. (S113)

[0273] In the following, content that overlaps with the previously mentioned explanation may be summarized or omitted.

[0274] Specifically, in the embodiment, the computing system (1000) can input the acquired test sample image into a pre-trained model.

[0275] Additionally, in the embodiment, the computing system (1000) can obtain a feature map for a test sample image from a pre-trained model that receives the test sample image as input. A detailed description thereof follows the description of step S101 described above.

[0276] Additionally, in the embodiment, the computing system (1000) can extract a plurality of patch features based on the acquired feature map. A detailed description thereof follows the description of step S103 described above.

[0277] Additionally, in the embodiment, the computing system (1000) can input a plurality of extracted patch features into the reconpatch layer described above.

[0278] Then, the reconstruction layer can output multiple reconstruction features based on multiple input patch features.

[0279] Thus, in the embodiment, the computing system (1000) can obtain a reconstructed feature data set (hereinafter, reconstructed target data set) in a form that reduces the variance of similar patch features and increases the difference of heterogeneous patch features for a plurality of input patch features.

[0280] Additionally, in the embodiment, the computing system (1000) can perform outlier detection based on the acquired reconstructed features. (S115)

[0281] That is, in the embodiment, the computing system (1000) can perform outlier detection on the test sample image based on the reconstructed sampling data set obtained based on each training image used for learning and the reconstructed target data set obtained based on the test sample image.

[0282] In other words, in the embodiment, the computing system (1000) can perform anomaly detection on a test sample image based on a reconstructed sampling data set and a reconstructed target data set.

[0283] In detail, in an embodiment, the computing system (1000) can calculate a similarity (hereinafter, outlier detection similarity) between at least some of the reconstructed sampling data sets stored in a database and the reconstructed target data set.

[0284] In addition, in the embodiment, the computing system (1000) can generate an Anomaly Score Map based on the calculated outlier detection similarity.

[0285] Here, for reference, an anomaly score map may refer to an indicator that represents how far a state deviates from a predetermined steady state based on a value (score) assigned by the model.

[0286] Additionally, in the embodiment, the computing system (1000) can determine that the higher the score according to the generated anomaly score map, the closer the test sample image is to an abnormal state, and the lower the score according to the anomaly score map, the closer the test sample image is to a normal state.

[0287] Thus, in the embodiment, the computing system (1000) can perform outlier detection on a test sample image.

[0288] As described above, in an embodiment of the present invention, the computing system (1000) performs a process (in the embodiment, a ReConPatch Process) to build discriminative features for outlier detection by distilling the main features of a dataset of outlier detection targets in a pre-trained model based on semi-supervised learning as described above, and can perform anomaly detection using the learned outlier detection model (ODM) through this.

[0289] Thus, in the embodiment, the computing system (1000) can directly and significantly improve the outlier detection performance and quality based on the outlier detection model (ODM) according to the embodiment of the present invention.

[0290] In summary, the knowledge distillation-based anomaly detection learning method and system according to one embodiment of the present invention have the effect of providing an anomaly detection model (ODM) that performs more efficient data processing and further improves the task processing performance and quality for anomaly detection by performing patch feature-based learning on a predetermined pretrained model based on an image data set of anomaly detection targets.

[0291] In addition, the knowledge distillation-based anomaly detection learning method and system according to one embodiment of the present invention has the effect of improving the accuracy and efficiency of anomaly detection even in a limited learning environment, and simultaneously enhancing task processing performance accordingly, by performing patch feature-based learning that reduces the variance of mutually similar patch features and increases the difference between mutually heterogeneous patch features.

[0292]

[0293] [Knowledge Distillation-Based Anomaly Detection Learning Method]

[0294] Hereinafter, a method for providing a knowledge distillation-based anomaly detection learning service (i.e., an anomaly detection learning service) that performs adaptive update learning reflecting newly accumulated data while preserving previously learned information, using a computing system (1000) according to an embodiment of the present invention, is described in detail.

[0295] Generally, in the field of conventional anomaly detection, a technology is disclosed that detects defective data after training a model with predetermined good product data, which offers the advantage of enabling efficient training and rapid application.

[0296] However, these existing methods can lead to a decrease in outlier detection performance, such as an increase in over-checks (false positives) due to a gap between previously learned information and new data, when the feature distribution of data changes due to changes in the inspection environment or the emergence of new defect patterns.

[0297] To address this, retraining can be performed to reflect the changed data feature distribution, but this process has the limitation of losing previously learned information.

[0298] Accordingly, a computing system (1000) according to one embodiment of the present invention aims to provide a knowledge distillation-based anomaly detection learning method that performs adaptive update learning reflecting the feature distribution of newly accumulated data while preserving previously learned information.

[0299] Hereinafter, a knowledge distillation-based anomaly detection learning method according to an embodiment of the present invention will be described in more detail with reference to the attached drawings.

[0300] FIG. 9 is a flowchart illustrating a knowledge distillation-based anomaly detection learning method according to one embodiment of the present invention.

[0301] Referring to FIG. 9, a knowledge distillation-based anomaly detection learning method according to one embodiment of the present invention may include the steps of: acquiring a set of test sample images (S301); calculating an anomaly representative score based on the acquired set of test sample images (S303); determining whether to perform adaptive update learning based on the calculated anomaly representative score (S305); determining a defective candidate image based on the calculated anomaly representative score (S307); detecting defective candidate patch features based on the determined defective candidate image (S309); performing adaptive update adapter learning based on the detected defective candidate patch features (S311); performing third network learning based on the learned adaptive update adapter (S313); and performing memory bank updates based on the learned third network (S315).

[0302] Specifically, a computing system (1000) according to one embodiment of the present invention can acquire a set of test sample images. (S301)

[0303] Here, in other words, the test sample image according to the embodiment may refer to an image for which the presence or absence of anomalies is to be detected, that is, image data capturing the target for anomaly detection.

[0304] Specifically, in the embodiment, the computing system (1000) can acquire a plurality of test sample images (i.e., a set of test sample images) based on a predetermined user input and / or interaction with an external server.

[0305] In an example, the computing system (1000) can acquire a set of test sample images by accumulating and collecting test sample images over a predetermined period.

[0306] In addition, a computing system (1000) according to one embodiment of the present invention can calculate an anomaly representative score based on an acquired set of test sample images. (S303)

[0307] Specifically, in an embodiment, the computing system (1000) can calculate an anomaly score for each of a plurality of test sample images included in a test sample image set.

[0308] Here, the Anomaly Score according to the embodiment may refer to a numerical value (score) indicating how much each patch feature of a given image deviates from normal state data (i.e., possibility of defect).

[0309] That is, in the embodiment, the anomaly score can represent a value converted into a numerical value indicating how much the patch features of a predetermined test sample image differ from the data in a normal (good) state.

[0310] More specifically, in the embodiment, the computing system (1000) can calculate an anomaly score per test sample image by applying the description disclosed in steps S113 to S115 of the patch feature learning method for anomaly detection described above.

[0311] In other words, the computing system (1000) can obtain an anomaly score for each test sample image as described above during the process of obtaining reconstructed features for each test sample image and performing anomaly detection based on the obtained reconstructed features.

[0312] Additionally, in the embodiment, the computing system (1000) can obtain an anomaly representative score for each test sample image based on the anomaly scores for each acquired test sample image.

[0313] Here, the anomaly representative score according to the embodiment may refer to the anomaly score having the maximum value among the anomaly scores for each patch feature of a predetermined image.

[0314] That is, in the embodiment, the computing system (1000) can detect an anomaly score having the maximum value among the anomaly scores for each patch feature of a test sample image and set it as the representative anomaly score for the corresponding test sample image.

[0315] Thus, the computing system (1000) can obtain an anomaly representative score for each of the multiple test sample images included in the test sample image set.

[0316] In addition, a computing system (1000) according to one embodiment of the present invention can determine whether to perform adaptive update learning based on the calculated anomaly representative score. (S305)

[0317] Here, adaptive update learning according to the embodiment may refer to model learning based on knowledge distillation that reflects new data while maintaining previously learned information to a certain level. In the embodiment, adaptive update learning may include the process described below.

[0318] Specifically, in an embodiment, the computing system (1000) can determine whether to perform adaptive update learning for the aforementioned outlier detection model (ODM) using the anomaly representative score per test sample image calculated as above.

[0319] More specifically, in the embodiment, the computing system (1000) can perform data shift detection based on anomaly representative scores for each acquired test sample image.

[0320] FIG. 10 is an example of a drawing for explaining data shift detection according to an embodiment of the present invention.

[0321] Here, referring to FIG. 10, data shift detection according to the embodiment may mean a process of detecting a change in the feature distribution of data. This is intended to prevent performance degradation of the model by detecting a gap between information previously learned in the outlier detection model (ODM) and new data. In an embodiment of the present invention, when a data shift is detected, the computing system (1000) can continuously maintain and improve the performance of the model by performing learning based on the new data while reflecting such changes.

[0322] Specifically, in the embodiment, the computing system (1000) can compare each of the anomaly representative scores obtained as described above with a preset first threshold.

[0323] At this time, the computing system (1000) can calculate the number of anomaly representative scores having a value greater than or equal to a first threshold (hereinafter, the number of suspected anomalies).

[0324] And the computing system (1000) can perform adaptive update learning if the number of suspected cases of overexamination calculated is greater than or equal to a second threshold set in advance.

[0325] In other words, the computing system (1000) can perform adaptive update learning according to the process described below when the number of suspected cases is greater than or equal to a second threshold.

[0326] As such, in the embodiment, the computing system (1000) can effectively prevent the performance degradation of the model by automatically detecting the changing data distribution and performing retraining (update) that reflects it.

[0327] In addition, through this, the computing system (1000) can enable the model to adapt quickly and efficiently to a new data environment, thereby reducing resource requirements for maintenance and significantly improving the accuracy, reliability, and stability of outlier detection.

[0328] According to an embodiment, the computing system (1000) may perform adaptive update learning at preset intervals.

[0329] In addition, a computing system (1000) according to one embodiment of the present invention can determine a defective candidate image based on a calculated anomaly representative score. (S307)

[0330] Here, the defective candidate image according to the embodiment may mean a test sample image in which the anomaly representative score satisfies a first condition that has been pre-set.

[0331] In an example, the first condition may include a condition of being greater than or equal to a predetermined third threshold and / or a condition of being in the upper P1%, etc.

[0332] At this time, for example, the computing system (1000) may set the P1 value based on a statistical analysis process through experiments, or may set the P1 value based on a production yield calculated over a predetermined period. For example, assuming there is a process line with a daily production yield of 95%, the computing system (1000) may determine the top 5% of test sample images based on an anomaly representative score as defective candidate images.

[0333] That is, in the embodiment, the computing system (1000) can determine at least one test sample image as a defective candidate image in which the anomaly representative score is higher than a predetermined standard (high probability of defect).

[0334] In addition, a computing system (1000) according to one embodiment of the present invention can detect defective candidate patch features based on a determined defective candidate image. (S309)

[0335] Here, the defective candidate patch feature according to the embodiment may mean a patch feature having an anomaly score that satisfies a previously set second condition among the anomaly scores for each patch feature of the defective candidate image.

[0336] In an example, the second condition may include a condition of being greater than or equal to a predetermined fourth threshold and / or a condition of being in the top P2%, etc. For example, the computing system (1000) may set the P2 value based on a statistical analysis process through experiments.

[0337] That is, in the embodiment, the computing system (1000) can detect at least one patch feature having an anomaly score higher than a predetermined standard (high probability of containing a defect pattern) among the anomaly scores for each patch feature of the determined defect candidate image as a defect candidate patch feature.

[0338] Accordingly, the computing system (1000) can implement adaptive update learning that reflects the characteristic that outliers are more likely to occur in some areas than in the entire area of ​​a defective candidate image.

[0339] In addition, a computing system (1000) according to one embodiment of the present invention can perform adaptive update adapter learning based on detected defective candidate patch features. (S311)

[0340] FIG. 11 is a conceptual diagram illustrating an adaptive update adapter learning method according to one embodiment of the present invention.

[0341] Specifically, referring to FIG. 11, an outlier detection model (ODM) according to an embodiment of the present invention comprises a first adapter ( ) and the second adapter ( It may further include an adaptive update adapter including ).

[0342] Here, the first adapter according to the embodiment may refer to a network that learns new data while maintaining existing learning information at a certain level as an adapter that is learned in the current cycle.

[0343] This first adapter can perform learning that continuously improves the performance of the model by reflecting the output of the second adapter described below and the characteristics of the new data.

[0344] That is, in the embodiment, the first adapter can support the outlier detection model (ODM) to adapt to a data environment that changes over time by integrating previously learned information and new information based on new data.

[0345] In addition, the second adapter according to the embodiment may refer to a network in which the characteristics of past data are learned as an adapter learned in the previous cycle.

[0346] Meanwhile, to put it another way, the feature representation layer disclosed in FIG. 11 ( ) can refer to an initially trained network that learns patch features of good and defective data as shown in Figure 4, making the distance between feature vectors closer for similar features and the distance between feature vectors further apart for different features.

[0347] Also, to put it another way, the spatial projection layer disclosed in FIG. 11 ( ) is a layer that projects a feature representation according to a predetermined patch feature into a predetermined feature representation space, and in this embodiment, it may mean a network that projects a feature vector learned on the feature representation space of the previous cycle into the feature representation space of the current cycle.

[0348] Through this, the spatial projection layer enables the adaptive update adapter to learn efficiently by integrating new information while maintaining existing information to a certain level.

[0349] Specifically, in the embodiment, the computing system (1000) can configure a batch for model training based on a plurality of patch features based on a test sample image set (TIS).

[0350] In an example, the computing system (1000) may configure a configuration that includes at least some of the remaining patch features (GPF: hereinafter, good candidate patch features) excluding the defective candidate patch feature (FPF) detected as described above, in a predetermined ratio.

[0351] For example, the computing system (1000) may set a predetermined percentage (%) of the batch configuration to at least one defective candidate patch feature (FPF) randomly selected from a plurality of defective candidate patch features (FPF), and set the remaining percentage (%) of the batch configuration to at least one good candidate patch feature (GPF) randomly selected from a plurality of good candidate patch features (GPF).

[0352] Through this, the computing system (1000) can perform model learning to distinguish various characteristic differences between good products and defective products more clearly and precisely, thereby enhancing the accuracy and consistency of outlier detection.

[0353] Additionally, in an embodiment, the computing system (1000) can perform the first adapter learning described above based on the arrangement configured as above (i.e., at least one defective candidate patch feature (FPF) and at least one good candidate patch feature (GPF)).

[0354] Referring further to FIG. 4, in detail, in an embodiment, the computing system (1000) has a first loss function ( ), second loss function( ) and third loss function( The first adapter learning can be performed based on ).

[0355] More specifically, in an embodiment, the computing system (1000) can 1) measure a first loss function.

[0356] Here, the first loss function (Relaxed Contrastive Loss) according to the embodiment may refer to a loss function for feature representation learning that optimizes the distance between corresponding patch features based on the similarity between given patch features. Through this, the computing system (1000) can more clearly distinguish the feature difference between good patches and defective patches.

[0357] Specifically, in the embodiment, the computing system (1000) can calculate pairwise similarity and contextual similarity between patch features.

[0358] Additionally, the pairwise similarity according to the embodiment may refer to the similarity between the embedding vectors of two patch features.

[0359] In an embodiment, the computing system (1000) can calculate the pairwise similarity described above according to the aforementioned [Equation 1 - (a)], [Equation 1 - (b)] and [Equation 2].

[0360] Here, [Equation 1 - (a)] and [Equation 1 - (b)] represent the embedding vectors of each different patch feature, and σ in [Equation 2] represents the variance of the Gaussian distribution.

[0361] Additionally, contextual similarity according to the embodiment may refer to bidirectional similarity indicating how much the K nearest neighbor patch features overlap for each patch feature.

[0362] In an embodiment, the computing system (1000) can calculate the context similarity described above according to [Equation 3] and [Equation 4].

[0363] In this case, Nk in [Equation 3] and [Equation 4] refers to the k nearest neighbor patch features for patch feature z.

[0364] Here, according to an embodiment, the computing system (1000) can calculate the final context similarity according to the following [Equation 8] by considering the possibility that the context similarity calculated through [Equation 3] and [Equation 4] is asymmetric.

[0365] [Mathematical Formula 8]

[0366]

[0367] Additionally, in the embodiment, the computing system (1000) can calculate the Normalized Euclidean Distance between patch features.

[0368] Here, the normalized Euclidean distance according to the embodiment may refer to data obtained by calculating the straight-line distance (i.e., Euclidean distance) between different patch features and then adjusting the value to fit a certain range or condition.

[0369] In detail, in an embodiment, the computing system (1000) can calculate a normalized Euclidean distance as described above according to [Equation 9] below.

[0370] [Mathematical Formula 9]

[0371]

[0372] In addition, in the embodiment, the computing system (1000) can measure the first loss function described above according to [Equation 10] based on the pairwise similarity between patch features, contextual similarity, and normalized Euclidean distance calculated as above.

[0373] [Mathematical Formula 10]

[0374]

[0375] Here, the first term of [Equation 10] signifies that learning occurs in the direction where the normalized Euclidean distance between two patch features becomes closer, and the second term signifies that the normalized Euclidean distance between the two patch features It means that learning is done in a direction that moves away by that amount.

[0376] That is, in the embodiment, the computing system (1000) can measure a first loss function according to [Equation 10], in which the distance between two patch features becomes closer when the similarity between the two patch features is high and the distance between the two patch features becomes farther.

[0377] In addition, in the embodiment, the computing system (1000) can measure 2) a second loss function.

[0378] Here, the second loss function (Supervised Contrastive Loss) according to the embodiment is a loss function for feature representation learning that improves the class (i.e., good or defective) discrimination power of each patch feature, and may mean a loss function that trains patch features belonging to the same class to become closer to each other and patch features belonging to different classes to become farther apart from each other.

[0379] That is, in the embodiment, the second loss function may be a loss function that optimizes the distance between corresponding patch features according to the class to which each patch feature is classified.

[0380] Thus, in the embodiment, the computing system (1000) can strengthen the distinction between classes by minimizing the distance between patch features of the same class within a batch and maximizing the distance between patch features of different classes within a batch through a second loss function.

[0381] In detail, in an embodiment, the computing system (1000) can measure the second loss function described above according to [Equation 11] based on at least one good candidate patch feature (GPF) (i.e., a patch feature belonging to the good class) and at least one defective candidate patch feature (FPF) (i.e., a patch feature belonging to the defective class) in the batch.

[0382] [Mathematical Formula 11]

[0383]

[0384] Here, P(i) in [Equation 11] represents the set of samples belonging to the same class within the batch, and A(i) represents the set of all samples in the batch, and represents the Temperature Scaling parameter for model calibration.

[0385] In addition, in the embodiment, the computing system (1000) can measure 3) a third loss function.

[0386] Here, the third loss function (Knowledge Distillation Loss) according to the embodiment may refer to a loss function for feature representation learning that maintains previously learned patch feature information. Through this, the computing system (1000) can improve model performance by continuously reflecting new data while maintaining previously learned information to a certain level.

[0387] To this end, in the embodiment, the third loss function can perform feature representation learning that optimizes the learning similarity between patch features corresponding to the same patch.

[0388] Specifically, in an embodiment, the computing system (1000) has an existing learned patch feature (z) according to the following [Equation 12] i , that is, the patch features learned in the second adapter) and the newly learned patch features (z j , that is, when the newly learned patch features (that is, through the first adapter) represent the same patch, they are induced to be learned similarly to each other, and the cosine similarity between the two patch features can be calculated.

[0389] [Mathematical Formula 12]

[0390]

[0391] Thus, the computing system (1000) can measure a third loss function based on the cosine similarity calculated as above.

[0392] Continuing with further reference to FIG. 4, in the embodiment, the computing system (1000) can perform first adapter learning based on 4) the first loss function, the second loss function, and the third loss function measured as described above.

[0393] In detail, in an embodiment, the computing system (1000) can perform first adapter learning reflecting the first loss function, the second loss function, and the third loss function according to the following [Equation 13].

[0394] [Mathematical Formula 13]

[0395]

[0396] Additionally, in the embodiment, the computing system (1000) can utilize information learned through the first adapter in the current cycle through the second adapter in the next cycle (i.e., future cycle).

[0397] In this way, in the embodiment, the computing system (1000) can train the adaptive update adapter to adapt to new data representing a changed data distribution while maintaining existing knowledge to a certain level.

[0398] In other words, the computing system (1000) can build an outlier detection model (ODM) that achieves continuous performance improvement by flexibly learning new information accumulated over time while maintaining previously learned information.

[0399] Thus, the computing system (1000) can easily maintain outlier detection accuracy and consistency over the long term.

[0400] In addition, during this process, the computing system (1000) can further enhance its performance by strengthening the distinction between patch features used for learning.

[0401] In addition, a computing system (1000) according to one embodiment of the present invention can perform third network learning based on a learned adaptive update adapter. (S313)

[0402] FIG. 12 is a conceptual diagram illustrating a third network learning method according to an embodiment of the present invention.

[0403] Referring specifically to FIG. 12, the outlier detection model (ODM) according to an embodiment of the present invention may further include a third network (h).

[0404] Here, the third network according to the embodiment may mean a neural network that transforms the output feature of the second adapter corresponding to the same patch (hereinafter, the first patch) (i.e., the patch feature previously learned for the first patch) to be the same as the output feature of the first adapter (i.e., the patch feature newly learned for the first patch).

[0405] In an embodiment, the computing system (1000) can train the above-mentioned third network in a way that minimizes the difference in output between the first adapter and the second adapter.

[0406] Through this, the computing system (1000) can implement an adaptive update adapter that preserves patch feature information learned in the previous cycle (i.e., preserves information stored in the existing memory bank) while adapting to patch feature information according to new data learned in the current cycle (i.e., updates the memory bank by applying new information), thereby enabling the outlier detection model (ODM) to maintain consistently consistent performance.

[0407] In detail, in an embodiment, the computing system (1000) can compare the output of the second adapter with the output of the first adapter and perform third network learning to minimize the difference between the two outputs based on an L1 loss function.

[0408] Here, the L1 loss function described above can be adjusted by comparing the outputs of the second adapter and the first adapter so that the two adapters have matching output values ​​for the same patch (i.e., the first patch).

[0409]

[0410] Through this, the computing system (1000) can perform third network learning that converts the output features of the second adapter, which is previously learned for the first patch, to follow the output features of the first adapter, which is newly learned for the first patch.

[0411] That is, in the embodiment, the computing system (1000) can build a robust outlier detection model (ODM) that ensures a connection between existing learning information and new learning information, thereby providing stable performance even with changes in data distribution over time.

[0412] In addition, a computing system (1000) according to one embodiment of the present invention can perform a memory bank update based on a learned third network. (S315)

[0413] FIG. 13 is a conceptual diagram illustrating a memory bank update method according to one embodiment of the present invention.

[0414] Referring specifically to FIG. 13, in an embodiment, the computing system (1000) can update a memory bank (M) that stores existing learned patch feature information based on patch features converted through a third network and / or patch features obtained through a test sample image set (TIS) (in the embodiment, defective candidate patch features (FPF) and / or good candidate patch features (GPF)).

[0415] At this time, the memory bank (M) according to the embodiment may be implemented and operated in various embodiments, such as being included in the computing system (1000) or being implemented as a separate device. However, the most preferred embodiment may be one in which it is implemented through linkage with an anomaly detection model (ODM) on a predetermined anomaly detection device.

[0416] That is, the computing system (1000) can support an outlier detection model (ODM) that uses data from the memory bank (M) to effectively adapt to and operate in the new environment by updating the memory bank (M) to distill new information collected from the new environment on top of existing information and utilize it together.

[0417] Accordingly, the computing system (1000) can ensure continuous performance improvement and stability maintenance of the outlier detection model (ODM).

[0418] At this time, according to the embodiment, the computing system (1000) can perform the memory bank (M) update described above through coreset sampling.

[0419] In other words, for reference, core set sampling can be described as a process of extracting a set of representative samples that preserves the statistical characteristics or structure of the original data set as much as possible while reducing the size of the data set, as one of the methods for efficiently processing large data sets.

[0420] That is, in the embodiment, the computing system (1000) may select a set of representative samples having key features while maintaining diversity among the patch features in the memory bank (M), and perform an update of the memory bank (M) based on the selected representative samples.

[0421] Through this, the computing system (1000) can improve the efficiency and information density of the memory bank (M) by removing unnecessary information and preserving meaningful information.

[0422] FIG. 14 is a conceptual diagram illustrating an anomaly detection inference process using a learned outlier detection model (ODM) according to one embodiment of the present invention.

[0423] Referring to FIG. 14, thus, in an embodiment of the present invention, the computing system (1000) can perform anomaly detection according to the inference process of FIG. 14 on a predetermined test sample image (TI) using the memory bank (M) updated as above and the outlier detection model (ODM) on which adaptive update learning has been performed.

[0424] Accordingly, the computing system (1000) according to the embodiment can provide stable and improved anomaly detection performance even in a changing data environment.

[0425] In summary, the knowledge distillation-based anomaly detection learning method and system according to one embodiment of the present invention performs adaptive update learning that preserves previously learned information while reflecting newly accumulated data, thereby ensuring a connection between existing learning information and new learning information. This ensures that the model can provide a robust model that stably maintains anomaly detection accuracy and consistency even in a data environment that changes over time and continuously improves its performance.

[0426] In addition, the knowledge distillation-based anomaly detection learning method and system according to one embodiment of the present invention automatically detects data distributions that change over time and performs retraining (updates) reflecting them, thereby enabling the model to adapt quickly and efficiently to new data environments, which reduces resource requirements for maintenance and simultaneously facilitates the improvement of anomaly detection performance.

[0427]

[0428] Meanwhile, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be modified into one or more software modules to perform processing according to the present invention, and vice versa.

[0429] The specific embodiments described in this invention are examples and do not limit the scope of the invention in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as “essential,” “importantly,” etc., a component may not be strictly necessary for the application of the invention.

[0430] Furthermore, although the detailed description of the present invention has been explained with reference to preferred embodiments of the invention, those skilled in the art or those with ordinary knowledge in the relevant technical field will understand that various modifications and changes can be made to the invention without departing from the spirit and technical scope of the invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

[0431]

[0432] The present invention relates to a knowledge distillation-based anomaly detection learning method and system, and since it is applicable to the artificial intelligence industry, it has industrial applicability.

Claims

1. A method for a computing system including memory and a processor to perform knowledge distillation-based anomaly detection learning, wherein A step of acquiring multiple test sample images; A step of calculating an Anomaly Score, which is data quantifying the probability of defects for each patch feature for each of the plurality of test sample images; A step of determining a test sample image that satisfies a predetermined first condition based on the above anomaly score as a defective candidate image; A step of detecting a patch feature among the patch features of the above defective candidate image, wherein the anomaly score satisfies a predetermined second condition, as a defective candidate patch feature; A step of performing a first adapter learning responsible for learning the current cycle based on the above defective candidate patch features; and Based on the above learning, the method includes the step of updating a memory bank that stores learning information of past cycles of a predetermined outlier detection model. The first adapter is a network that performs learning in the current cycle, which reflects new information according to the plurality of test sample images while maintaining the learning information of the past cycle at a certain level. Knowledge distillation-based anomaly detection learning method.

2. In Paragraph 1, The step of acquiring the above plurality of test sample images is, A step of collecting accumulated test sample images over a specified period Knowledge distillation-based anomaly detection learning method.

3. In Paragraph 1, The step of calculating the above anomaly score is, The method includes the step of calculating a representative anomaly score for each of the plurality of test sample images based on the anomaly scores for each of the plurality of test sample images. The above anomaly representative score is the anomaly score having the maximum value among the above anomaly scores. Knowledge distillation-based anomaly detection learning method.

4. In Paragraph 3, The step of determining the above defective candidate image is, The above anomaly representative score includes the step of determining a test sample image satisfying the above predetermined first condition as a defective candidate image. Knowledge distillation-based anomaly detection learning method.

5. In Paragraph 1, The step of performing the above-mentioned first adapter learning is, A step comprising: a first loss function which optimizes the distance between said patch features based on a predetermined similarity between said patch features; a second loss function which optimizes the distance between said patch features based on a predetermined classification class for each patch feature; and a third loss function which optimizes the learning similarity between said patch features corresponding to the same patch. Knowledge distillation-based anomaly detection learning method.

6. In Paragraph 5, The step of performing the above-mentioned first adapter learning is, A step of configuring a batch comprising, at least some of the defective candidate patch features and the remaining patch features excluding the defective candidate patch features, which are good candidate patch features, according to the plurality of test sample images above, in a predetermined ratio; The method further includes the step of performing the first adapter learning based on the above-configured arrangement. Knowledge distillation-based anomaly detection learning method.

7. In Paragraph 6, The step of performing the first adapter learning based on the first loss function is, The method includes the step of performing the first adapter learning based on pairwise similarity, which is the similarity between the embedding vectors of each of the patch features, and contextual similarity, which is the similarity between the nearest neighbor patches of each of the patch features. Knowledge distillation-based anomaly detection learning method.

8. In Paragraph 7, The step of performing the first adapter learning based on the first loss function is, The method further includes the step of performing the first adapter learning based on the Euclidean distance that specifies the straight-line distance between the patch features. Knowledge distillation-based anomaly detection learning method.

9. In Paragraph 6, The step of performing the first adapter learning based on the second loss function is: A step comprising performing the first adapter learning to optimize the distinguishing power between a defect class corresponding to a defect candidate patch feature within the batch and a good product class corresponding to a good product candidate patch feature within the batch. Knowledge distillation-based anomaly detection learning method.

10. In Paragraph 6, The step of performing the first adapter learning based on the third loss function is: A step comprising: performing the first adapter learning based on the cosine similarity between the patch features learned in the past period and the patch features learned in the current period for a predetermined first patch. Knowledge distillation-based anomaly detection learning method.

11. In Paragraph 1, The method further includes the step of using the learning information of the current cycle learned through the first adapter as the learning information of the past cycle when learning the future cycle. Knowledge distillation-based anomaly detection learning method.

12. In Paragraph 1, The method further includes the step of performing third network learning based on the learning information of the past cycle and the output of the first adapter, and The above third network is a network that minimizes the difference between a patch feature according to the learning information of the past cycle and a patch feature according to the output of the first adapter for a predetermined first patch. Knowledge distillation-based anomaly detection learning method.

13. In Paragraph 12, The step of performing the above third network learning is, A step comprising minimizing the difference between the patch features based on an L1 loss function Knowledge distillation-based anomaly detection learning method.

14. In Paragraph 1, The step of updating the memory bank above is, A step of updating the memory bank based on coreset sampling Knowledge distillation-based anomaly detection learning method.

15. In Paragraph 1, The method further includes the step of determining whether to learn the knowledge distillation-based anomaly detection based on the anomaly scores for each of the plurality of test sample images. Knowledge distillation-based anomaly detection learning method.

16. At least one memory; and It includes at least one processor that reads at least one application stored in the memory and performs knowledge distillation-based anomaly detection learning; The instructions of the above processor are, A step of acquiring multiple test sample images, and A step of calculating an Anomaly Score, which is data quantifying the probability of defects for each of the plurality of test sample images above, and The step of determining a test sample image that satisfies a predetermined first condition with the above anomaly score as a defective candidate image, and A step of detecting a patch feature among the patch features of the above defective candidate image, wherein the anomaly score satisfies a predetermined second condition, as a defective candidate patch feature; and A step of performing a first adapter learning responsible for learning the current cycle based on the above defective candidate patch feature, and It includes a command that performs a step of updating a memory bank storing past cycle learning information of a predetermined outlier detection model based on the above learning, and The first adapter is a network that performs learning in the current cycle, which reflects new information according to the plurality of test sample images while maintaining the learning information of the past cycle at a certain level. Knowledge distillation-based anomaly detection learning system.

Images