Patch Feature Learning Method and System for Anomaly Detection

Abstract

The patch feature learning method and system for anomaly detection according to an embodiment of the present invention performs patch feature-based learning on a predetermined pretrained model based on an image dataset of anomaly detection targets.

Description

[Technical Field]

[0001] The present invention relates to a patch feature learning method and system for anomaly detection. More specifically, it relates to a method and system for performing patch feature-based learning on a predetermined pretrained model based on an image dataset of anomaly detection targets. [Background technology]

[0002] Anomaly detection refers to the process of identifying abnormal patterns, outliers, and / or exceptional values ​​from given data.

[0003] In other words, anomaly detection can be a process that detects elements that deviate from the attributes of normal data.

[0004] Systems that enable such anomaly detection are actively used in various application areas where the identification of abnormal patterns is important, such as process monitoring, security intrusion detection, fraud identification, and / or medical diagnosis.

[0005] However, there is a problem in achieving the performance of anomaly detection tasks based on this when the data required for training an anomaly detection model is relatively scarce, diverse, or insufficient, such as when it is difficult to collect abnormal data containing a specified defect, when the data with specified labels is limited, or when attempting to train a large amount of data without labels.

[0006] Furthermore, in conventional methods, particularly in the field of vision testing, anomaly detection based on specific images is actively performed. However, since such images belong to the category of high-dimensional data, there is a problem in that the data processing and computational costs are inefficient when all the data for the entire image is used at once to detect anomalies.

[0007] In addition, in most cases, outliers are observed as abnormal patterns of various sizes and shapes in small parts of the image, but conventional methods have the problem of having very low discriminatory power against such local patterns on the overall image.

[0008] Therefore, there is a need for the development of new technologies that can further improve the accuracy and efficiency of anomaly detection under limited environmental requirements, while simultaneously improving task processing performance. [Overview of the project] [Problems that the invention aims to solve]

[0009] One embodiment of the present invention aims to provide a patch feature learning method and system for anomaly detection, which performs patch feature-based learning on a predetermined pretrained model based on an image dataset of anomaly detection targets.

[0010] In this embodiment, the present invention aims to provide a patch feature learning method and system for anomaly detection that performs patch feature-based learning, which reduces the variance of mutually similar patch features and increases the difference between mutually heterogeneous patch features.

[0011] However, the technical problems that the present invention and its embodiments aim to solve are not limited to the technical problems described above, and other technical problems may exist.

Means for Solving the Problem

[0012] As a technical means for achieving the above-described technical problem, the patch feature learning method for anomaly detection according to an embodiment of the present invention is a method in which a computing system including a memory and a processor performs patch feature learning for anomaly detection, the method including: obtaining a feature map by a first image data set; obtaining a plurality of patch features based on local data in a predetermined image based on the obtained feature map; performing feature representation learning based on the obtained plurality of patch features; obtaining a re-con patch feature by the performed feature representation learning; and performing anomaly detection based on the obtained re-con patch feature, wherein the re-con patch feature is data obtained by reconstructing a feature representation by the plurality of patch features according to a similarity calculated based on the plurality of patch features.

[0013] In another aspect, the patch feature is a feature extracted from a patch that identifies at least a part of an area in a predetermined image.

[0014] In another aspect, the step of obtaining the plurality of patch features includes any one of: obtaining a plurality of patches with a predetermined patch size based on a first image included in the first image data set and extracting features for each of the obtained plurality of patches; and extracting features for the entire first image included in the first image data set and dividing the extracted overall features into a predetermined patch size.

[0015] On the other hand, the step of performing feature representation learning based on the plurality of patch features includes the step of performing semi-supervised concept learning.

[0016] On the other hand, the step of performing feature representation learning based on the plurality of patch features includes the step of performing the learning based on a first network which is a neural network for calculating similarity between predetermined features and a second network which is a neural network for realizing the feature representation learning.

[0017] On the other hand, each of the first network and the second network includes a feature representation layer which is a layer for reconstructing a feature representation by predetermined features and a space projection layer which is a layer for projecting a feature representation by predetermined features into a feature representation space.

[0018] On the other hand, the step of performing feature representation learning based on the plurality of patch features further includes the step of gradually distilling data corresponding to the intermediate variables of the second network into data corresponding to the intermediate variables of the first network based on an exponential moving average algorithm.

[0019] [[ID=]17] On the other hand, the step of performing feature representation learning based on the plurality of patch features further includes the step of projecting a first patch feature pair including a predetermined first patch feature and a second patch feature into the feature representation space, and the step of calculating pairwise similarity which is data for measuring similarity between the first patch feature and the second patch feature based on the first patch feature pair projected into the feature representation space.

[0020] In other respects, the step of performing feature representation learning based on the plurality of patch features further includes the step of calculating contextual similarity, which is data that measures the bidirectional similarity between K nearest neighbors to the first patch feature and K nearest neighbors to the second patch feature, based on the first patch feature pair projected onto the feature representation space.

[0021] In other respects, the step of performing feature representation learning based on the multiple patch features further includes the step of calculating integrated similarity, which is data obtained by linearly combining the calculated pairwise similarity and the contextual similarity.

[0022] In other respects, the step of performing feature representation learning based on the plurality of patch features further includes the step of training the second network based on the calculated integrated similarity.

[0023] In other respects, the step of training the second network based on the combined similarity includes training the second feature representation layer which maps the first patch feature and the second patch feature to each other in the feature representation space in accordance with the combined similarity.

[0024] In other respects, the step of performing the anomaly detection includes the steps of acquiring a first test sample image, acquiring the recompatch feature using the acquired first test sample image, and performing the anomaly detection based on the recompatch feature obtained by feature representation learning and the recompatch feature using the first test sample image.

[0025] In other respects, the step of performing the anomaly detection further includes the step of generating an anomaly score map based on the similarity between the recompatch features obtained by the feature representation learning and the recompatch features obtained by the first test sample image, and the step of performing the anomaly detection based on the generated anomaly score map.

[0026] In other aspects, a patch feature learning method for anomaly detection according to an embodiment of the present invention further includes the steps of coreset sampling of the recompatch features obtained by the feature representation learning, and performing the anomaly detection based on the coreset sampled recompatch features.

[0027] On the other hand, a patch feature learning system for anomaly detection according to an embodiment of the present invention includes at least one memory; and at least one processor that reads at least one application stored in the memory and performs patch feature learning for anomaly detection, wherein the processor acquires a feature map from a first image dataset, acquires multiple patch features based on local data in a predetermined image based on the acquired feature map, performs feature representation learning based on the acquired multiple patch features, acquires a reconpatch feature which is data in which the feature representation of the multiple patch features has been reconstructed according to the similarity calculated based on the multiple patch features, and performs anomaly detection based on the acquired reconpatch feature.

[0028] On the other hand, a computing device according to an embodiment of the present invention includes at least one memory; and at least one processor that reads at least one application stored in the memory and performs patch feature learning for anomaly detection, wherein the processor's instruction words include instructions for: acquiring a feature map from a first image data set; acquiring a plurality of patch features based on local data in a predetermined image based on the acquired feature map; performing feature representation learning based on the acquired plurality of patch features; acquiring a reconpatch feature, which is data obtained by the feature representation learning performed, which is data in which the feature representation of the plurality of patch features is reconstructed according to the similarity calculated based on the plurality of patch features; and performing anomaly detection based on the acquired reconpatch feature. [Effects of the Invention]

[0029] A patch feature learning method and system for anomaly detection according to one embodiment of the present invention has the effect of providing an anomaly detection model that performs patch feature-based learning on a predetermined pretrained model based on an image dataset of anomaly detection targets, thereby enabling more efficient data processing and further improving the task processing performance and quality for anomaly detection.

[0030] Furthermore, the patch feature learning method and system for anomaly detection according to one embodiment of the present invention has the effect of improving the accuracy and efficiency of anomaly detection even in a restrictive learning environment, while simultaneously enhancing task processing performance, by performing patch feature-based learning that reduces the variance of mutually similar patch features and increases the difference between mutually heterogeneous patch features.

[0031] However, the effects obtained by the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood from the following description. [Brief explanation of the drawing]

[0032] [Figure 1] An example block diagram of a computing system that implements a patch feature training service for anomaly detection according to one embodiment of the present invention is shown. [Figure 2] An example block diagram of a computing device that implements a patch feature training service for anomaly detection using one embodiment of the present invention is shown. [Figure 3] An example of a block diagram of other aspects of a computing device that implements a patch feature training service for anomaly detection according to one embodiment of the present invention is shown. [Figure 4] This is an example of a block flowchart for explaining an anomaly detection model according to one embodiment of the present invention. [Figure 5] This is a flowchart illustrating a patch feature learning method for anomaly detection according to one embodiment of the present invention. [Figure 6] This is a flowchart illustrating a patch feature-based feature representation learning method according to one embodiment of the present invention. [Figure 7] This is an example of a figure illustrating an example of measuring the similarity between patch features using one embodiment of the present invention. [Figure 8]This is an example of a diagram illustrating an application example of the ReConPatch Process according to one embodiment of the present invention. [Modes for carrying out the invention]

[0033] The present invention can be modified in various ways and has many different embodiments; therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, and how they are achieved, will become clear when you refer to the embodiments described in detail below, along with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be realized in many different forms. In the following embodiments, terms such as "first," "second," etc., are not restrictive and are used to distinguish one component from another. Also, singular expressions include plural expressions unless they have a clearly different meaning in context. Also, terms such as "includes" or "has" mean that the features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added. Also, the sizes of components may be exaggerated or reduced in the drawings for the sake of illustration. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for the sake of illustration, and the present invention is not necessarily limited to what is shown.

[0034] Embodiments of the present invention will be described in detail below with reference to the attached drawings. When referring to the drawings, identical or corresponding components will be given the same reference numerals, and redundant explanations related thereto will be omitted.

[0035] [An exemplary system providing patch-feature training services for anomaly detection]

[0036] The following describes in detail, with reference to the attached diagrams, an exemplary system for implementing a patch feature training service for anomaly detection, which performs patch feature-based learning on a predetermined pretrained model based on an image dataset of anomaly detection targets.

[0037] Figure 1 shows an example block diagram of a computing system that implements a patch feature training service for anomaly detection according to one embodiment of the present invention.

[0038] Referring to Figure 1, the computing system 1000 that realizes the patch feature training service for anomaly detection according to the present invention includes a user computing device 110, a server computing system 130, and a training computing system 150, the devices being able to communicate via a network 170.

[0039] The patch feature learning method for anomaly detection according to embodiments of the present invention can be 1) implemented and provided locally by a user computing device 110, 2) implemented and provided in web service format by a server computing system 130 communicating with the user computing device 110, or 3) implemented and provided by the user computing device 110 and the server computing system 130 working together.

[0040] In this embodiment, the user computing device 110 and / or the server computing system 130 can train the machine learning models 120 and / or 140 through interaction with the training computing system 150, which is communicatively connected via the network 170. The training computing system 150 may be separate from the server computing system 130, or it may be part of the server computing system 130.

[0041] In this case, the artificial intelligence model (in this embodiment, such as an anomaly detection model) can be trained in three ways: 1) directly locally by the user computing device 110; 2) by interaction between the server computing system 130 and the user computing device 110 via the network 170; or 3) by a separate training computing system 150 using various training and learning techniques. Furthermore, the training computing system 150 can transmit and provide / update the trained artificial intelligence model to the user computing device 110 and / or the server computing system 130 via the network 170.

[0042] In some embodiments, the training computing system 150 may be part of the server computing system 130 or part of the user computing device 110.

[0043] The user computing device 110 may include all other types of computing devices, such as smartphones, mobile phones, digital broadcasting devices, PDAs (personal digital assistants), PMPs (portable multimedia players), desktops, wearable devices, embedded computing devices, and / or tablet PCs.

[0044] Such a user computing device 110 includes at least one processor 111 and memory 112. Here, the processor 111 may consist of at least one or more electrically connected processors from among a central processing unit (CPU), graphics processing unit (GPU), ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions.

[0045] The memory 112 may include one or more non-temporary / temporary computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof, and may also include web storage of a server that performs memory storage functions over the internet. Such memory 112 can store data 113 and instruction words 114 necessary for at least one of the processors 111 to train an artificial intelligence model or to perform functional operations such as performing vision checks using the artificial intelligence model.

[0046] In one embodiment, the user computing device 110 can store at least one or more machine learning models 120.

[0047] More specifically, the machine learning model 120 may be a variety of machine learning models, such as multiple neural networks (e.g., deep neural networks), or other types of machine learning models including nonlinear and / or linear models, or a combination thereof.

[0048] In this case, the neural network may include at least one of the following: feed-forward neural networks, recurrent neural networks (e.g., long-short-term memory recurrent neural networks), convolutional neural networks, and / or other forms of neural networks.

[0049] In one embodiment, the user computing device 110 receives at least one or more machine learning models 120 from the server computing system 130 via the network 170, stores them in the memory 112, and then executes the stored machine learning models 120 using the processor 111 to perform anomaly detection and the like.

[0050] In another embodiment, the server computing system 130 can operate using at least one machine learning model 140 and communicate with the user computing device 110 and related data in a manner that allows it to work in conjunction with the user computing device 110 to provide the user with a patch feature training service for anomaly detection.

[0051] For example, the user computing device 110 can perform a patch feature training service for anomaly detection in which the server computing system 130 provides output for user input using a machine learning model 140 via the web.

[0052] Furthermore, the artificial intelligence model can also be realized in a manner in which at least a portion of the machine learning models 120 and / or 140 are executed on the user computing device 110, and the remainder is executed on the server computing system 130.

[0053] Furthermore, the user computing device 110 may include at least one input component 121 that senses user input. For example, the user input component 121 may include a touch sensor (e.g., a touchscreen and / or touchpad) that senses touch from the user's input medium (e.g., a finger or stylus), an image sensor that senses the user's motion input, a microphone that senses the user's voice input, buttons, a mouse and / or keyboard, etc. Also, if the user input component 121 receives input to an external controller (e.g., a mouse and / or keyboard) via an interface, it may include an interface and an external controller.

[0054] The server computing system 130 includes at least one processor 131 and memory 132. Here, the processor 131 may consist of at least one or more electrically connected processors from among central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), digital signal processors (DSSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions.

[0055] The memory 132 may also include one or more non-temporary / temporary computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and magnetic disks, or combinations thereof. Such a memory 132 can store data 133 and instruction words 134 necessary for the processor 131 to train an artificial intelligence model or to perform functional operations such as anomaly detection by the artificial intelligence model.

[0056] In one embodiment, the server computing system 130 may be implemented including at least one computing device. For example, the server computing system 130 may be implemented so that multiple computing devices operate in a sequential computing architecture, a parallel computing architecture, or a combination thereof. The server computing system 130 may also include multiple computing devices connected by a network 170.

[0057] Furthermore, the server computing system 130 can store at least one or more machine learning models 140. For example, the server computing system 130 may include neural networks and / or other multilayer nonlinear models as machine learning models 140. Exemplary neural networks may include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0058] The training computing system 150 includes at least one processor 151 and memory 152. Here, the processor 151 may consist of at least one or more electrically connected processors from among central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), digital signal processors (DSSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions.

[0059] The memory 152 may also include one or more non-temporary / temporary computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, and magnetic disks, or combinations thereof. Such a memory 152 can store data 153 and instruction words 154 necessary for the processor 151 to perform tasks such as training an artificial intelligence model.

[0060] For example, the training computing system 150 may include a model trainer 160 that trains machine learning models 120 and / or 140 stored in the user computing device 110 and / or server computing system 130 using a variety of training or learning techniques, such as backward propagation of errors (according to the framework shown in Figure 3).

[0061] For example, such a model trainer 160 can perform backpropagation-based updates to one or more parameters of the machine learning models 120 and / or 140 based on a defined loss function.

[0062] In some implementations, error back propagation may include truncated back propagation through time. The model trainer 160 can perform a number of generalization techniques (e.g., weight reduction, dropout, and / or knowledge distillation) to improve the generalization capabilities of the machine learning models 120 and / or 140 being trained.

[0063] In particular, the model trainer 160 can train the machine learning models 120 and / or 140 based on a series of training data 161. Here, the training data 161 may include data of different forms, such as images, audio samples and / or text. Examples of image types that can be used may include video frames, LiDAR point clouds, X-ray images, computed tomography scans, superspectroscopic images and / or various other forms of images.

[0064] Such training data 161 can be provided by the user computing device 110 and / or the server computing system 130. When the training computing device trains the machine learning models 120 and / or 140 on specific data from the user computing device 110, the machine learning models 120 and / or 140 can be characterized as personalized models.

[0065] Furthermore, the Model Trainer 160 includes computer logic that is used to provide the desired functionality.

[0066] The model trainer 160 may also be implemented as hardware, firmware, and / or software that control a general-purpose processor. In one implementation, the model trainer 160 includes a program file stored in a storage device, which is loaded into memory 152 and can be executed by one or more processors 151. In another implementation, the model trainer 160 includes one or more sets of computer-executable data 153 and instruction words 154 stored in a computer-readable storage medium such as a RAM hard disk or an optical or magnetic medium.

[0067] Network 170 includes, but is not limited to, 3GPP (3rd Generation Partnership Project) networks, LTE (Long Term Evolution) networks, WiMAX (World Interoperability for Microwave Access) networks, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth® networks, satellite broadcasting networks, analog broadcasting networks, and / or DMB (Digital Multimedia Broadcasting) networks.

[0068] Generally, communication over network 170 can be conducted using any type of wired and / or wireless connection, and can be carried out using various communication protocols (e.g., TCP / IP, HTTP, SMTP, and / or FTP), encoding or formatting (e.g., HTML and / or XML), and / or protection schemes (e.g., VPN, secure HTTP, and / or SSL).

[0069] Figure 2 shows an example block diagram of a computing device that implements a patch feature training service for anomaly detection according to one embodiment of the present invention.

[0070] Including Figure 2, the computing device 100 included in the user computing device 110, the server computing system 130, and the training computing system 150 contains numerous applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, applications may include image processing applications (e.g., Detection, Classification, and / or Segmentation), text messaging applications, email applications, writing applications, virtual keyboard applications, browser applications, and / or chatbot applications.

[0071] In one embodiment, the computing device 100 may include a model trainer 160 for training an artificial intelligence model, and by storing and operating the trained artificial intelligence model, it can provide output data based on predetermined input data (such as image data in one embodiment).

[0072] Each application of the computing device 100 can communicate with a number of other components of the computing device 100, such as at least one sensor, a context manager, a device status component, and / or additional components. In one embodiment, each application can communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0073] Figure 3 shows an example of a block diagram of another aspect of a computing device 100 that implements a patch feature training service for anomaly detection according to one embodiment of the present invention.

[0074] Referring to Figure 3, the computing device 300 includes a number of applications (e.g., Application 1 to Application N). Each application can communicate with the central intelligence layer. For example, applications may include an image processing application, a text messaging application, an email application, a writing application, a virtual keyboard application, and / or a browser application. In one embodiment, each application can communicate with the central intelligence layer (and the models stored therein) using an API (e.g., a common API across all applications).

[0075] The central intelligence layer may include multiple machine learning models. For example, as shown in Figure 3, at least a portion of each machine learning model may be provided to each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine learning model. For example, in some implementations, the central intelligence layer may provide a single model to all applications. In some implementations, the central intelligence layer may be implemented within or separately from the operational structure of the computing device 300.

[0076] The central intelligence layer can communicate with the central device data layer. The central device data layer may be a centralized data storage for the computing device 300. As shown in Figure 3, the central device data layer can communicate with many other components of the computing device 300, such as one or more sensors, a context manager, a device status component, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0077] The technologies described herein can refer not only to servers, databases, software applications, and other computer-based systems, but also to actions taken and information transmitted to or from such systems. The inherent flexibility of computer-based systems will be recognized as allowing for a wide range of possible configurations, combinations, and divisions of work, as well as functionality between and from components. For example, the processes described herein can be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications can be implemented in a single system or in a distributed system across multiple systems. Distributed components can operate sequentially or in parallel.

[0078] [Outlier Detection Model (ODM)]

[0079] Figure 4 is an example of a block flowchart illustrating an outlier detection model (ODM) according to one embodiment of the present invention.

[0080] Referring to Figure 4, the anomaly detection model (ODM) according to an embodiment of the present invention may mean an image deep learning model that performs anomaly detection based on a predetermined input image and classifies and / or recognizes the image based on this.

[0081] For reference, anomaly detection can be defined as the process of identifying abnormal patterns, outliers, and / or exceptional values ​​from specific data.

[0082] In other words, anomaly detection can be a process that detects elements that deviate from the attributes of normal data.

[0083] As an embodiment, anomaly detection can be implemented based on a method that groups predetermined data into clusters and considers points separated from the clusters as outliers.

[0084] Therefore, in this embodiment, the anomaly detection model (ODM) can determine whether a given input image contains specific abnormal attributes, and classify and / or recognize the image according to the determination result.

[0085] In the embodiments, the anomaly detection model (ODM) may include a first network (TN: Teacher Network) and a second network (SN: Student Network).

[0086] More specifically, the first network (TN) in the embodiment may mean a neural network that calculates similarity between predetermined features (in the embodiment, patch features).

[0087] In some embodiments, such a first network (TN) may include a first feature representation layer (f(·)) and a first spatial projection layer (g(·)).

[0088] Here, the feature representation layer according to the embodiment may mean a layer that reconfigures (adjusts) the feature in a way that improves the performance of the feature representation by a predetermined feature (in the embodiment, a patch feature).

[0089] To this end, the feature representation layer can be trained to extract meaningful features from a given feature (in this embodiment, a patch feature) with greater accuracy and to reconstruct (adjust) the relevant feature based on this.

[0090] Furthermore, the spatial projection layer in the embodiment may mean a layer that projects a feature representation by a predetermined feature (a patch feature in the embodiment) into a predetermined feature representation space.

[0091] In one embodiment, such a spatial projection layer can be trained to project feature representations of predetermined features (in one embodiment, patch features) into a feature representation space in which the goals of model learning can be more effectively applied.

[0092] On the other hand, the second network (SN) in the embodiment may refer to a neural network that implements feature representation learning.

[0093] For reference, feature representation learning can be defined as the process by which a deep learning model automatically detects and learns useful features from given data.

[0094] Feature representation learning enables deep learning models to effectively encode useful information contained in data, generate meaningful features usable in a variety of deep learning tasks, understand the complex structure and patterns of the data, and make accurate predictions based on this understanding.

[0095] In this embodiment, the second network (SN) that realizes the feature representation learning described above may include a second feature representation layer (f(·)) and a second spatial projection layer (g(·)).

[0096] In this embodiment, the first feature representation layer (f(·)) and the second feature representation layer (f(·)), and the first spatial projection layer (g(·)) and the second spatial projection layer (g(·)) are used to distinguish and explain the feature representation layers and spatial projection layers included in the first network (TN) and the feature representation layers and spatial projection layers included in the second network (SN). Therefore, the explanation of the feature representation layers and spatial projection layers of the second network (SN) follows the explanation of the feature representation layers and spatial projection layers of the first network (TN) described above.

[0097] Furthermore, a more detailed explanation of the first network (TN) and the second network (SN) according to the embodiment will be described later in the section on the patch feature learning method for anomaly detection.

[0098] In another embodiment, the anomaly detection model (ODM) can perform various functional operations necessary for a patch feature training service for anomaly detection in conjunction with a model pre-trained to perform concept learning (hereinafter referred to as the pre-trained model).

[0099] Here, the pre-trained model according to the embodiment may be an Image Deep-learning Model that has been pre-trained to perform concept learning based on a predetermined training dataset (e.g., a predetermined natural image dataset and / or a predetermined normal image dataset).

[0100] For reference, concept learning can be defined as the process of inferring general rules, concepts, or patterns from given data and classifying them.

[0101] In other words, in this embodiment, the pre-trained model may be an image deep learning model that takes a predetermined image as input data, learns common features of objects and patterns in the input image, groups image components having similar features based on this, and supports the classification or recognition of a specific image based on this.

[0102] Specifically, a pre-trained model can 1) extract feature maps for a given input image.

[0103] More specifically, a pre-trained model can automatically extract feature maps based on the original pixel data of an input image using a predetermined image deep learning neural network (for example, a Convolutional Neural Network (CNN)).

[0104] Such feature maps can show a variety of visual attributes of the image, such as edges, color, and / or texture.

[0105] Furthermore, the pre-trained model can perform 2) crowding based on the feature space.

[0106] More specifically, the pre-trained model can classify the input image and / or objects within the input image into groups with similar features based on the feature maps extracted as described above.

[0107] In one embodiment, the pre-trained model can classify the extracted feature maps by feature space using a predetermined clustering algorithm (e.g., K-means, DBSCAN, and / or hierarchical clustering algorithms) or dimensionality reduction algorithm (e.g., t-SNE and / or UMAP).

[0108] Furthermore, the pre-trained model can 3) assign labels to each group.

[0109] In other words, a pre-trained model can define a representative concept (hypothesis) for each group and set it as a label for that group.

[0110] In this process, the pre-trained model can manually or semi-automatically assign labels to each group and learn the characteristics of images belonging to specific concepts or categories.

[0111] Furthermore, the pre-trained model can 4) test and adjust the assigned hypotheses.

[0112] In more detail, the pre-trained model can verify the initially defined concepts (clustered feature groups) as described above and adjust the hypotheses as needed.

[0113] In this process, the pre-trained model can use new image data to detect and improve data that has been mis-crowded.

[0114] Furthermore, the pre-trained model can be run by repeating the process described in 5) above.

[0115] In other words, a pre-trained model can learn to classify or recognize images more accurately by continuously improving its understanding of crowded features and associated concepts through new image data and additional feedback.

[0116] In this embodiment, the pre-trained model may be directly included in the Outlier Detection Model (ODM) or implemented as a separate device and / or server from the Outlier Detection Model (ODM).

[0117] In the following explanation, the pre-trained model is described as being implemented as part of the Outlier Detection Model (ODM), but is not limited to this.

[0118] Furthermore, in Figure 4, in order to prevent ambiguity regarding the features of the embodiment of the present invention, it is explained that the ODM includes the components described above. However, it is obvious to a person of ordinary skill in the art that, depending on the embodiment, other general-purpose components may be included in addition to those shown in Figure 4, or some of the components shown in Figure 4 may be omitted.

[0119] [Patch Feature Learning Method for Onomatopoeia Detection]

[0120] The following describes in detail how a computing system 1000 according to an embodiment of the present invention realizes a patch feature training service for anomaly detection, which performs patch feature-based learning on a predetermined pretrained model based on an image dataset of anomaly detection targets.

[0121] The patch feature learning method for anomaly detection of a computing system 1000 according to an embodiment of the present invention can improve the performance and quality of various anomaly detection-based services by utilizing an anomaly detection model (ODM) trained according to an embodiment of the present invention.

[0122] In this case, the patch feature learning method for anomaly detection of the computing system 1000 according to an embodiment of the present invention can effectively provide an improved performance anomaly detection model (ODM) by performing patch feature-based learning that reduces the variance of similar patch features and increases the difference between heterogeneous patch features.

[0123] The patch feature learning method for anomaly detection according to embodiments of the present invention will be described in more detail below with reference to the attached drawings.

[0124] Figure 5 is a flowchart illustrating a patch feature learning method for anomaly detection according to one embodiment of the present invention.

[0125] Referring to Figures 4 and 5, the patch feature learning method for anomaly detection according to an embodiment of the present invention may include the steps of: acquiring a feature map based on a pretrained model (S101); extracting a plurality of patch features based on the acquired feature map (S103); performing feature representation learning based on the extracted plurality of patch features (S105); acquiring reconpatch features based on the performed feature representation learning (S107); performing coreset sampling based on the acquired reconpatch features (S109); acquiring a test sample image (S111); acquiring reconpatch features based on the acquired test sample image (S113); and performing anomaly detection based on the acquired reconpatch features (S115).

[0126] Specifically, the computing system 1000 according to an embodiment of the present invention acquires a feature map based on a pretrained model (S101).

[0127] In this embodiment, the computing system 1000 can acquire a feature map of a predetermined anomaly detection target using an image dataset (hereinafter referred to as the target image dataset) by using a model that has been pre-trained to perform concept learning (i.e., a pre-trained model).

[0128] In other words, the pre-trained model according to the embodiment may be an Image Deep-learning Model that has been pre-trained to perform concept learning based on a predetermined training dataset (e.g., a predetermined natural image dataset and / or a predetermined normal image dataset).

[0129] In other words, in this embodiment, the computing system 1000 can acquire a feature map based on a predetermined target image dataset (i.e., an image dataset containing multiple images of a predetermined anomaly detection target) in conjunction with the pre-trained model described above.

[0130] In more detail, in this embodiment, the computing system 1000 can input a target image dataset (for example, an image dataset containing multiple images of a given electronic circuit element) into a pre-trained model.

[0131] In this way, the pre-trained model can output a feature map of the input target image dataset and provide it to the computing system 1000.

[0132] This allows the computing system 1000 to acquire a feature map corresponding to the target image dataset.

[0133] In addition, in this embodiment, the computing system 1000 extracts multiple patch features based on the acquired feature map (S103).

[0134] Here, a patch feature according to the embodiment may mean a feature extracted from a patch that represents a small portion of a given image.

[0135] Specifically, the patch may be a rectangular region representing a particular part of a given image. Such a patch is a subset containing some of the information of the overall image, and mainly includes local information and texture information.

[0136] Furthermore, the features are characteristic information extracted from a predetermined image or patch, and can summarize or represent important attributes of the image (e.g., pattern, texture, color, and / or shape).

[0137] Therefore, the patch feature may be data that represents local attributes within a given image on a patch-by-patch basis.

[0138] In more detail, in this embodiment, the computing system 1000 can extract multiple patch features based on the feature map acquired as described above.

[0139] More specifically, in one embodiment, the computing system 1000 can divide a predetermined image (hereinafter referred to as the target training image) contained in the target image dataset into predetermined patch size units before inputting the target training image into the aforementioned pre-trained model.

[0140] The computing system 1000 can then input each divided patch into a pre-trained model to obtain a corresponding feature map for each patch.

[0141] In other words, the computing system 1000 can divide the target training image into predetermined patch size units, input them into a pre-trained model, and then obtain a feature map for each patch.

[0142] In this embodiment, the computing system 1000 can perform coreset sampling on the acquired patch-specific feature maps.

[0143] For reference, core set sampling is one method for efficiently processing large datasets, and it can be said to be a process of extracting a representative set of samples that preserves the statistical characteristics and structure of the original dataset to the greatest extent possible while reducing the size of the dataset.

[0144] As an embodiment, the computing system 1000 can perform core set sampling using an approximation algorithm that maintains the characteristics of the original dataset within a predetermined error range, taking into account the distribution of the given data, while simultaneously selecting a portion of the samples that can represent the entire dataset.

[0145] Alternatively, the computing system 1000 can perform core set sampling using an importance sampling method that assigns sampling probabilities based on the importance of each given data point and preferentially selects data points with high importance.

[0146] This allows the computing system 1000 to acquire multiple patch features on which core set sampling has been performed.

[0147] On the other hand, in another embodiment, the computing system 1000 can acquire a feature map (hereinafter referred to as the full feature map) for the entire region of the target training image.

[0148] The computing system 1000 can then divide the acquired overall feature map into predetermined patch size units.

[0149] This allows the computing system 1000 to extract multiple patch features from the target training image.

[0150] Thus, in this embodiment, the computing system 1000 can extract multiple patch features from the target training image using at least one of the methods described above.

[0151] In this embodiment, the computing system 1000 can aggregate surrounding feature vectors within a specific patch size and extract each patch feature.

[0152] Alternatively, depending on the embodiment, the computing system 1000 may use the pixel values ​​within each patch themselves as features.

[0153] Alternatively, by some embodiment, the computing system 1000 may use a statistical summary (e.g., mean, variance, and / or histogram, etc.) for the pixel values ​​within each patch as a feature.

[0154] Alternatively, depending on the embodiment, the computing system 1000 may analyze the texture patterns within each patch and use them as features. For example, the computing system 1000 may extract texture-based patch features using techniques such as Gabor filters, LBP (Local Binary Patterns), and / or HOG (Histogram of Oriented Gradients).

[0155] Alternatively, depending on the embodiment, the computing system 1000 may utilize a deep learning algorithm such as a convolutional neural network (CNN) to automatically learn and extract high-dimensional features within each patch for use as features.

[0156] Thus, in this embodiment, the computing system 1000 enhances processing efficiency during the data learning and analysis process by extracting features at the patch level and supporting anomaly detection using these features, while simultaneously enabling more detailed detection of abnormal local patterns that mainly appear in small parts of the image.

[0157] In addition, in this embodiment, the computing system 1000 performs feature representation learning based on the extracted patch features (S105).

[0158] In other words, feature representation learning can be defined as the process by which a deep learning model (in some embodiments, an outlier detection model (ODM)) automatically detects and learns useful features from given data.

[0159] In more detail, in the embodiment, the computing system 1000 can learn feature representations for the multiple patch features extracted as described above based on the anomaly detection model (ODM) according to the embodiment of the present invention.

[0160] In other words, the computing system 1000 can perform feature representation learning for an anomaly detection model (ODM) that performs anomaly detection based on the extracted patch features.

[0161] In other words, anomaly detection can be defined as the process of identifying abnormal patterns, outliers, and / or exceptional values ​​from specific data; that is, the process of detecting components that deviate from the attributes of normal data.

[0162] Therefore, in this embodiment, the computing system 1000 can determine whether a given input image contains specific abnormal attributes, and based on the result of the determination, perform feature representation learning (such as concept learning in this embodiment) to classify and / or recognize the image based on the aforementioned multiple patch features.

[0163] In this embodiment, the computing system 1000 can perform the aforementioned feature representation learning based on a semi-supervised learning method.

[0164] In other words, the computing system 1000 can build an anomaly detection model (ODM) that enables semi-supervised learning-based anomaly detection.

[0165] For reference, semi-supervised learning can be defined as a deep learning method that trains a model using not only data with labels (i.e., training data) but also data without labels (i.e., untrained data).

[0166] Generally, when collecting foundational data for building an anomaly detection system, it is difficult to acquire a sufficient amount of abnormal data for smooth learning (for example, image data of the abnormal state of the target of anomaly detection), which may limit anomaly detection learning that attempts to recognize diverse shapes of abnormal states (anomalies) with high accuracy.

[0167] Therefore, in embodiments of the present invention, a pre-trained model (i.e., a pre-trained model) is constructed mainly using normal data (for example, image data of the normal state of the object to be detected for anomaly detection), and a semi-supervised learning-based anomaly detection is realized that performs anomaly detection based on pseudo-labels using the said pre-trained model.

[0168] For reference, pseudo-labels can refer to labels predicted by a trained model on unlabeled data.

[0169] Such pseudo-labels can be used primarily when the labeled data is limited or when a large amount of unlabeled data is used to train a model.

[0170] This allows the computing system 1000 in this embodiment to easily achieve model learning and performance improvement for building an anomaly detection process, even when the labeled data is relatively scarce or diverse and limited.

[0171] More specifically, in the embodiment, the computing system 1000 can perform feature representation learning using multiple patch features based on the first network (TN) and second network (SN) of the anomaly detection model (ODM).

[0172] Figure 6 is a flowchart illustrating a patch feature-based feature representation learning method according to one embodiment of the present invention.

[0173] Specifically, referring to Figure 6, in one embodiment, the computing system 1000 projects an arbitrary patch feature pair into a predetermined feature representation space (S201).

[0174] In the embodiment, the computing system 1000 has an arbitrary first patch feature (Pi) and a second patch feature (P j A patch feature pair consisting of a pair of ) (hereinafter referred to as the first patch feature pair) can be projected onto the feature representation space.

[0175] As an embodiment, the computing system 1000 represents the first patch feature (Pi) projected onto the feature representation space as shown in "Equation 1 (a)" below, and the second patch feature (P) projected onto the feature representation space j ) can be expressed as shown in "Number 1 (b)" below.

[0176]

number

[0177] In addition, in this embodiment, the computing system 1000 calculates pairwise similarity based on patch feature pairs projected onto the feature representation space (S203).

[0178] Here, pairwise similarity according to the embodiment is defined as the first patch feature (P) that any patch feature pair contains. i ) and the second patch feature (P j This could mean data measuring the similarity between )

[0179] In other words, in this embodiment, the computing system 1000 includes the first patch feature (P) that the first patch feature pair contains i ) and the second patch feature (P j Pairwise similarity, which indicates the similarity between ) can be measured.

[0180] In this embodiment, the computing system 1000 can calculate the aforementioned pairwise similarity according to the following [Equation 2].

[0181]

number

[0182] Figure 7 is an example of a diagram showing an example of similarity measurement between patch features using one embodiment of the present invention.

[0183] However, referring to FIG. 7, when measuring similarity only in terms of the relationship between a first patch feature (P i ) and a second patch feature (P j ) included in any patch feature pair, the pairwise similarity is the same. However, when the first feature and the second feature must be classified into different labels from each other as shown in (a) of FIG. 7 (i.e., when they are further separated from each other and approach the correct label), and when the first feature and the second feature must be classified into the same label from each other as shown in (b) of FIG. 7 (i.e., when they get closer and approach the correct label), the discrimination degree may decrease.

[0184] In other words, when only measuring the pairwise similarity, there is a possibility that the accuracy decreases when performing label prediction without considering the mutual similarity in the group relationship including the K nearest neighbors N i (i) for the first patch feature (P k ) and the K nearest neighbors N j (j) for the second patch feature (P k ).

[0185] Thus, in an embodiment, the computing system 1000 calculates a contextual similarity based on the patch feature pairs projected into the feature representation space (S205).

[0186] Here, the contextual similarity according to the embodiment may mean data obtained by measuring the bidirectional similarity between the K nearest neighbors N i (i) for the first patch feature (P k ) and the K nearest neighbors N j (i) for the second patch feature (P k ) included in any patch feature pair.

[0187] At this time, in the embodiment, the bidirectional similarity is the K nearest neighbor features N i for the first patch feature (P k(i)) and the second patch feature (P j K nearest neighbor features (N) for ) k It can be calculated based on the average similarity between (j)).

[0188] In more detail, in this embodiment, the computing system 1000 can calculate the aforementioned contextual similarity using the K-Nearest Neighbors (K-NN) algorithm, which makes predictions based on the distance between data points as shown in [Equation 3] below, and [Equation 4].

[0189]

number

[0190]

number

[0191] In other words, in this embodiment, the computing system 1000 includes the first patch feature (P) that the first patch feature pair contains i ) and the second patch feature (P j The more neighbors (or elements) share in common, the higher the contextual similarity can be considered to be.

[0192] Thus, the computing system 1000 has a first patch feature (P i ) and the second patch feature (P j ) can be learned to recognize feature representations in group relationships and incorporate them into the pseudo-label prediction process.

[0193] In other words, the computing system 1000 is the first patch feature (P i ) and the second patch feature (P jBy extracting the K closest feature samples from the set, calculating contextual similarity to measure how many samples intersect, and using this along with pairwise similarity to train an anomaly detection model (ODM), the trained ODM can extract features of better quality.

[0194] This allows the computing system 1000 to more accurately determine whether the pairwise similarity between the first and second features is the same, but the first and second features must be classified into different labels (i.e., they should be further apart to approach the correct label) or whether they should be classified into the same label (i.e., they should be closer to each other to approach the correct label), and to reflect this in pseudo-label prediction. Therefore, the computing system 1000 can directly improve the task processing quality and performance of anomaly detection models (ODMs) that perform semi-supervised learning-based anomaly detection.

[0195] Returning to the previous example, the computing system 1000 also calculates an Integrated Similarity based on the pairwise similarity and contextual similarity calculated as described above (S207).

[0196] Here, the integrated similarity according to the embodiment is the first patch feature (P) that any patch feature pair contains. i ) and the second patch feature (P j This could mean data that combines pairwise similarity and contextual similarity between the two.

[0197] In detail, in the embodiment, the computing system 1000 calculates the first patch feature (P) contained in the first patch feature pair according to [Equation 5] below. i ) and the second patch feature (P j The pairwise similarity and contextual similarity between the two can be linearly combined.

[0198]

number

[0199] In this case, the integrated similarity according to the embodiment can be defined as a linear combination of two similarities satisfying "α∈[0,1]".

[0200] In addition, in this embodiment, the computing system 1000 trains the second network (SN) of the anomaly detection model (ODM) based on the calculated integrated similarity (S209).

[0201] In other words, in this embodiment, the computing system 1000 can perform second network (SN) learning that realizes feature representation learning using integrated similarity.

[0202] In detail, in this embodiment, the computing system 1000 uses the integrated similarity calculated for the first patch feature pair as the relaxed contrast loss (L RC This can be applied to train a second network (SN).

[0203] Here, the relaxation contrast loss (L) according to the embodiment RC ) is as shown in [Equation 6] below.

[0204]

number

[0205] Here, in [Equation 6], "z" is the embedding vector inferred by "g(f(p))", "N" is the mini-batch, i.e., the number of patch instances, "m" is the repelling margin, and "w ij This can be a parameter that determines the weighting of the induction and rebound loss terms.

[0206] Figure 8 is an example of a diagram illustrating the application of the ReConPatch Process according to one embodiment of the present invention.

[0207] Referring to Figure 8, in this embodiment, the computing system 1000 uses integrated similarity to determine the first patch feature (P i ) and the second patch feature (P j If the first patch feature (P) contained in the positive feature pair is determined to be a patch feature pair that should be classified into different labels (hereinafter referred to as a positive feature pair), then the first patch feature (P) contained in the positive feature pair i ) and the second patch feature (P j The second feature representation layer (f(·), embedding function) of the second network (SN) can be trained to map the features of the second network (SN) so that they are separated from each other in the feature representation space.

[0208] On the other hand, in the embodiment, the computing system 1000 uses integrated similarity to determine the first patch feature (P i ) and the second patch feature (P j If the patch feature pairs (hereinafter referred to as negative feature pairs) that should be classified as having the same label are determined to be the first patch feature (P) included in the negative feature pair i ) and the second patch feature (P j The second feature representation layer (f(·), embedding function) of the second network (SN) can be trained to map the features of the second network (SN) to be closer to each other in the feature representation space.

[0209] Thus, in this embodiment, the computing system 1000 can train the second feature representation layer (f(·)) to reduce the variance of similar patch features and increase the difference of heterogeneous patch features.

[0210] In other words, the computing system 1000 can be trained so that the second feature representation layer (f(·)) extracts arbitrary patch features in a form that is closer to the correct pseudo-label.

[0211] Thus, in this embodiment, the computing system 1000 can directly improve the feature representation performance of the anomaly detection model (ODM) itself by training the second feature representation layer (f(·)) to more accurately extract significant features from arbitrary patch features, and at the same time improve the processing quality of various tasks (such as anomaly detection in this embodiment) based on this performance.

[0212] Furthermore, in this embodiment, the computing system 1000, which has trained a second network (SN) based on integrated similarity, can also train the first network (TN) of the anomaly detection model (ODM).

[0213] In detail, in this embodiment, the computing system 1000 calculates the parameter (θ) of the second network (SN) according to the following [Equation 7] using the exponential moving average (EMA) method. f,g The data corresponding to the parameters (θ) of the first network (TN) f￣,g￣ The data can be gradually distilled according to the specified criteria.

[0214]

number

[0215] In other words, in this embodiment, the computing system 1000 can perform first network (TN) learning of the anomaly detection model (ODM) by gradually distilling the information learned in the second network (SN) into the first network (TN) according to the aforementioned [Equation 7].

[0216] In this embodiment, the computing system 1000 can perform the first network (TN) learning described above by further applying an update rate adjustment variable, which is a variable that adjusts the rate of information distillation.

[0217] As a result, in this embodiment, the computing system 1000 can implement multiple patch feature-based feature representation learning based on the first network (TN) and second network (SN) of the anomaly detection model (ODM).

[0218] Thus, in this embodiment, the computing system 1000 can perform a process (in this embodiment, a ReConPatch Process) to construct discriminant features for anomaly detection by distilling the main features of the dataset for anomaly detection targets into a pre-trained model based on semi-supervised learning, as described above.

[0219] This allows the computing system 1000 to build a high-performance anomaly detection model (ODM) that has been trained to more accurately classify the relevant features at a given patch level into ground truth pseudo-labels.

[0220] Therefore, the computing system 1000 can directly and effectively improve the processing performance and quality of various tasks (such as anomaly detection in the embodiment) that utilize the anomaly detection model (ODM).

[0221] Returning to Figure 5, in this embodiment, the computing system 1000 acquires a ReConPatch Feature through the feature representation learning performed (S107).

[0222] Here, the recompatch feature according to this embodiment may mean a patch feature output via the feature representation layer (hereinafter referred to as the recompatch layer) on which the aforementioned feature representation learning has been performed.

[0223] In other words, in this embodiment, the reconpatch feature may be a patch feature output based on the second feature representation layer (f(·)) of the second network (SN) on which feature representation learning has been performed.

[0224] In more detail, in this embodiment, the computing system 1000 can work in conjunction with the recompatch layer to acquire recompatch features for each target training image included in the aforementioned target image dataset.

[0225] In other words, in this embodiment, the computing system 1000 can obtain a recompated feature dataset (hereinafter referred to as the recompated training dataset) in which the variance of similar patch features is reduced and the differences between heterogeneous patch features are increased for multiple patch features extracted from each target training image.

[0226] In addition, in this embodiment, the computing system 1000 performs coreset sampling based on the acquired recompatch features (S109).

[0227] In other words, core set sampling according to the embodiment is one method for efficiently processing large datasets, and can be said to mean a process of extracting a representative set of samples that preserves the statistical characteristics and structure of the original dataset to the greatest extent possible while reducing the size of the dataset.

[0228] In more detail, as an embodiment, the computing system 1000 can perform core set sampling using an approximation algorithm that maintains the characteristics of the original recompatched learning dataset within a predetermined error range, taking into account the distribution of the acquired recompatched learning dataset, while simultaneously selecting a portion of the samples that can represent the entire recompatched learning dataset.

[0229] In another embodiment, the computing system 1000 can perform core set sampling using an importance sampling method that assigns sampling probabilities based on the importance of each acquired recompatch feature data and preferentially selects data points with high importance.

[0230] This allows the computing system 1000 to obtain a recompatched training dataset (hereinafter referred to as the recompatched sampling dataset) that has undergone core set sampling.

[0231] In addition, in this embodiment, the computing system 1000 can store and manage the acquired recompatch sampling dataset on a predetermined database.

[0232] This allows the computing system 1000 to detect anomalies with high accuracy while reducing data processing costs.

[0233] In another embodiment, the computing system 1000 acquires a test sample image (S111).

[0234] Here, the test sample image according to the embodiment may mean an image used to detect the presence or absence of anomalies, that is, image data of an object for which anomalies are to be detected.

[0235] In more detail, in the embodiment, the computing system 1000 can acquire the aforementioned test sample image based on predetermined user input and / or interaction with an external server.

[0236] In addition, in this embodiment, the computing system 1000 acquires recompatch features using the acquired test sample image (S113).

[0237] In the following, any information that overlaps with the explanation above may be summarized or omitted.

[0238] In more detail, in this embodiment, the computing system 1000 can input the acquired test sample images into a pre-trained model.

[0239] Furthermore, in this embodiment, the computing system 1000 can obtain a feature map for the test sample image from a pre-trained model into which the test sample image has been input. A detailed explanation of this will be provided by following the description of step S101 above.

[0240] Furthermore, in this embodiment, the computing system 1000 can extract multiple patch features based on the acquired feature map. A detailed explanation of this will be provided by following the description of step S103 above.

[0241] In addition, in this embodiment, the computing system 1000 can input the extracted patch features into the aforementioned recompation layer.

[0242] This allows the recompatch layer to output multiple recompatch features corresponding to the multiple patch features input.

[0243] Therefore, in this embodiment, the computing system 1000 can obtain a recompatched feature dataset (hereinafter referred to as the recompatched target dataset) in which the distribution of similar patch features is reduced and the differences between heterogeneous patch features are increased for a plurality of input patch features.

[0244] In addition, in this embodiment, the computing system 1000 performs anomaly detection based on the acquired recompatch features (S115).

[0245] In other words, in this embodiment, the computing system 1000 can perform anomaly detection on the test sample image based on the recompatch sampling dataset acquired based on each training image used for learning and the recompatch target dataset acquired based on the test sample image.

[0246] In other words, in an embodiment, the computing system 1000 can perform anomaly detection on a test sample image based on a recompatch sampling dataset and a recompatch target dataset.

[0247] In more detail, in this embodiment, the computing system 1000 can calculate the similarity between at least a portion of the recompatch sampling dataset stored in the database and the recompatch target dataset (hereinafter referred to as the anomaly detection similarity).

[0248] Furthermore, in this embodiment, the computing system 1000 can generate an anomaly score map based on the calculated anomaly detection similarity.

[0249] For reference, an anomaly score map can be defined as an index that shows how far a state deviates from a predetermined normal state, based on the values ​​(scores) assigned by the model.

[0250] Furthermore, in this embodiment, the computing system 1000 can determine that the higher the score on the generated anomaly score map, the closer the test sample image is to an abnormal state, and the lower the score on the anomaly score map, the closer the test sample image is to a normal state.

[0251] Therefore, in this embodiment, the computing system 1000 can perform anomaly detection on the test sample image.

[0252] As described above, in embodiments of the present invention, the computing system 1000 performs a process (in embodiments, a ReConPatch Process) to construct discriminant features for anomaly detection by distilling the main features of the dataset for the anomaly detection target into a pre-trained model based on semi-supervised learning, as described above, and can perform anomaly detection using the anomaly detection model (ODM) thus learned.

[0253] Therefore, in this embodiment, the computing system 1000 can directly and significantly improve the anomaly detection performance and quality based on the anomaly detection model (ODM) according to the embodiment of the present invention.

[0254] As described above, the patch feature learning method and system for anomaly detection according to one embodiment of the present invention has the effect of providing an anomaly detection model (ODM) that performs patch feature-based learning on a predetermined pretrained model based on an image dataset of the target of anomaly detection, thereby enabling more efficient data processing and further improving the task processing performance and quality for anomaly detection.

[0255] Furthermore, the patch feature learning method and system for anomaly detection according to one embodiment of the present invention has the effect of improving the accuracy and efficiency of anomaly detection even in a restrictive learning environment, while simultaneously enhancing task processing performance, by performing patch feature-based learning that reduces the variance of mutually similar patch features and increases the difference between mutually heterogeneous patch features.

[0256] On the other hand, the embodiments of the present invention described above can be implemented in the form of program instructions that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., individually or in combination. The program instructions recorded on the computer-readable recording medium may be specifically designed and configured for the present invention or may be publicly known and usable by those skilled in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code that can be produced by a compiler, but also advanced language code that can be executed by a computer using an interpreter or the like. Hardware devices may be modified into one or more software modules to perform the processing according to the present invention, and vice versa.

[0257] The specific executions described herein are merely embodiments and do not in any way limit the scope of the invention. For the sake of brevity, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of such systems may be omitted. Furthermore, the connections of lines or connecting members between components shown in the drawings are illustrative representations of functional and / or physical or circuit connections and may be replaced or represented as a variety of additional functional, physical, or circuit connections in actual devices. Also, components that are not necessarily required for the application of the invention may not be necessary unless specifically mentioned, such as "essential" or "important."

[0258] Furthermore, while the detailed description of the present invention has been provided with reference to preferred embodiments, a person skilled in the art or someone with ordinary knowledge of the art will understand that the present invention can be modified and altered in various ways, within the scope of the concept and technical domain of the invention as described in the claims. Therefore, the technical scope of the present invention is not limited to what is described in the detailed description of the specification, but must be defined by the claims.

[0259] The mode for carrying out the invention is the same as the best mode for carrying out the invention described above. [Industrial applicability]

[0260] The present invention relates to a patch feature learning method and system for anomaly detection, and is industrially applicable as it can be used in the artificial intelligence industry.

Claims

1. A method for a computing system including memory and a processor to perform patch feature learning for anomaly detection, The steps include obtaining a feature map using the first image dataset, The steps include: obtaining multiple patch features based on local data within a predetermined image based on the acquired feature map; The steps include: performing Feature Representation Learning based on the multiple patch features obtained; The steps include obtaining a recompatch feature (ReCompatch Feature) through the feature representation learning performed above, The process includes the step of performing anomaly detection based on the acquired recompatch features, The recon patch feature is data obtained by reconstructing the feature representation of the multiple patch features according to the similarity calculated based on the multiple patch features. A patch feature learning method for anomaly detection.

2. The aforementioned patch feature is a feature extracted from a patch that identifies at least a portion of a given image. A patch feature learning method for anomaly detection according to claim 1.

3. The step of obtaining the aforementioned multiple patch features is, The steps include: acquiring multiple patches of a predetermined patch size based on the first image contained in the first image dataset, and extracting features from each of the acquired multiple patches; The method includes one of the following steps: extracting features from the entire first image contained in the first image dataset, and dividing the extracted whole features into predetermined patch sizes. The patch feature learning method for anomaly detection according to claim 2.

4. The step of performing feature representation learning based on the aforementioned multiple patch features is: This includes steps for conducting semi-supervised concept learning. A patch feature learning method for anomaly detection according to claim 1.

5. The step of performing feature representation learning based on the aforementioned multiple patch features is: The process includes a step of performing the learning based on a first network, which is a neural network that calculates the similarity between predetermined features, and a second network, which is a neural network that realizes the feature representation learning. A patch feature learning method for anomaly detection according to claim 4.

6. The first network and the second network are A feature representation layer is a layer that reconstructs a feature representation according to a predetermined feature, Each of these includes a spatial projection layer, which is a layer that projects a feature representation corresponding to a given feature into the feature representation space, A patch feature learning method for anomaly detection according to claim 5.

7. The step of performing feature representation learning based on the aforementioned multiple patch features is: The method further includes the step of progressively distilling the data corresponding to the parameters of the second network into the data corresponding to the parameters of the first network based on an exponential moving average algorithm. The patch feature learning method for anomaly detection according to claim 6.

8. The step of performing feature representation learning based on the aforementioned multiple patch features is: The steps include projecting a first patch feature pair (Patch Feature Pair) containing predetermined first and second patch features onto the feature representation space, The process further includes the step of calculating pairwise similarity, which is data measuring the similarity between a first patch feature and a second patch feature based on a first patch feature pair projected onto the feature representation space. A patch feature learning method for anomaly detection according to claim 7.

9. The step of performing feature representation learning based on the aforementioned multiple patch features is: The process further includes the step of calculating contextual similarity, which is data measuring the bidirectional similarity between K nearest neighbors to a first patch feature and K nearest neighbors to a second patch feature, based on a first patch feature pair projected onto the feature representation space. A patch feature learning method for anomaly detection according to claim 8.

10. The step of performing feature representation learning based on the aforementioned multiple patch features is: The step further includes calculating Integrated Similarity, which is data obtained by linearly combining the calculated pairwise similarity and the contextual similarity, The patch feature learning method for anomaly detection according to claim 9.

11. The step of performing feature representation learning based on the aforementioned multiple patch features is: The step further includes training the second network based on the calculated integrated similarity, A patch feature learning method for anomaly detection according to claim 10.

12. The step of training the second network based on the combined similarity is: The step includes training the second feature representation layer to map the first patch feature and the second patch feature to each other in the feature representation space in accordance with the combined similarity, The patch feature learning method for anomaly detection according to claim 11.

13. The step of performing the aforementioned anomaly detection is: The first step is to obtain a test sample image, The steps include: acquiring the recompatch feature using the acquired first test sample image; The step includes performing anomaly detection based on the recompatch features obtained by the feature representation learning and the recompatch features obtained by the first test sample image, A patch feature learning method for anomaly detection according to claim 1.

14. The step of performing the aforementioned anomaly detection is: The steps include generating an anomaly score map based on the similarity between the recompatch features obtained by the feature representation learning and the recompatch features obtained by the first test sample image, The step of performing anomaly detection based on the generated anomaly score map further includes: The patch feature learning method for anomaly detection according to claim 13.

15. The steps include: Coreset sampling of the recompatch features obtained through the aforementioned feature representation learning; The step of performing the anomaly detection based on the core set sampled recompatch features further includes: The patch feature learning method for anomaly detection according to claim 13.

16. At least one memory, The system includes at least one processor that reads at least one application stored in the memory and performs patch feature learning for anomaly detection, The aforementioned processor, Obtain a feature map using the first image dataset. Based on the acquired feature map, multiple patch features based on local data within a predetermined image are obtained. Based on the multiple patch features obtained, feature representation learning is performed. Based on the similarity calculated from the aforementioned multiple patch features, a reconpatch feature, which is data obtained by reconstructing the feature representation of the multiple patch features, is acquired through the feature representation learning performed above. Anomaly detection is performed based on the recompatch features obtained. A patch feature learning system for anomaly detection.

17. At least one memory, The system includes at least one processor that reads at least one application stored in the memory and performs patch feature learning for anomaly detection, The instruction words of the aforementioned processor are: The steps include obtaining a feature map using the first image dataset, The steps include: obtaining multiple patch features based on local data within a predetermined image based on the acquired feature map; The steps include: performing Feature Representation Learning based on the multiple patch features obtained; The steps include obtaining a reconpatch feature, which is data obtained by reconstructing the feature representation of the multiple patch features according to the similarity calculated based on the multiple patch features, through the feature representation learning performed above, The instructions include a step of performing anomaly detection based on the acquired recompatch features, Computing device.

Images