Apparatus and method for manufacturing anomaly detection

Abstract

A method and apparatus for manufacturing anomaly detection are provided. The method includes generating second image data by applying a first anomaly detection model to first image data, generating third image data corresponding to an image difference between the first image data and the second image data by performing a first logical operation on the first image data and the second image data, generating image mask data having feature information between the first image data and the second image data by applying a second anomaly detection model to the first image data and the second image data, and generating fourth image data having anomaly indication information by performing a second logical operation on the third image data and the generated image mask data. The method can include learning the first anomaly detection model and the second anomaly detection model using a plurality of pieces of image data of a predetermined good product.

Description

Technical Field

[0001] This disclosure relates to an apparatus and method for manufacturing anomaly detection. Background Technology

[0002] Equipment used for anomaly inspection or detection examines images of manufactured articles (hereinafter referred to as "manufacturing images") in the manufacturing industry for anomalies, for example, for quality control or anomaly correction. Typical equipment for anomaly inspection / detection may employ typical image processing, for example, based on image processing algorithms.

[0003] Alternatively, if anomaly checking / detection is performed using deep learning techniques (such as through convolutional neural networks (CNNs)), a large amount of training data and labels will be required or expected for training the CNN, but the availability of examples with appropriate data and labels will be limited.

[0004] (Existing technical literature)

[0005] (Patent Document 1) Japanese Patent No. 2020-139905 (Published on April 3, 2019)

[0006] (Patent Document 2) Korean Patent No. 10-2020-0135730 (Published on May 22, 2019)

[0007] (Patent Document 3) Korean Patent No. 10-2021-0050186 (Published on October 28, 2019) Summary of the Invention

[0008] Technical issues

[0009] One aspect of this disclosure is provided to present the chosen concept in a simplified form, which is further described below in detail in the specific embodiments. This disclosure is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0010] Solutions to technical problems

[0011] In one general aspect, an apparatus for manufacturing anomaly detection includes: an image generator configured to learn a first anomaly detection model using multiple image data of a predetermined good article, and to apply the learned first anomaly detection model to the first image data to generate second image data; a first logic operator configured to perform a first logic operation on the first image data and the second image data, and to output third image data corresponding to image differences between the first image data and the second image data; a feature extractor configured to learn a second anomaly detection model using the multiple image data of the predetermined good article, and to apply the learned second anomaly detection model to the first image data and the second image data to generate image mask data having feature information between the first image data and the second image data; and a second logic operator configured to perform a second logic operation on the third image data and the image mask data to generate fourth image data having anomaly indication information.

[0012] The learning of the first anomaly detection model may include performing compression-recovery learning based on the predetermined good artifact.

[0013] The learned first anomaly detection model can be configured to: when the first image data corresponds to the data of a defective article, perform compression-recovery on the data of the defective article to generate second image data corresponding to one of a plurality of predetermined good articles.

[0014] For the first logical operation, the first logical operator may include a subtraction logical operation unit, which is configured to subtract the first image data from the second image data.

[0015] The first logic unit can be configured to perform processing of the learning structure similarity (SSIM)-autoencoder algorithm.

[0016] The feature extractor can be configured to: extract feature vector information between the first image data and the second image data, and generate image mask data having the feature information based on the feature vector information.

[0017] The learning of the second anomaly detection model may include learning a reverse embedding algorithm, which compares the image information input to the first anomaly detection model with the image information output by the first anomaly detection model of the predetermined good product for each of a plurality of corresponding block units, and classifies the defective product and the defective product based on the comparison result for each of the plurality of corresponding block units.

[0018] The feature extractor can be configured to: extract feature vector information from the first image data using the learned second anomaly detection model for a corresponding block unit of the first image data, and based on the extracted feature vector information, generate image mask data for each of the corresponding block units, having a selection between defect-free product information and defective product information.

[0019] The second logic unit may include a multiplication logic unit configured to multiply the third image data by the image mask data.

[0020] In one general aspect, an apparatus for manufacturing anomaly detection includes a processor configured to: reconstruct defect-free product image data from input image data using a learned first anomaly detection model, wherein the learned first anomaly detection model includes a structural similarity (SSIM) autoencoder; generate image mask data having feature information between the input image data and the reconstructed defect-free product image data using a learned second anomaly detection model based on a reverse embedding algorithm; and generate defect indication information based on the result of the SSIM autoencoder and the generated image mask data.

[0021] In one general aspect, a method for manufacturing anomaly detection includes: generating second image data by applying a learned first anomaly detection model to first image data; generating third image data corresponding to image differences between the first image data and the second image data by performing a first logical operation on the first image data and the second image data; generating image mask data having feature information between the first image data and the second image data by applying a learned second anomaly detection model to the first image data and the second image data; and generating fourth image data having anomaly indication information by performing a second logical operation on the third image data and the generated image mask data.

[0022] The method may further include: learning the first anomaly detection model using multiple image data of the predetermined good product; and learning the second anomaly detection model using the multiple image data of the predetermined good product.

[0023] The learning of the first anomaly detection model may include performing compression-recovery learning based on the predetermined good artifact.

[0024] The learning of the first anomaly detection model may include learning a structural similarity (SSIM) autoencoder algorithm, and the generation of the third image data may include implementing the learned SSIM autoencoder algorithm.

[0025] The learning of the second anomaly detection model may include learning a reverse embedding algorithm, which compares the image information input to the first anomaly detection model with the image information output by the first anomaly detection model of the predetermined good product for each of a plurality of corresponding block units, and classifies the defective product and the defective product based on the comparison result for each of the plurality of corresponding block units.

[0026] The generation of the image mask data may include: extracting feature vector information from the first image data for a corresponding block unit of the first image data, and generating, based on the extracted feature vector information, the image mask data for each of the corresponding block units, having a selection between defect-free product information and defective product information.

[0027] When the first image data corresponds to the data of a defective product, the learned first anomaly detection model can perform compression-recovery on the data of the defective product to generate the second image data corresponding to one of a plurality of predetermined good products.

[0028] Performing the first logical operation on the first image data and the second image data may include subtracting the first image data from the second image data.

[0029] The generation of the image mask data may include extracting feature vector information between the first image data and the second image data, and generating the image mask data based on the extracted feature vector information.

[0030] The execution of the second logical operation may include multiplying the third image data with the image mask data.

[0031] Other features and aspects will be readily understood from the following detailed description, drawings, and claims.

[0032] Beneficial effects of the invention

[0033] According to one or more embodiments, regarding the original image and the image recovered by an image generator such as an autoencoder, for manufacturing images in the manufacturing process, contrast learning of feature vectors can be performed by learning a reverse embedding algorithm for each block unit to reduce the difference between feature vectors at the same location and increase the difference between feature vectors at different locations, thereby quickly detecting defects in the manufacturing image.

[0034] Additionally, in one or more embodiments, learning can be performed only on good articles of manufacture, thus enabling timely production of articles of manufacture. In some cases, it may be desirable to measure the dimensions of defective articles of manufacture. In this example, to leverage deep learning to address the aforementioned problem, a segmentation method can be used. For instance, when an automatically generated segmentation mask is used as a segmentation learning label, there is a significant reduction in the time periods constituting the data used for segmentation. Attached Figure Description

[0035] Figure 1 This is a diagram illustrating an apparatus for anomaly detection according to one or more embodiments.

[0036] Figure 2 This illustrates one or more embodiments. Figure 1 A diagram showing example image data.

[0037] Figure 3 This is a diagram illustrating example operations of an anomaly detection model according to one or more embodiments.

[0038] Figure 4 This is a diagram illustrating an example learning operation of an anomaly detection model of a device for anomaly detection using a reverse embedding method according to one or more embodiments.

[0039] Figure 5 This is a diagram illustrating example anomaly detection results according to one or more embodiments.

[0040] Figure 6 This is a diagram illustrating example results of anomaly detection for an example capacitor assembly according to one or more embodiments.

[0041] Figure 7 This is a diagram illustrating example results of anomaly detection for an example camera module component according to one or more embodiments.

[0042] Figure 8 This is a diagram illustrating an example method for anomaly detection according to one or more embodiments.

[0043] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or set forth, the same reference numerals will be understood to refer to the same or similar elements, features, and structures. The drawings may not be drawn to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0044] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, after understanding the disclosure of this application, various modifications, variations, and equivalents of the methods, apparatus, and / or systems described herein will be readily apparent. For example, the order of operations described herein is merely illustrative and is not limited to the order set forth herein; rather, changes that will be readily understood after understanding the disclosure of this application are possible, except for operations that must occur in a specific order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0045] The features described herein may be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples provided herein are merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein that will be readily understood upon understanding the disclosure of this application.

[0046] The terminology used herein is for the purpose of describing various examples only and is not intended to limit this disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” enumerate the presence of the stated features, quantities, operations, components, elements, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0047] Throughout this specification, when a component is described as "connected to" or "bonded to" another component, that component may be directly "connected to" or directly "bonded to" another component, or there may be one or more other components in between. In contrast, when an element is described as "directly connected to" or "directly bonded to" another element, there are no other elements in between. As used herein, the term "and / or" includes any one of the associated listed items or any combination of any two or more items.

[0048] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts will not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Therefore, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0049] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains, based on an understanding of the disclosure of this application. Terms (such as those defined in common dictionaries) shall be interpreted as having a meaning consistent with their meaning in the relevant field and in the context of the disclosure of this application, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein. The use of the term “may” in relation to examples or embodiments (e.g., what an example or embodiment may include or implement) implies the existence of at least one example or embodiment that includes or implements such a feature, and is not limited to all examples or embodiments that include or implement such a feature.

[0050] As mentioned above, if anomaly checking / detection (hereinafter referred to as "anomaly detection") is performed using deep learning techniques (such as by using a typical convolutional neural network (CNN)), the availability of appropriate data and labels can be limited. For example, appropriate defective data and labels may be very limited in manufacturing facilities where anomaly detection may be required (such as manufacturing facilities that may need or expect to change rapidly).

[0051] For example, a relatively large amount of data can typically be collected to train such a CNN. For example, if anomaly detection is performed using such a CNN, such training data can be collected from a balance of data on good artifacts and data on defective artifacts. However, a potential drawback of such anomaly detection methods based on example CNNs is that collecting data in this way can take a long time (e.g., two (2) weeks to one (1) month or more), especially when defects occur at a very low frequency.

[0052] Since typical deep learning methods applied to anomaly detection are highly data-dependent, their performance may still be degraded even when algorithms based on social network service (SNS) data or, as will be studied, general images, and unsupervised learning is considered, especially given the difficulty of collecting and learning from flawed data. Some methods can be implemented within typical unsupervised learning algorithms used to collect incorrect data. However, in the case of applying such unsupervised learning to anomaly detection, if unsupervised learning is applied to data collection, learning can still be primarily based on good artifacts (e.g., on sufficiently good artifacts for learning).

[0053] Therefore, although unsupervised learning can generally have the following advantages: it can greatly shorten the data collection cycle and does not require operations for data labeling, there may be a problem: this typical unsupervised learning method may not be suitable for data with a large number and variety of good artifacts, such as in the case of anomaly detection, because anomaly detection learning may also be expected to be based on defective artifacts.

[0054] Typically, in general unsupervised learning, autoencoder structures can be deep learning structures that effectively compress data and are widely used in unsupervised learning. For example, in applying this unsupervised learning to anomaly detection, an autoencoder with a CNN structure can learn the data distribution of an image. For instance, when data of good artifacts can be learned, data of defective artifacts can be passed through the autoencoder structure, and the data of defective artifacts can be recovered to that of good artifacts. Therefore, the image differences from the original image can be used to detect anomalies.

[0055] However, in this example of applying such unsupervised learning to anomaly detection, the autoencoder may have a bottleneck structure, and therefore, when recovering the data, there may be ambiguity issues in high-frequency regions. This may make it difficult to distinguish between subtle defect areas and high-frequency normal areas, and thus the performance of the desired anomaly detection may deteriorate.

[0056] As another example, anomaly detection methods can be employed using generative adversarial networks (GANs) or variational autoencoders (VAEs) instead of autoencoder structures. However, in GANs, there may be drawbacks such as the random vectors potentially needing fine-tuning during prediction, and in VAEs, there may be issues such as the quality of the recovered image potentially deteriorating during the operation in the autoencoder example.

[0057] Therefore, when applying anomaly detection in a manufacturing facility, it may be desirable to address potential data loss issues caused by bottlenecks in the example autoencoder. This may also benefit the applicability of anomaly detection to a wide variety of good articles.

[0058] Figure 1 This is a diagram illustrating an apparatus for anomaly detection according to one or more embodiments, and Figure 2 This illustrates one or more embodiments. Figure 1 A diagram showing example image data.

[0059] Reference Figure 1 and Figure 2 The device 10 for anomaly detection may include, for example, an image generator 100, a first logic unit 200, a feature extractor 300, and a second logic unit 400. Device 10 and Figures 2 to 4The devices described herein each represent one or more processors and one or more non-transitory memories, wherein the learning and / or inference operations are implemented in hardware or by a combination of hardware and software, such as by instructions stored in one or more memories, which, when executed by at least one of the one or more processors, configure at least one or any combination of the one or more processors to implement any, any combination, or all of the operations or methods described herein. As another non-limiting example, any one of the image generator 100, the first logic unit 200, the feature extractor 300, and the second logic unit 400 respectively represents one or more such processors and may represent one or more such non-transitory memories, which may further store such corresponding instructions for such corresponding learning and / or inference operations.

[0060] Image generator 100 can use multiple images of good quality as learning targets to learn (i.e., train) a first anomaly detection model. Image generator 100 can apply the learned first anomaly detection model to the first image data VD1, which is the object of inspection, to generate second image data VD2.

[0061] The first logic operator 200 may be configured to perform a first logic operation on first image data VD1 corresponding to the original image and second image data VD2, which is an image recovered by the image generator 100, and may output third image data corresponding to the image difference between the first image data VD1 and the second image data VD2. As a non-limiting example, the first logic operator 200 may be a first logic operation unit.

[0062] The feature extractor 300 can use multiple image data sets of a good product as learning targets to learn (i.e., train) a second anomaly detection model. These multiple image data sets of the good product can be multiple image data sets of the same good product as those used as learning targets in the learning of the first anomaly detection model, or they can be multiple image data sets of different good products. The feature extractor 300 can apply the learned second anomaly detection model to the first image data VD1 and the second image data VD2 as the objects of inspection to output image mask data VMD containing feature information between the first image data VD1 and the second image data VD2.

[0063] For example, a reverse embedding method can be used in the second anomaly detection model, in which a comparison of first image data and second image data of good articles as learning targets can be learned for each block unit, so that good articles and defective articles can be classified with respect to the first image data (learning targets) for each block unit.

[0064] For example, the second anomaly detection model can implement a reverse embedding learning method, which will be referred to below for example. Figure 4 Equation 2 is described in more detail.

[0065] In one example, a combined model can be derived by combining the second anomaly detection model learned by the feature extractor 300 and the first anomaly detection model learned by the image generator 100. In another example, an image generator 100 with a first anomaly detection model can be predetermined, and the feature extractor 300 with a learned second anomaly detection model can be combined with the image generator 100. The result of the image generator 100 is compared with the result of the feature extractor 300, and the final anomaly detection result is generated based on the result of the comparison.

[0066] Therefore, as a non-limiting example, one or more embodiments may use two such models / networks (or example combined models / networks) to perform anomaly detection, wherein the final result of anomaly detection is based on the corresponding results of the two such models / networks.

[0067] Image generator 100 can be configured to compress and then recover first image data VD1 of a good article. In this case, when data of a defective article passes through image generator 100, a second image data VD2 similar to the data of a good article can be generated during recovery. When the difference between the two image data VD1 and VD2 is obtained by the implemented structural similarity (SSIM) algorithm, a third image data VD3 with a first anomaly detection map can be generated.

[0068] The feature extractor 300 can perform a reverse embedding learning method on the image data of the good artifact as the learning target, and then compare the block unit feature vectors between the first image data VD1 of the original image as the object of inspection and the second image data VD2 recovered by the image generator 100 to generate image mask data VMD with a second anomaly detection map.

[0069] As a non-limiting example, the feature extractor 300 may compare first image data VD1 of each block N×32×32 with second image data VD2 of each block N×32×32. In this case, as a non-limiting example, N represents the number of blocks, and "32×32" represents the size of an image with thirty-two (32) pixels by thirty-two (32) pixels.

[0070] A second logical operation (e.g., multiplication) can be performed on the third image data VD3 and the image mask data VMD to generate a fourth image data VD4 with a final anomaly detection map. For example, the fourth image data VD4 can be used to detect anomalous data and can also be used as a segmentation mask. Therefore, the fourth image data VD4 may be or include information indicating anomalies.

[0071] The second logic operator 400 may be configured to perform a second logic operation (e.g., multiplication) on the third image data VD3 and the image mask data VMD to generate a fourth image data VD4 with anomaly information. As a non-limiting example, the second logic operator 400 may be a second logic operation unit.

[0072] Therefore, when the good product as the learning target has been learned by the first anomaly detection model in the image generator 100, the first image data corresponding to the data of the defective product in the inspection process can be input, the recovery operation can be performed, and the result of the first anomaly detection model in the image generator 100 can be used to generate a second anomaly detection model corresponding to the data of the good product.

[0073] The first logic operation of the first logic arithmetic unit 200 may include a subtraction logic operation that subtracts the first image data VD1 and the second image data VD2.

[0074] As a non-limiting example, the first logical operation of the first logic unit 200 may correspond to the processing of the learning structure similarity (SSIM)-autoencoder algorithm.

[0075] For example, as a non-limiting example, the image generator 100 can be learned using the SSIM-autoencoder algorithm and the processing of SSIM, which uses the loss function presented in Equation 1 below.

[0076] [Formula 1]

[0077]

[0078] As a non-limiting example, in Equation 1, p and q are obtained by cropping the first image data VD1, the original image, and the second image data VD2 recovered by the autoencoder.<k×k> The obtained image data, μ p and μ q It is the average strength, σ p and σ q It is the variance, σ pq It is the covariance, and c1 and c2 are constants, typically 0.01 and 0.03 respectively.

[0079] As described above, when learning image data of good products through an autoencoder, the data distribution of good products can be learned. Therefore, when image data of defective products is passed through the autoencoder, the image data of defective products can be recovered to image data corresponding to the data of good products.

[0080] Figure 3 This is a diagram illustrating example operations of an anomaly detection model according to one or more embodiments.

[0081] Reference Figure 3 The operational processing of the first anomaly detection model will be described. For example, the first anomaly detection model may have a learned data distribution of the first image data VD1, and may include processing to restore the input image to a good artifact.

[0082] For example, when inputting an N×N size image, the input image of size w×h can be compressed into a one-dimensional vector of size M, and then the compressed vector can be restored to an image with size w×h (i.e., the original size of the input image before compression).

[0083] In this process, key elements of the VD1 data can be compressed into a vector of size M. When unlearned data (e.g., defective images) is input as VD1 into a first anomaly detection model, one of the previously learned images can be recovered from the input VD1.

[0084] Figure 4 This is a diagram illustrating an example learning operation of an anomaly detection model of a device for anomaly detection using a reverse embedding method according to one or more embodiments.

[0085] Reference Figure 4 The feature extractor 300 can extract the feature vector information FVI between the first image data VD1 and the second image data VD2, and can generate image mask data VMD with feature information based on the feature vector information FVI.

[0086] For example, the feature extractor 300 can learn an inverse embedding algorithm for each block unit to compare the first image data VD1 and the second image data VD2 of good products (learning target), and can classify the good products and defective products on the first image data VD1 for each block unit.

[0087] Therefore, the feature extractor 300 can apply the learned inverse embedding algorithm to extract feature vector information from the first image data that is the object of inspection for each block unit, and can output image mask data VMD with good product information and defective product information for each block unit based on the feature vector information FVI.

[0088] For example, as a non-restrictive example, the loss function used in the learning process of the inverse embedding algorithm is presented below in Equation 2.

[0089] [Equation 2]

[0090]

[0091] In Equation 2, z i It is a feature vector, where the feature extractor 300 obtains blocks of the first image data (original image), and z j is the feature vector, where the feature extractor 300 obtains blocks of the second image data recovered by the image generator 100. sim is the cosine similarity, and τ is a hyperparameter.

[0092] Reference Figure 4 The feature extractor 300 may include, for example, a cropping operator 310, a contrast learner 330, and an anomaly score calculator 350.

[0093] First, the cropping operator 310 can crop the original image and the image recovered by the image generator 100 for each block unit.

[0094] The contrast learner 330 can learn to increase the cosine similarity at the same location in an image and can learn to decrease the cosine similarity at different locations in an image. In this case, the contrast learner 330 may also include a projector. For example, the projector can serve to compress only the important elements indicating such correspondence in the vector extracted by the feature extractor.

[0095] Image generator 100 can render feature vectors at the same locations that are consistent with each other, and can render feature vectors at different locations that are different from each other, to clearly generate differences depending on whether anomalies exist. Therefore, as a non-limiting example, anomaly score calculator 350 can use the above to calculate anomaly scores.

[0096] For example, in the anomaly detection map M, the first anomaly detection map M can be included in the third image data obtained by the image generator 100 through the SSIM algorithm. ssim The second anomaly detection map M is included in the image mask data VMD obtained by the inverse embedding algorithm of the feature extractor 300. contra The products are multiplied to obtain the final anomaly detection map M. As a non-limiting example, as presented in Equation 3 below, the anomaly score can be chosen, for example, the maximum value from the anomaly detection map M.

[0097] [Formula 3]

[0098] M = M ssim ×M contra

[0099] AnomalyScore = max i，j M

[0100] For example, the second logic operation of the second logic unit 400 can be performed by a multiplication logic unit configured to multiply the third image data VD3 and the image mask data VMD.

[0101] In addition, as mentioned above, refer to Figure 4 As in Equation 2, for example, the second anomaly detection model may correspond to a reverse embedding learning algorithm. For example, for each block unit, the second anomaly detection model can be trained by performing reverse embedding learning on second image data VD2 corresponding to the image recovered by image generator 100 and first image data VD1 corresponding to the original image, in order to reduce the differences between feature vectors at the same location and increase the differences between feature vectors at different locations.

[0102] Figure 5 This is a diagram illustrating example anomaly detection results according to one or more embodiments.

[0103] Reference Figure 5 , Figure 5 The table shown represents the example learning results based on one or more embodiments as an accuracy check.

[0104] First of all, Figure 1 In an example implementation of device 10, example capacitor (e.g., MLCC) data is applied, for example, such that capacitor assembly 1 is learned using images of 6880 good articles, while testing is performed using images of 587 defective articles and 830 good articles. In this case, as... Figure 5 As shown, the inspection accuracy achieved by capacitor assembly 1 is 97.1%.

[0105] Next, in Figure 1 In an example implementation of device 10, another example capacitor (e.g., MLCC) data is applied, for example, such that capacitor assembly 2 is learned using images of 2201 good articles, while testing is performed using images of 639 good articles and 1013 defective articles. In this case, as... Figure 5 As shown, the inspection accuracy achieved by capacitor assembly 2 is 94.5%.

[0106] Figure 6 This is a diagram illustrating example results of anomaly detection for a capacitor assembly according to one or more embodiments, and Figure 7 This is a diagram illustrating example results of anomaly detection for a camera module component according to one or more embodiments.

[0107] Reference Figure 6 For the capacitor assembly, the original images (corresponding to the first image data VD1) of cases C1 to C6 with six (6) different defects were examined. As a result, in Figure 6 As can be seen, the defects in the original image are accurately displayed on the inspected image (corresponding to the fourth image data VD4).

[0108] Reference Figure 7 For the camera module component, the original images (VD1) of cases C1 to C4 with four (4) different defects were examined. As a result, in Figure 7 As can be seen, the defects in the original image are accurately displayed on the inspection image (VD4).

[0109] In short, refer to Figures 1 to 7 The above examples also illustrate the corresponding operations of the example methods used for anomaly detection. Similarly, the above and following descriptions of the example methods for anomaly detection can be derived from the above descriptions regarding... Figures 1 to 7 The devices and components described, and any combination thereof, are used to implement this. Therefore, the following is about Figure 8 In the example method description, repeated descriptions may be omitted.

[0110] Figure 8 This is a diagram illustrating an example method for anomaly detection according to one or more embodiments.

[0111] In operation S100, multiple image data of good artifacts can be used as learning targets to learn a first anomaly detection model, and the learned first anomaly detection model can be applied to the first image data VD1, which is the object of inspection, to generate second image data VD2. As a non-limiting example, this can be performed in the image generator 100 discussed above.

[0112] In operation S200, a first logical operation can be performed on the first image data VD1 and the second image data VD2, and a third image data VD3 corresponding to the image difference between the first image data VD1 and the second image data VD2 can be output. As a non-limiting example, operation S200 can be executed by the first logic operator 200 described above.

[0113] In operation S300, multiple image data of a good product can be used as learning targets to learn a second anomaly detection model, and the learned second anomaly detection model can be applied to the first image data VD1 and the second image data VD2, which are the objects of inspection, to output image mask data VMD containing feature information between the first image data VD1 and the second image data VD2. As a non-limiting example, operation S300 can be performed by the feature extractor 300 discussed above.

[0114] In operation S400, a second logical operation can be performed on the third image data VD3 and the image mask data VMD to generate fourth image data VD4 with anomaly information. As a non-limiting example, operation S400 can be performed by the second logical operator 400 described above.

[0115] The learned first anomaly detection model can perform compression-recovery learning on good products, and when the input is the first image data corresponding to the defective product and the recovery operation is performed, it can generate the second image data corresponding to the good product.

[0116] For example, the first logical operation of operation S200 may include a subtraction logical operation that subtracts the first image data VD1 and the second image data VD2.

[0117] For example, the first logical operation of operation S200 can correspond to the processing of the learning structure similarity (SSIM)-autoencoder algorithm.

[0118] In operation S300, feature vector information FVI between the first image data VD1 and the second image data VD2 can be extracted, and image mask data VMD with feature information based on feature vector information FVI can be generated.

[0119] In operation S300, based on the first anomaly detection model or while learning the first anomaly detection model, a reverse embedding algorithm can be learned for each block unit to compare the first image data VD1 and the second image data VD2, which are good products as learning targets, and the good products and defective products on the first image data VD1 can be classified for each block unit.

[0120] In operation S300, the learned inverse embedding algorithm can be applied to extract feature vector information FVI from the first image data VD1, which is the object of inspection, for each block unit, and output image mask data VMD with good product information and defective product information for each block unit based on the feature vector information FVI.

[0121] The second logical operation of operation S400 may include a multiplication logical operation that multiplies the third image data VD3 and the image mask data VMD.

[0122] Devices used for anomaly detection (including electronic devices, image generators, logic operators, feature extractors, cropping manipulators, contrast learners, anomaly score calculators, projectors, and more) are described here. Figures 1 to 8Other devices, apparatuses, units, modules, and components described herein are implemented by hardware components. Examples of hardware components that can be used to perform the operations described herein include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described herein. In other examples, one or more of the hardware components performing the operations described herein are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer can be implemented by one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond in a defined manner and execute instructions to obtain desired results). In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented via processors or computers can execute instructions or software, such as an operating system (OS) and one or more software applications running on the OS, to perform the operations described in this application. The hardware component can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For simplicity, the singular terms "processor" or "computer" are used to describe the examples described in this application, but in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor or two or more processors or a processor and a controller. One or more hardware components may be implemented by one or more processors or a processor and a controller, or by one or more other processors or another processor and another controller. One or more processors or a processor and a controller may implement a single hardware component or two or more hardware components. The hardware components may have one or more of different processing configurations, examples of which include single processor, discrete processor, parallel processor, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0123] The methods for performing the operations described in this application are executed by computing hardware (e.g., one or more processors or a computer) implemented as described above, which executes instructions or software to perform the operations described in this application performed by the methods. For example, a single operation or two or more operations may be performed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be performed by one or more processors or a processor and a controller, and one or more other operations may be performed by one or more other processors or another processor and another controller. One or more processors or a processor and a controller may perform a single operation or two or more operations.

[0124] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above can be written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure one or more processors or computers to operate as a machine computer or special-purpose computer to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by one or more processors or computers. In another example, the instructions or software include high-level code that is executed by one or more processors or computers using an interpreter. The instructions or software can be written in any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding description herein (which discloses algorithms for performing operations by the hardware components and methods described above).

[0125] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, may be recorded, stored, or fixed on or in one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage devices, hard disk drives (HDDs), solid-state drives (SSDs), card-type storage devices such as multimedia cards (e.g., Security Digital (SD) or Extreme Digital (XD)), magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state drives, and any other means configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide instructions or software and any associated data, data files, and data structures to one or more processors or computers to make the instructions executable. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system, such that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed by one or more processors or computers in a distributed manner.

[0126] While this disclosure includes specific examples, it will be readily understood upon understanding the disclosure of this application that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be understood for descriptive purposes only and not for limiting purposes. The description of features or aspects in each example will be considered applicable to similar features or aspects in other examples. Suitable results may be obtained if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner and / or if components in the described system, architecture, apparatus, or circuit are replaced or supplemented by other components or their equivalents. Therefore, the scope of this disclosure is not limited by the specific embodiments but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents will be construed as included in this disclosure.

[0127] [Explanation of reference numerals in the attached figures]

[0128] 100: Image Generator

[0129] 200: First Logic Unit

[0130] 300: Feature Extractor

[0131] 400: Second Logic Unit

[0132] VD1: First image data

[0133] VD2: Second Image Data

[0134] VD3: Third Image Data

[0135] VMD: Image Mask Data

[0136] VD4: Fourth Image Data

Claims

1. An apparatus for manufacturing anomaly detection, comprising: an image generator configured to learn a first anomaly detection model using a plurality of pieces of image data of a predetermined good product, and apply the learned first anomaly detection model to first image data to generate second image data; a first logical operator configured to perform a first logical operation on the first image data and the second image data, and output third image data corresponding to an image difference between the first image data and the second image data; a feature extractor configured to learn a second anomaly detection model using the plurality of pieces of image data of the predetermined good product, and apply the learned second anomaly detection model to the first image data and the second image data to generate image mask data having feature information between the first image data and the second image data; and a second logical operator configured to perform a second logical operation on the third image data and the image mask data to generate fourth image data having anomaly indication information. The learning of the first anomaly detection model includes performing compression-recovery learning based on the predetermined good product.

2. The apparatus of claim 1, wherein, The learned first anomaly detection model is configured to perform compression-recovery on data of a defective product when the first image data corresponds to the data of the defective product to generate the second image data corresponding to one of a plurality of predetermined good products.

3. The apparatus of claim 1, wherein, For the first logical operation, the first logical operator includes a subtraction logical operation unit configured to subtract the first image data and the second image data.

4. The apparatus of claim 1, wherein, The first logical operator is further configured to perform a process of learning a structural similarity autoencoder algorithm.

5. The apparatus of claim 1, wherein, The feature extractor is configured to extract feature vector information between the first image data and the second image data, and generate the image mask data having the feature information based on the feature vector information.

6. The apparatus of claim 1, wherein, The learning of the second anomaly detection model includes learning a reverse embedding algorithm that compares, for each of a plurality of corresponding block units, image information input to the first anomaly detection model with image information output by the first anomaly detection model for the predetermined good product, and classifies, for each of the plurality of corresponding block units, a non-defective product and a defective product based on a result of the comparison.

7. The apparatus of claim 1, wherein, The feature extractor is configured to extract, for a corresponding block unit of the first image data, feature vector information from the first image data using the learned second anomaly detection model, and based on the extracted feature vector information, generate, for each of the corresponding block unit, the image mask data having one selected between non-defective product information and defective product information.

8. The apparatus of claim 7, wherein, The second logical operator includes a multiplication logical operation unit configured to multiply the third image data and the image mask data.

9. The apparatus of claim 1, wherein, 10.An apparatus for manufacturing anomaly detection, comprising: ​ a processor configured to: apply a learned first anomaly detection model to input image data to obtain non-defective product image data, wherein the learned first anomaly detection model comprises a structural similarity autoencoder; generate image mask data having feature information between the input image data and the non-defective product image data using a learned second anomaly detection model based on a reverse embedding algorithm; and generate defect indication information based on results of the structural similarity autoencoder and the generated image mask data.

11. A method for manufacturing anomaly detection, comprising: generating second image data by applying a learned first anomaly detection model to first image data; generating third image data corresponding to image differences between the first image data and the second image data by performing a first logical operation on the first image data and the second image data; generating image mask data having feature information between the first image data and the second image data by applying a learned second anomaly detection model to the first image data and the second image data; and generating fourth image data having anomaly indication information by performing a second logical operation on the third image data and the generated image mask data.

12. The method of claim 11, further comprising: learning the first anomaly detection model using a plurality of image data of predetermined good products; and learning the second anomaly detection model using the plurality of image data of the predetermined good products. The learning of the first anomaly detection model includes performing a compression-recovery learning based on the predetermined good products.

14. The method of claim 12, 13. The method of claim 12, wherein, the learning of the first anomaly detection model includes learning a structural similarity autoencoder algorithm, and wherein the generation of the third image data includes implementing the learned structural similarity autoencoder algorithm. wherein 15. The method of claim 12, the learning of the second anomaly detection model includes learning a reverse embedding algorithm that compares, for each of a plurality of respective block units, image information input to the first anomaly detection model with image information output by the first anomaly detection model for the predetermined good products, and classifies, for each of the plurality of respective block units, a non-defective product and a defective product based on results of the comparison. the generation of the image mask data includes, for a respective block unit of the first image data, extracting feature vector information from the first image data, and based on the extracted feature vector information, generating, for each of the respective block unit, the image mask data having one selected between non-defective product information and defective product information. wherein when the first image data corresponds to data of a defective product, the learned first anomaly detection model performs compression-recovery on the data of the defective product to generate the second image data corresponding to one of a plurality of predetermined good products.

16. The method of claim 15, wherein, ​ 17. The method of claim 11, wherein, ​ 18. The method of claim 11, wherein, Performing the first logical operation on the first image data and the second image data includes subtracting the first image data and the second image data.

19. The method of claim 11, wherein, The generating of the image mask data includes extracting feature vector information between the first image data and the second image data, and generating the image mask data based on the extracted feature vector information.

20. The method of claim 11, wherein, The performing of the second logical operation includes multiplying the third image data and the image mask data.

Images