Artificial intelligence based material defect detection system and method considering type and distribution of actual defect images

By generating diverse virtual defect and non-defective images, the method addresses data scarcity and imbalance, improving defect detection accuracy and reducing manual annotation requirements in artificial intelligence-based systems.

JP2024541329AActive Publication Date: 2024-11-08ライトビジョン インク
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024527457
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-15
Filing Date
2021-11-16
Publication Date
2024-11-08
Estimated Expiration
2041-11-16

AI Technical Summary

Technical Problem

The challenge in conventional automatic defect detection systems is the scarcity of defect images due to data imbalance and the difficulty in obtaining non-defective images, which are often trade secrets, hindering effective training of artificial neural networks.

Method used

An artificial intelligence-based method generates a large number of virtual non-defective and defective images from a small number of actual defect images, considering the types and distribution of defects to create a diverse training dataset, using techniques like patch matching and GANs to enhance model learning.

Benefits of technology

This approach effectively addresses data imbalance and improves defect detection performance by generating diverse virtual images, enabling accurate defect prediction and reducing the need for manual annotation, thus enhancing model uniformity across various use scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024541329000001_ABST
    Figure 2024541329000001_ABST
Patent Text Reader

Abstract

A multi-sensor handover system capable of tracking a moving object and a method of providing a parking service therein are disclosed. A computing device used in the handover system includes a communication unit which is a communication connection path between a first sensor monitoring a first handover area and a second sensor monitoring a second handover area, and a handover unit which acquires a first image from the first sensor, acquires a second image from the second sensor, detects an exit of a moving object from the first handover area through the acquired first image, detects an entry of a moving object into the second handover area through the acquired second image, and determines whether a moving object that has entered the second handover area through the detected exit and entry of the moving object is the same moving object that has exited the first handover area. Here, the second handover area is an area that is matched with the first handover area.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an artificial intelligence based materials defect detection system and method that takes into account the type and distribution of real defect images. [Background technology]

[0002] Recently, with the development of artificial intelligence technology, the number of cases where artificial intelligence is applied in the field of automatic defect detection is rapidly increasing. For example, S. Marino, A. Smolarz, and P. Beauseroy, “Potato defects classification and localization with convolutional neural networks,” in Fourteenth International Conference on Quality Control by Artificial Vision, vol. 11472, 2019, pp. 110-117. and H. Lin, B. Li, X. Wang, Y. Shu, and S. Niu, “Automated defect inspection of LED chip using deep convolutional neural network,” Journal of Intelligent Manufacturing, vol. 30, pp. 2525-2534, 2019. use a deep learning technique called Class Activation Mapping (CAM) to automatically detect defects in potatoes and LED boards, respectively. CAM is a technique that visualizes which areas of the input image the artificial neural network trained by the classifier observed to derive the classification result. In the above example, an artificial neural network is trained to classify good / defective and CAM is used as a tool to locate defects.

[0003] In order to train an artificial neural network for conventional automatic defect detection, a large number of images of good and defective products are required, but due to the nature of defects, it is difficult to obtain images of defects (for example, defects occur in the ppm range in the case of semiconductors), and there is also the problem of data imbalance with images of good products, which are relatively easy to obtain.In addition, information on good and defective products is a trade secret, making it difficult to disclose, and there is also the problem of high difficulty in building a dataset. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention provides an artificial intelligence based materials defect imaging method and defect detection system.

[0005] The present invention also provides an artificial intelligence-based material defect image generation method and defect detection system that can generate a large number of virtual good and defective images from a small number of defect images to construct a training data set.

[0006] In addition, the present invention provides an artificial intelligence-based material defect image generation method and defect detection system that, when a defect image is input by photographing a virtual training data set, generates a non-defective product image that accurately corresponds to the defect image and is capable of training an artificial neural network.

[0007] Furthermore, the defect detection system of the present invention generates virtual defect images / virtual good product images that are as diverse in type and uniform in distribution as possible, taking into consideration the types and distribution of actual defect images. A model trained using the virtual defect / good product images solves the problem of data imbalance according to the types of defect images, and exhibits uniform performance in various situations that may be encountered in actual use. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, a defect detection system according to an embodiment of the present invention includes a learning data generation unit that generates a plurality of virtual good product images or a plurality of virtual defect images using defect images, a learning unit that trains a model for defect detection using the generated virtual good product images or the generated virtual defect images, and a defect detection unit that detects the presence or absence of defects in an input image using the trained model, wherein the learning data generation unit analyzes types or distribution of the defect images to detect defect forms or background forms, and generates the virtual good product images or the virtual defect images with different types, numbers, or resolutions depending on the detected defect forms or background forms.

[0009] A defect detection system according to another embodiment of the present invention includes a learning data generation unit that generates a plurality of virtual good product images or a plurality of virtual defect images using defect images, a learning unit that trains a model for defect detection using the generated virtual good product images or the generated virtual defect images, and a defect detection unit that detects the presence or absence of defects in an input image using the trained model. Here, the learning data generation unit determines the types, number, or resolution of the generated virtual good product images or virtual defect images by analyzing types or distribution of the defect images using artificial intelligence, and determines the types, number, or resolution of the generated virtual good product images or virtual defect images to be different when requested by a user.

[0010] A defect detection method according to an embodiment of the present invention includes the steps of generating a number of virtual good product images or a number of virtual defect images using defect images, learning a model for defect detection using the generated virtual good product images or the generated virtual defect images, and detecting the presence or absence of defects in an input image using the learned model, wherein a type or distribution of the defect images is analyzed to detect a defect form or a background form, and the type, number or resolution of the virtual good product images or the virtual defect images is different depending on the detected defect form or background form. Effect of the Invention

[0011] By providing an artificial intelligence-based material defect detection system and method that takes into account the type and distribution of actual defect images according to one embodiment of the present invention, it is possible to generate a good product image that accurately corresponds to an input defect image and effectively detect unpredicted defect areas.

[0012] In addition, since the present invention belongs to unsupervised learning in which learning can be performed without annotation information, there is an advantage in that the labor required in the process of generating annotation information can be reduced.

[0013] Furthermore, the present invention has the advantage that problems caused by data imbalance and a lack of data can be reduced since a large number of virtual good / defective images are generated from a small number of defective images.

[0014] Furthermore, the present invention generates virtual images taking into account the types and distribution of actual defect images so that virtual defect images / virtual good product images have as diverse a type and uniform distribution as possible, so that a model trained with the corresponding virtual image can exhibit uniform performance in various situations that may be encountered in actual use. [Brief description of the drawings]

[0015] [Figure 1] 1 is a block diagram that generally illustrates a defect detection system in accordance with an embodiment of the present invention; [Diagram 2] 1 is a diagram illustrating an example of types and distributions of defect images; [Diagram 3] 1 is a diagram illustrating a process of generating a virtual non-defective image using a defect image according to an embodiment of the present invention; [Figure 4] 1 is a diagram illustrating a process of generating a virtual non-defective image using a non-defective image according to an embodiment of the present invention; [Diagram 5] 1 is a diagram illustrating a process of generating a virtual defect image using a defect image according to an embodiment of the present invention; [Figure 6] 11 is a diagram illustrating a process of converting a randomly generated virtual low-resolution defect image into a high-resolution defect image. [Figure 7] 4 is a diagram illustrating a defective area detection process according to an embodiment of the present invention; [Figure 8] 11 is a diagram illustrating a defect image generated by modifying the type and position of a defect in an original defect image according to another embodiment of the present invention; [Figure 9] 4 is a flowchart illustrating a model learning process according to an embodiment of the present invention. [Figure 10] 4 is a flowchart illustrating a defect detection method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] As used herein, singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprise" or "include" should not be interpreted as including all of the components or steps described in the specification, and some components or steps may not be included, or additional components or steps may be included. In addition, terms such as "... unit" and "module" described in the specification mean a unit that processes at least one function or operation, which is embodied in hardware or software, or a combination of hardware and software.

[0017] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Fig. 1 is a block diagram illustrating a schematic diagram of a defect detection system according to an embodiment of the present invention, Fig. 2 is a diagram illustrating an example of the type and distribution of defect images, and Fig. 3 is a diagram illustrating a process of generating a virtual good image using a defect image according to an embodiment of the present invention. Fig. 4 is a diagram illustrating a process of generating a virtual good image using a good image according to an embodiment of the present invention, Fig. 5 is a diagram illustrating a process of generating a virtual defect image using a defect image according to an embodiment of the present invention, and Fig. 6 is a diagram illustrating a process of converting a randomly generated virtual low-resolution defect image into a high-resolution defect image. Fig. 7 is a diagram illustrating a defect area detection process according to an embodiment of the present invention, and Fig. 8 is a diagram illustrating a defect image generated by modifying the type and position of a defect in an original defect image according to another embodiment of the present invention.

[0018] Referring to FIG. 1, a defect detection system 100 according to an embodiment of the present invention includes a learning data generation unit 110, a learning unit 115, an image generation model 120, a defect detection unit 125, a memory 130, and a processor 135.

[0019] The learning data generating unit 110 analyzes the types and distributions of a small number of stored defect images, and generates a large number of virtual defect images or a large number of virtual non-defective product images based on the analysis results.

[0020] The learning data generating unit 110 includes an image type and distribution analyzing unit 112, a non-defective image generating unit 114, and a defective image generating unit .

[0021] The image type and distribution analysis unit 112 analyzes the type and distribution of the input defect image to determine the type, number, resolution, etc. of the required virtual defect image or virtual non-defective image.

[0022] For example, as shown in FIG. 2, it is assumed that a defect image is input as an input image. The image type and distribution analysis unit 112 classifies the defect morphology and background morphology in the input defect image and analyzes that there are two types of defects and three types of backgrounds. When analyzing the defect morphology and background morphology, the image type and distribution analysis unit 112 extracts hand-crafted features such as Haar-like features and histogram of oriented gradient from the input image and uses clustering. The algorithm for analyzing the defect morphology and background morphology is not limited to the method using hand-crafted features and clustering.

[0023] Meanwhile, the image type and distribution analysis unit 112 can analyze the defect type and background type of one input image, or can analyze the defect type and background type of a plurality of input images as a whole. Here, the input image is not a virtual image but an actual image. Also, the input image is a concept that includes an actual image detected as defective using the virtual non-defective image or virtual defect image of the present invention.

[0024] The good-quality image generating unit 114 uses the input defect images to generate as many virtual good-quality images as determined by the image type and distribution analyzing unit 112. In this case, the good-quality image generating unit 114 may generate different numbers of virtual good-quality images based on the analyzed defect shape, may generate different numbers of virtual good-quality images based on the analyzed background shape, or may generate different numbers of virtual good-quality images based on the analyzed defect shape and the analyzed background shape.

[0025] For example, the good image generator 114 removes the defective area from the defective image and then generates a virtual good image through a patch matching technique.

[0026] This will be explained in more detail. For example, assume that defect images are input as shown in Fig. 3. The good-quality image generating unit 114 removes the defect area from each of the input defect images to generate a virtual good-quality image.

[0027] For example, the good image generator 114 generates a good image based on the defect image using a patch matching technique. Specifically, the good image generator 114 generates a good image by removing a defect area in the defect image and restoring the corresponding defect area using information surrounding the defect area. As shown in FIG 3, the defect image does not include the same defect area but includes defects of various types.

[0028] Therefore, the good image generator 114 applies different imaging techniques according to the defect pattern to remove the defective area and then generates a good image.

[0029] As another example, the good-quality image generating unit 114 may generate a number of virtual good-quality images using the input good-quality image.

[0030] According to another embodiment, the good product image generating unit 114 may generate a virtual good product image by using both the defect image and the good product image. For example, a virtual good product image may be generated by removing a defect area in the defect image and then replacing the removed defect area with an area in the good product image corresponding to the removed defect area.

[0031] According to another embodiment, the good image generating unit 114 may generate a virtual good image by using information about the vicinity of the defective area in the defective image or information about the area in the good image depending on the state of the defective area. For example, if the size of the defective area is large or the surrounding area is also damaged, the virtual good image is generated by replacing the removed defective area with an area in the good image, and if the size of the defective area is small, the virtual good image is generated by using the information about the vicinity of the defective area.

[0032] This will be explained with reference to FIG. For example, the good-quality image generating unit 114 generates a good-quality image having a resolution equal to or higher than a reference resolution of the input good-quality image, and then randomly cuts out the image to generate a large number of virtual good-quality images.

[0033] For example, the good-quality image generator 114 generates a good-quality image having a resolution equal to or higher than the reference resolution using PSGAN, and then randomly crops the image to generate a virtual good-quality image as shown in FIG.

[0034] As another example, the good-quality image generating unit 114 may enlarge a virtual good-quality image generated from a defective image, and then randomly cut the image to generate a number of good-quality images.

[0035] To summarize, the good-quality image generating unit 114 generates a large number of virtual good-quality images by using a small number of defect images. The defect image generator 116 generates a large number of virtual defect images using a small number of defect images.

[0036] For example, assume that a defect image is input as shown in FIG. 5. The defect image generator 116 generates a number of virtual defect images using DCGAN. Here, DCGAN randomly generates images from latent variables. Since the result of DCGAN is limited to 64 x 64, a super-resolution defect image is generated using CycleGAN, which is an artificial neural network capable of performing a super-resolution function to create a defect image with a higher resolution. CycleGAN is an artificial neural network that performs unpaired image-to-image translation, and an artificial neural network that performs a super-resolution function is obtained by learning one domain with a low-resolution image and the other domain with a high-resolution image (see FIG. 6).

[0037] That is, the defect image generator 116 converts the input low-resolution defect image into a high-quality high-resolution defect image using DCGAN and CycleGAN, and then generates a number of virtual defect images through a random image conversion process. Of course, as long as virtual defect images are generated, the algorithm is not limited to DCGAN and CycleGAN.

[0038] In FIG 5, the defect positions in the input image (defect image) and the defect positions in the generated virtual defect image are the same or similar. However, the defect positions in the virtual defect image may be different from the defect positions in the input image (see FIG 8). Also, the size or shape of the defect in the virtual defect image may be different from the size or shape of the defect in the input image.

[0039] For example, assuming that contaminants may be attached to the edge region of a substrate during a deposition process in semiconductor manufacturing, the defect image generating unit 116 uses an input image having a defect with contaminants attached to the center of an edge region of one side of the substrate to generate various virtual defect images having defects with contaminants attached to the center of an edge region of another side or on a non-center edge side, etc. In this case, it is assumed in advance that the contaminants may be attached to various parts of the edge during the process.

[0040] As another example, if the impact applied to the substrate during processing and the magnitude or range of the resulting cracks are preset, the defect image generator 116 generates multiple defect images with different crack depths, lengths, or positions from an input image of a substrate having a crack.

[0041] That is, the defect image generator 116 of the present invention can generate a virtual defect image having the same position, shape, or size as the defect in the input image, as well as a virtual defect image having a different position, shape, or size from the defect in the input image.

[0042] In addition, when the defect is a contaminant, the defect image generator 116 may generate a defect image having a contaminant different from the contaminant of the input image, in which the shape or size of the contaminant adhering to the substrate may differ depending on the contaminant.

[0043] Although the above describes the case where a virtual defect image is generated for one defect, a virtual defect image may be generated by combining a plurality of different types of defects. For example, when an image of a defect with contaminants and an image of a defect with cracks are input, a virtual defect image in which contaminants or cracks are formed at various positions is generated.

[0044] According to another embodiment, the defect image generator 116 may generate a virtual defect image by using not only a defect image but also a good product image. For example, it may extract defects from a defect image and display the extracted defects at various positions in the good product image to generate various virtual defect images.

[0045] In summary, the training data generator 110 generates a large number of virtual training data sets for model training using a small number of defect images. The large number of virtual training data sets are at least one of a large number of virtual non-defect images and virtual defect images.

[0046] Meanwhile, the learning data generating unit 110 may automatically operate the good product image generating unit 114 and the defect image generating unit 116 selectively or both depending on the defect type or background type of the defect image, or may operate the good product image generating unit 114 and the defect image generating unit 116 selectively or both depending on a user's request. For example, the learning data generating unit 110 may generate a number of virtual good product images or virtual defect images determined by the artificial intelligence, or a user may arbitrarily determine the number of virtual good product images or virtual defect images regardless of the number determined by the artificial intelligence.

[0047] In this way, once the training data set has been generated by the training data generating unit 110, it is used to train a model. The learning unit 115 learns the image generation model 120 by using at least one of the plurality of virtual non-defective images and the plurality of virtual defective images.

[0048] The learning unit 115 learns a model using at least one of a plurality of virtual good product images and virtual defect images, so that the image generation model 120 is trained to generate a good product image using a defect image.

[0049] The image generation model 120 is trained by the learning unit 115 using a training dataset (at least one of a number of virtual good images and virtual defect images), and then generates a comparison image (good image or defect image) corresponding to the input image during the defect detection process.

[0050] For example, the image generation model 120 is trained using a large number of virtual good product images and virtual defect images, so that when a defect image is input, a corresponding good product image can be generated.

[0051] The defect detection unit 125 is a means for detecting the presence or absence of a defect by using an input image and a comparison image. For example, if the input image is a defective image, the comparison image is a virtual non-defective image corresponding to the defective image. Therefore, the defect detection unit 125 derives a difference image between the input image and the comparison image to detect a defective area.

[0052] The defective area detection process will now be described in more detail with reference to FIG. For example, assuming that the input image is a defect image 610, a virtual good image as shown at 620 in Fig. 7 is generated through the learned image generation model 120. Then, the defect detection unit 125 derives a difference image between the defect image 610 and the virtual good image 620, and detects a defect area through a hysteresis threshold and a morphology operation.

[0053] However, the image generation model 120 generates an appropriate number of virtual good images in consideration of the defect form or background form of the defect image 610. For example, the image generation model 120 generates 10,000 virtual good images having the same or similar defects as the defects of the defect image 610 but with various backgrounds, and 5,000 virtual good images having the same or similar background as the background of the defect image 610 but with various defects. As another example, the image generation model 120 may generate 10,000 virtual good images in the case of an A defect and 3,000 virtual good images in the case of a B defect. That is, the image generation model 120 may generate different numbers of virtual good images according to the form, type, or number of the defects of the defect image 610, and may generate different numbers of virtual good images according to the form, type, or number of the background of the defect image 610.

[0054] The memory 130 is a means for storing instructions (program codes) required to perform a defect detection method according to an embodiment of the present invention. The processor 135 is a means for controlling the internal components of the defect detection system 100 according to one embodiment of the present invention (e.g., the learning data generation unit 110, the learning unit 115, the image generation model 120, the defect detection unit 125, the memory 130, etc.).

[0055] In addition, at least one of the image of the defect detected by the defect detection unit 125 or the virtual non-defective image and the virtual defect image generated by the image generation model 120 is reused for learning the image generation model 120.

[0056] According to another embodiment, when the input image is a defect image, a virtual good product image and a virtual defect image are generated through the learned image generation model 120. In this case, a defect area is detected by comparing the defect image with the virtual good product image, and a type, pattern, etc. of the defect is detected by comparing the defect image with the virtual defect image.

[0057] FIG. 9 is a flow chart illustrating a model learning process according to an embodiment of the present invention. In step S900, the defect detection system 100 generates at least one of a large number of virtual good images and virtual defect images using a small number of defect images.

[0058] According to one embodiment, the defect detection system 100 utilizes a small number of defect images to generate a large number of virtual good product images.

[0059] As another example, the defect detection system 100 utilizes a small number of defect images to generate a large number of virtual defect images.

[0060] As yet another example, the defect detection system 100 utilizes a small number of good product images to generate a large number of good product images.

[0061] As yet another example, the defect detection system 100 may utilize a small number of defect images to generate a large number of both virtual good images and virtual defect images.

[0062] In this way, the defect detection system 100 uses a small number of input images to generate a learning data set for learning the image generation model 120. This is similar to what has been described with reference to Figs. 1 to 5, so a duplicated description will be omitted.

[0063] In step S902, the defect detection system 100 trains the image generation model 120 using at least one of a large number of virtual good product images and virtual defect images.

[0064] The image generation model 120 is an image generation model that uses artificial intelligence and generates a comparison image corresponding to an input image.

[0065] For example, if the input image is a defect image, the image generation model 120 generates a non-defective image that corresponds to the defect image.

[0066] To this end, the image generation model 120 may be pre-trained using a training data set consisting of a virtual defect image and a virtual non-defective product image. The training process of an artificial intelligence model is obvious to those skilled in the art, and therefore a separate description thereof will be omitted.

[0067] In general, when it comes to artificial intelligence models, it is difficult to obtain a sufficient training data set for model training. This makes it difficult to train the actual model sufficiently, and there are many difficulties in applying it. In one embodiment of the present invention, a small number of defect images are used to generate a large number of virtual non-defective images and virtual defect images required for model training, and the model is trained using these images.

[0068] The advantage of model training using many training data sets is that the accuracy of the model can be further improved.

[0069] A method for detecting actual defects using a trained model in this manner will be described with reference to FIG. Fig. 10 is a flowchart showing a defect detection method according to an embodiment of the present invention. In the following, the subsequent operations will be described on the assumption that the image generation model 120 has been trained using the training data set as described in Fig. 10.

[0070] In step S1000, the defect detection system 100 acquires an input image. In step S1002, the defect detection system 100 applies the input image to the learned image generation model 120 to generate a comparison image.

[0071] For example, if the input image is a defective image, the comparison image is a non-defective image in which the defective area in the defective image has been removed.

[0072] When an input image is applied to the trained image generation model 120, the trained image generation model 120 generates and outputs a virtual non-defective image as a comparison image in which the defective area has been removed from the defective image, which is the input image.

[0073] In step S1004, the defect detection system 100 derives a difference image between the input image and the comparison image, and detects a defect area using the difference image.

[0074] For example, the defect detection system 100 can set a defect image as the comparison image, can set a non-defective image, or can set both the non-defective image and the defect image.

[0075] The defect detection system 100 can determine the presence or absence of a defect after deriving a difference image between an input image and a defect image set as a comparison image.

[0076] As another example, the defect detection system 100 may detect a defect area after deriving a difference image between an input image and a non-defective image set as a comparison image. Of course, depending on the implementation method, the defect detection system 100 can set both the defect image and the non-defective image as the comparison image and determine the presence or absence of a defect through comparison with the input image.

[0077] Furthermore, the defect detection system 100 can also use the comparison video for model learning as described with reference to FIG.

[0078] As described above, according to one embodiment of the present invention, a large number of virtual learning data sets are generated using a small number of input images (defect images, good product images), which are used to learn a model and improve accuracy, and then used for defect inspection, thereby advantageously improving defect detection performance.

[0079] Meanwhile, in the above, since the learning dataset was insufficient, virtual defect images and virtual good product images were generated and used as the learning dataset. However, if a large number of actual defect images and good product images are obtained through the repeated defect detection process, the number of virtual defect images / virtual good product images can be reduced and used for learning.

[0080] Therefore, the defect detection system checks the number of actual defect images / good product images, and appropriately generates virtual defect images / virtual good product images according to the checked number of actual defect images / good product images to use for learning. For example, if the number of actual defect images / good product images is 1000 or less, 20000 virtual defect images / virtual good product images are generated to learn a model, and if up to 5000 actual defect images and good product images are secured through an iterative defect detection process, 15000 virtual defect images / virtual good product images are generated.

[0081] That is, the artificial intelligence can not only detect defects by generating virtual defect images / virtual good product images and learning a model, but also generate virtual defect images / virtual good product images by considering the number of actual defect images / good product images that have been secured, and learn a model. As a result, the more the defect detection process is repeated, the fewer the number of virtual defect images / virtual good product images are, and the more actual defect images / good product images are used, so that the accuracy of the image generation model can be improved. In this case, the rate of increase in the number of actual defect images / good product images and the rate of decrease in the virtual defect images / virtual good product images may be the same, but it is more efficient if they are different.

[0082] On the other hand, the components of the above-described embodiment can be easily understood from a process perspective, i.e., each component can be understood as a respective process, and the process of the above-described embodiment can be easily understood from the perspective of the components of the device.

[0083] In addition, the technical contents described above may be embodied in the form of program instructions executed by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, either alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language codes, such as those produced by a compiler, but also high-level language codes executed by a computer using an interpreter, and the like. The hardware devices may be configured to operate as one or more software modules to perform operations of the embodiments, and vice versa.

[0084] The above-described embodiments of the present invention have been disclosed for illustrative purposes, and a person skilled in the art having ordinary knowledge of the present invention may make various modifications, changes, and additions within the spirit and scope of the present invention, and such modifications, changes, and additions should be considered to fall within the scope of the following claims. [Explanation of symbols]

[0085] 100 Defect Detection System 110 Learning data generation unit 112 Video Type and Distribution Analysis Department 114 Quality Image Generation Department 115 Learning Department 116 Defect image generation unit 120 Image Generation Model 125 Defect detection section 130 Memory 135 processors 610 Defective Image 620 Virtual Good Product Video

Claims

1. a learning data generating unit that generates a large number of virtual non-defective images or a large number of virtual defect images by using the defect images; a learning unit that learns a model for defect detection using the generated virtual non-defective product image or the generated virtual defective image; a defect detection unit that detects the presence or absence of a defect in an input image by using the learned model, the learning data generation unit detects a defect form or a background form by analyzing a type or distribution of the defect image, and generates the virtual good product image or the virtual defect image with different types, numbers, or resolutions depending on the detected defect form or background form.

2. 2. The defect detection system of claim 1, wherein the learning data generator extracts hand-crafted features, such as a Haar-like feature or a histogram of oriented gradient, from the defect image and detects the defect shape or the background shape using clustering.

3. The learning data generation unit an image type and distribution analysis unit that analyzes the type and distribution of the defect image to detect the defect type or the background type; a good-quality image generating unit that generates the virtual good-quality image in response to the detected defect form or the detected background form; 2. The defect detection system of claim 1, further comprising: a defect image generating unit that generates the virtual defect image in response to the detected defect form or the detected background form.

4. The defect detection system of claim 3, wherein when a user specifies the type, number or resolution of the virtual good product image or the virtual defect image, the good product image generation unit or the defect image generation unit is operated to generate a virtual good product image or virtual defect image of the specified type, number or resolution regardless of the type, number or resolution determined by artificial intelligence by the image type and distribution analysis unit.

5. 4. The defect detection system of claim 3, wherein the non-defective image generating unit generates the virtual non-defective image by removing a defective area from the defect image and filling the corresponding area with an appropriate background.

6. the defect image generating unit generates a plurality of the virtual defect images by modifying a defect area in the defect image or converting a resolution of the defect image into a reference resolution, 4. The defect detection system of claim 3, wherein at least one of a position, a size, or a shape of the defect in the virtual defect image is different from a position, a size, or a shape of the defect in the defect image.

7. 2. The defect detection system of claim 1, wherein the defect detection unit generates a comparison image in which defects are removed from the input image using the model, and detects defects by comparing the generated comparison image with the input image.

8. a learning data generating unit that generates a large number of virtual non-defective images or a large number of virtual defect images by using the defect images; a learning unit that learns a model for defect detection using the generated virtual non-defective product image or the generated virtual defective image; a defect detection unit that detects the presence or absence of a defect in an input image by using the learned model, The defect detection method is characterized in that the learning data generation unit determines the types, number, or resolution of the virtual good product image or virtual defect image to be generated by analyzing the types or distribution of the defect images using artificial intelligence, and determines the types, number, or resolution of the generated virtual good product image or virtual defect image to be different when requested by a user.

9. 10. The defect detection method of claim 8, wherein the learning data generation unit analyzes a type or distribution of the defect image to detect a defect form or a background form, and generates the virtual good image or the virtual defect image with different types, numbers, or resolutions depending on the detected defect form or background form.

10. 9. The defect detection method of claim 8, wherein the non-defective image generating unit generates the virtual non-defective image by removing a defective area from the defect image and filling the corresponding area with an appropriate background.

11. the defect image generating unit generates a plurality of the virtual defect images by modifying a defect area in the defect image or converting a resolution of the defect image into a reference resolution, 9. The method of claim 8, wherein at least one of a position, a size, or a shape of the defect in the virtual defect image is different from a position, a size, or a shape of the defect in the defect image.

12. generating a plurality of virtual non-defective images or a plurality of virtual defect images using the defect images; learning a model for defect detection using the generated virtual non-defective product image or the generated virtual defective image; and detecting the presence or absence of defects in the input image using the learned model; a defect detection method for detecting a defect form or a background form by analyzing a type or distribution of the defect image, and generating the virtual non-defect image or the virtual defect image with different types, numbers or resolutions depending on the detected defect form or background form.

13. 13. The defect detection method of claim 12, wherein the learning data generator extracts hand-crafted features, such as a Haar-like feature or a histogram of oriented gradient, from the defect image and detects the defect shape or the background shape using clustering.

Citation Information

Patent Citations

  • Data analysis device, data analysis method, and program

    JP2017076287A

  • Data collection determination system, determination device, data collection determination device, and program

    JP2021144518A

  • AI-based material defect image generation method and defect detection system

    KR102263716B1