Visual inspection device

Abstract

To provide a visual inspection device that can perform good surface defect inspection by compensating for the decrease in judgment accuracy of neural networks due to domain shift. [Solution] The system includes a defect detection unit 11 that performs defect detection by binarizing an inspection image Id of the object's appearance and processing the image; a main AI unit 12 that uses a classification AI model or an object detection AI model and performs defect detection by inputting an inspection image Ida determined to have defects by the defect detection unit 11; and a sub-AI unit 13 that uses an anomaly detection AI model and performs defect detection by inputting the same inspection image Ida as that input to the main AI unit. The system determines that an inspection image that is determined to have no defects by the defect detection unit 11 and an inspection image that is determined to have no defects by both the main AI unit 12 and the sub-AI unit 13 are good product images.

Description

Technical Field

[0001] The present invention relates to an appearance inspection device for inspecting surface defects of a test object.

Background Art

[0002] As this type of appearance inspection device, Patent Document 1 discloses an appearance inspection (flaw detection) device for surface defects. Here, primary to tertiary image determination processing units are provided. In the primary image determination processing unit, the inspection image is binarized by luminance, and defects are determined based on the luminance and shape of the defect candidates. Defect candidates for which determination is impossible are sent to the secondary image determination processing unit, and their statistical feature amounts are calculated. These statistical feature amounts are input to the tertiary image determination processing unit configured by a neural network to determine the defects of the defect candidates.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when a neural network is used in an appearance inspection device, there is a problem that the determination accuracy of the neural network significantly decreases due to so-called domain shift, in which the distribution of the images learned by the neural network does not match the distribution of the images in actual appearance inspection over time.

[0005] Therefore, an object of the present invention is to solve such a problem and provide an appearance inspection device capable of performing good inspection of surface defects by compensating for the decrease in the determination accuracy of the neural network due to domain shift.

Means for Solving the Problems

[0006] To achieve the above objective, the first invention includes a defect detection unit (11) that performs defect detection by binarizing an inspection image (Id) of the appearance of an object and processing the image; a main AI unit (12) that uses a classification AI model or an object detection AI model and performs defect detection by inputting the inspection image (Ida) which has been determined to have a defect by the defect detection unit (11); and a sub-AI unit (13) that uses an anomaly detection AI model and performs defect detection by inputting the same inspection image (Ida) as that input to the main AI unit. The inspection image which has been determined to have no defects by the defect detection unit (11) and the inspection image which has been determined to have no defects by both the main AI unit (12) and the sub-AI unit (13) are determined to have no defects are good product images.

[0007] In this first invention, if no domain shift occurs, the main AI unit functions properly, and some of the inspection images that were judged as defective product images due to over-detection by the defect detection unit are re-judged as good product images. Inspection images that the main AI unit judges as good product images are also judged as good product images by the sub-AI unit, which has inferior defect detection ability compared to the main AI unit.

[0008] On the other hand, in the case of the main AI unit, due to domain shifting, even if the main AI unit determines that the inspection image is a good product image, the sub-AI unit of the anomaly detection AI model will detect defects in the inspection image and determine that it is a defective product image. In this way, the decrease in the judgment accuracy of the neural network due to domain shifting is compensated for.

[0009] In this second invention, the main AI unit (12) is composed of a supervised neural network, and the sub-AI unit (13) is composed of an unsupervised neural network.

[0010] In this second invention, the main AI unit exhibits good defect detection capabilities, while the sub-AI unit can be easily retrained, thus maintaining its defect detection capabilities without any decrease in performance.

[0011] In this third invention, the unsupervised neural network of the sub-AI unit periodically receives the inspection image (Idn) that has been determined to be free of defects by the defect detection unit (11) as training data.

[0012] In this third invention, high-quality images of good products that have been determined to be defect-free by the defect detection unit in an over-detection state are periodically input as training data for the unsupervised neural network, and retraining is performed, so that the defect detection capability of the sub-AI unit is maintained without any decrease.

[0013] The symbols in parentheses above are for reference only, indicating the correspondence with the specific means described in the embodiments described later. [Effects of the Invention]

[0014] As described above, the visual inspection apparatus of the present invention can compensate for the decrease in judgment accuracy of the neural network due to domain shift and perform good surface defect inspection. [Brief explanation of the drawing]

[0015] [Figure 1] This is a block diagram showing the configuration of a visual inspection device. [Modes for carrying out the invention]

[0016] The embodiments described below are merely examples, and various design improvements made by those skilled in the art without departing from the spirit of the present invention are also included within the scope of the present invention.

[0017] Figure 1 shows the overall configuration of the visual inspection device. The visual inspection device 1 has a built-in computer and includes a defect detection unit 11, a main AI unit 12, a sub-AI unit 13, and a memory unit 14, all of which are implemented by the computer.

[0018] The defect detection unit 11 receives an inspection image Id of the object's appearance, performs preprocessing such as cropping the inspection area, brightness correction, posture correction, and background removal, then binarizes the image and calculates geometric features such as the area and length of particles in the image, as well as statistical quantities such as the average brightness and variation of the particles. When the calculated values ​​exceed a threshold, it detects a defect. The defect detection threshold in the defect detection unit 11 is set low as a safety precaution. As a result, only inspection images with no defects or only very small defects that do not cause problems are judged as good images, and a good product judgment signal 11a at the "1" level is output. The defective product judgment signal is at the "0" level. However, this means that inspection images with only harmless small defects (e.g., dust) are judged as defective images, leading to the problem of so-called overdetection.

[0019] Therefore, in this embodiment, the inspection image Ida, which has been determined to be a defective product image by the defect detection unit 11, is input in parallel to the main AI unit 12 and the sub AI unit 13, and each unit performs the determination again.

[0020] The main AI unit 12 can use a classification model or an object detection model; specifically, a supervised neural network such as YOLO can be used for object detection. On the other hand, the sub-AI unit 13 can use an anomaly detection model; specifically, an unsupervised neural network such as PaDiM or PatchCore can be used.

[0021] Furthermore, the memory unit 14 stores inspection image Idn, which has been determined to be a good product image by the defect detection unit 11, while periodically updating it to the required number of images (for example, 200 images). These good product image Idn are then used as training data for the sub-AI unit 13, which is an unsupervised neural network.

[0022] In the appearance inspection device 1 with such a configuration, the inspection image Ida determined to be a defective product image with defects by the defect detection unit 11 set in the over-detection state is input to the main AI unit 12, and whether it is a non-defective product image is re-determined by a supervised neural network that has been sufficiently pre-trained with the required number of annotated non-defective product images (for example, 20,000 images) and defective product images as teacher data. As a result, some of the inspection images Ida determined to be defective product images due to over-detection by the defect detection unit 11 are re-determined as non-defective product images, and a non-defective product determination signal 12a at the "1" level is output. This relaxes the over-detection state of the defect detection unit 11.

[0023] By the way, over time, due to changes in the appearance of the object, etc., domain shift of the supervised neural network constituting the main AI unit 12 becomes a problem. This is because the determination accuracy of the main AI unit 12 decreases due to domain shift.

[0024] Here, by providing the sub-AI unit 12, a decrease in determination accuracy due to domain shift is prevented. This will be explained below. That is, the sub-AI unit 12 is an unsupervised neural network. For example, when using PadiM, its learning is to obtain the covariance matrix Σ and the mean μ of the feature vectors for each pixel of all the non-defective product images Idn imported from the memory unit 14 into this. This is simpler compared to the learning of the supervised neural network and re-learning can be performed in a relatively short cycle.

[0025] Then, for the inspection image Ida determined to be a defective product image and input to the sub-AI unit 13 simultaneously with the main AI unit 12, the sub-AI unit 12 calculates the feature vector for each pixel, obtains the Mahalanobis distance at each pixel, and re-determines whether it is a non-defective product image or a defective product image with the part where the Mahalanobis distance exceeds the threshold as the defective part.

[0026] The unsupervised neural network in the sub-AI unit 13 has sufficient ability to detect large defects, but its ability to detect small defects is generally low. Therefore, inspection image Ida, which is normally judged as a good product image by the main AI unit 12, is also judged as a good product image by the sub-AI unit 13, and a good product judgment signal 13a at the "1" level is issued.

[0027] Here, if a domain shift occurs in the supervised neural network of the main AI unit 12, the inspection image Ida, which is judged as a defective product image by the sub-AI 13, which has relatively low defect detection ability, may be judged as a good product image by the main AI unit 12. In this case, the output signal 14a of the AND gate 14 becomes "0" level, and the output signal 11a of the defect detection unit 11 is also at "0" level, so the output signal 15a of the OR gate 15 becomes a "0" level defective product judgment signal, preventing the shipment of the defective product. At the same time, it is thought that the defect detection ability of the main AI unit 12 has decreased due to a domain shift, and the need for retraining of the main AI unit 12 becomes clear.

[0028] This also serves a foolproof function, ensuring that even if a major defect is missed by chance by the main AI unit 12, the defect will be reliably detected by the sub-AI unit 12, which is regularly retrained using images of good products.

[0029] In the above embodiment, it is not necessarily required to periodically retrain the unsupervised neural network in the sub-AI unit. Furthermore, the combination of the classification AI model or object detection AI model in the main AI unit and the anomaly detection AI model in the sub-AI unit is not limited to those shown in the above embodiment. Furthermore, if the algorithms are different, both the main AI unit and the sub-AI unit can be used as either a classification AI model or an object detection AI model. [Explanation of symbols]

[0030] 1...Visual inspection device, 11...Defect detection unit, 12...Main AI unit, 13...Sub AI unit, Id...Inspection image, Ida...Inspection image determined to have a defect, Idn...Inspection image determined to have no defects.

Claims

1. An appearance inspection device comprising: a defect detection unit that performs defect detection by binarizing an inspection image of the appearance of an object and processing the image; a main AI unit that uses a classification AI model or an object detection AI model to perform defect detection by inputting the inspection image determined to have defects by the defect detection unit; and a sub-AI unit that uses an anomaly detection AI model to perform defect detection by inputting the same inspection image as that input to the main AI unit, wherein the inspection image determined to be free of defects by the defect detection unit and the inspection image determined to be free of defects by both the main AI unit and the sub-AI unit are determined to be free of defects.

2. The appearance inspection apparatus according to claim 1, wherein the main AI unit is composed of a supervised neural network and the sub-AI unit is composed of an unsupervised neural network.

3. The appearance inspection apparatus according to claim 2, wherein the unsupervised neural network of the sub-AI unit periodically inputs the inspection images determined to be free of defects by the defect detection unit as training data.

Images