Appearance inspection method and appearance inspection device

Abstract

The present invention determines, as a defect that is at high risk of being overlooked, a defect in which, in a first AI model after re-training, the extent of reduction in a failure probability before and after the re-training is greater than a prescribed value, and determines the quality by using a second AI model trained on the defect that is at high risk of being overlooked.

Description

Visual inspection method and visual inspection device

[0001] The present invention relates to a visual inspection method and a visual inspection device.

[0002] Patent Document 1 discloses a visual inspection device equipped with two pass / fail judgers. The first judger judges pass / fail using an AI model trained before the product is put into mass production, while the second judger judges pass / fail using an AI model trained after the product is put into mass production. This visual inspection device changes the weights of both judgers depending on the similarity between the judgment results of both judgers, and makes a final pass / fail judgment.

[0003] Japanese Patent Publication No. 2020-107104

[0004] Normally, with an appearance inspection device using an AI model, if a verification defective product is judged as NG before being applied to mass production, it can be applied to mass production, but because there is not enough verification time and the risk of overlooking something is high, a visual double-check by an operator is required, which is disadvantageous in terms of cost.One of the objects of the present invention is to provide an appearance inspection method and appearance inspection device that can omit the visual double-check by an operator.

[0005] In an appearance inspection method according to one embodiment of the present invention, a defect whose defect probability has decreased by a greater than predetermined amount before and after relearning in a first AI model is determined to be a defect with a high risk of being overlooked, and a pass / fail judgment is then made again using a second AI model that has been trained on defects with a high risk of being overlooked.

[0006] According to one embodiment of the present invention, visual double-checking by an operator can be omitted.

[0007] FIG. 5 is a schematic diagram of the appearance inspection device 1 of embodiment 1. FIG. 6 is a flowchart showing the flow of the appearance inspection method by the CPU 7 of embodiment 1. FIG. 7 is an example of the change in defect probability before and after relearning of the first AI model. FIG. 8 is a schematic diagram showing the true OK / NG boundary and the OK / NG boundary generated by the first AI model after initial learning. FIG. 9 is an image diagram showing the change in the OK / NG boundary before and after relearning in the first AI model. FIG. 10 is an enlarged view of the main part of FIG. 10. FIG. 11 is an example of the change in defect probability before and after relearning of the first AI model. FIG. 12 is a flowchart showing the flow of the appearance inspection method by the CPU 7 of embodiment 3. FIG. 13 is a flowchart showing the flow of the appearance inspection method by the CPU 7 of embodiment 4.

[0008] [Embodiment 1] FIG. 1 is a schematic diagram of an appearance inspection apparatus 1 of embodiment 1. The appearance inspection apparatus 1 of embodiment 1 includes a camera (imaging unit) 2, a robot 3, and a computer 4. The camera 2 captures an image of a crown surface 5a of a piston (inspection object) 5. The robot 3 changes the relative attitude (angle) of the piston 5 with respect to the camera 2. The computer 4 is, for example, a personal computer, and includes a memory 6 and a CPU (processing unit) 7. The memory 6 stores the results of machine learning performed using multiple images of defective product samples. The machine learning is AI learning using a neural network, and in embodiment 1, learning by deep learning is adopted.

[0009] Based on a program stored in memory 6, CPU 7 inspects whether or not there are defects (blobs, scratches, dents, etc.) on crown surface 5a of piston 5 from the image captured by camera 2 and the AI ​​learning results stored in memory 6. Pistons 5 inspected by appearance inspection device 1 are placed on a belt conveyor (not shown), and those judged OK (determined to have no defects) are transported to a shipping preparation point, while those judged NG (determined to have defects) are removed from a disposal chute.

[0010] 2 is a flowchart showing the flow of the visual inspection method by the CPU 7 of the first embodiment. In step S1, an image of the crown surface 5a of the piston 5 (image of the object to be inspected) is acquired by the camera 2 (imaging step). In step S2, a defect probability is calculated for the image of the crown surface 5a using a first AI model, and a pass / fail judgment (OK or NG) is made based on the defect probability (pass / fail judgment step). The first AI model judges the defect probability as OK if it is equal to or less than a preset threshold, and judges the defect probability as NG if it exceeds the threshold.

[0011] The first AI model extracts multiple feature quantities (brightness, brightness gradient, geometric characteristics, brightness variation, color intensity, etc.) from the image of the crown surface 5a and compares them with the AI ​​learning results stored in memory 6. Before mass production of the pistons 5, the first AI model learns the boundaries between the OK and NG areas (OK / NG boundaries) for each feature quantity using OK images and NG images for initial learning. The first AI model compares each feature quantity of the captured image of the crown surface 5a with the learning results, and calculates a higher probability of defect as the probability of NG increases.

[0012] In step S3, if the result of step S2 is judged to be OK, the process proceeds to step S7, and if the result of step S2 is judged to be NG, the process proceeds to step S4. In step S4, external information is used to determine whether the NG result by the first AI model is a false alarm (verification step). Here, the external information may be information from a source that judges pass / fail using at least a different logic from the first AI model. In step S5, if the result of step S4 is judged to be a false alarm, the process proceeds to step S6, and if the result is not judged to be a false alarm, the process proceeds to step S10.

[0013] In step S6, the image determined to be a false alarm or a pseudo image that simulates a similar image is used as training data (additional OK images for re-learning), and the first AI model is re-trained (first re-learning step). In step S7, the second AI model is used to calculate the defect probability for the image of the crown surface 5a, and a pass / fail judgment is made based on the defect probability (re-judgment step). Here, the second AI model is created for each defect type, and an oversight judgment is made for each defect type.

[0014] The second AI model determines defects with a high risk of overlooking if the first AI model, after re-learning due to a false alarm determination, shows a greater than predetermined decrease in defect probability before and after re-learning as defects with a high risk of overlooking, and learns about defects with a high risk of overlooking. For example, in the example shown in Figure 3, the defect probability of all workpieces is above the threshold, so all are judged as NG even after re-learning. However, workpieces 2, 5, and 7 are judged to have defects with a high risk of overlooking, and the second AI model trains by treating the images of workpieces 2, 5, and 7 as NG. For defects with a high risk of overlooking, the second AI model generates pseudo-defect images by superimposing similar pseudo-defects in multiple locations on the image of the top surface 5a, and generates the second AI model using these pseudo-defect images as training data. During training of the second AI model, learning hyperparameters, represented by the reduction rate, number of learning iterations, and batch size, are optimized.

[0015] In step S8, if the result of step S6 is judged as OK, the process proceeds to step S9, and if the result of step S6 is judged as NG, the process proceeds to step S10. In step S9, the piston 5 is transported to a shipping preparation point as a shipping item. In step S10, the piston 5 is discharged as a waste item.

[0016] Next, the effects of embodiment 1 will be described. In an AI-based visual inspection device, before mass production (applied to a manufacturing line), initial training of the AI ​​model is performed using initial training images (good) and initial training images (bad). The initial training images and verification images are all judged as bad, and a sufficient verification period is required to confirm that there are no problems with inspection accuracy. If a false report occurs after mass production, in which a good product is judged as bad, the AI ​​model is corrected by retraining the image that caused the false report. During retraining, the AI ​​model is applied to mass production after confirming that all initial training images and verification images are judged as bad. Since a sufficient verification period cannot be ensured, there is a high risk of oversight, which could lead to concerns about reduced product reliability. This requires a human operator to visually double-check for oversights, which increases manufacturing costs.

[0017] In contrast, in the visual inspection method of embodiment 1, the first AI model after re-learning through false alarm verification determines that defects for which the decrease in defect probability before and after re-learning is greater than a predetermined value are defects with a high risk of being overlooked, and the second AI model is trained on the defects with a high risk of being overlooked, and then the second AI model is used to perform a pass / fail judgment again. The effect of reducing the risk of overlooking defects by the second AI model will be described in detail below.

[0018] Figure 4 is an illustration of the true OK / NG boundary and the OK / NG boundary generated by the first AI model after initial training. The outside of the OK / NG boundary is the OK area, and the inside is the NG area. The defect probability is calculated in the range of 0 to 1 depending on the distance from the initial training OK / NG boundary. In other words, an image that is in the OK area but far from the OK / NG boundary is calculated to have a low defect probability. Conversely, an image that is in the NG area and far from the boundary is calculated to have a high defect probability. Note that in the example in Figure 4, the OK / NG boundary is set using two feature values ​​within the captured image to make it easier to illustrate the OK / NG boundary. As shown in Figure 4, the OK / NG boundary generated by the first AI model trained using initial training OK images and initial training NG images as training samples is roughly similar to the true OK / NG boundary, but deviations are observed in areas where training samples are sparse.

[0019] Subsequently, if a false alarm occurs in the first AI model, the first AI model is re-trained using the additional OK image for re-training (the image in which the false alarm occurred or a similar pseudo image) as a training sample, as shown in Figure 5. This changes the OK / NG boundary of the first AI model from the initial OK / NG boundary to a re-trained OK / NG boundary that includes the defect in the additional OK image for re-training in the OK area. At this time, the boundary is changed to be more inward than the true OK / NG boundary, resulting in a missed area as shown in Figure 5.

[0020] In the visual inspection method using the visual inspection apparatus 1 of embodiment 1, the second AI model is trained using sample images of defects with a high risk of overlooking defects (NG images for overlooking-specific learning). As shown in FIG. 6, defects whose distance to the OK / NG boundary has decreased after re-learning false alarms (defects whose distance to the OK / NG boundary in the re-learning process is closer than that to the OK / NG boundary in the initial learning process) have a significantly lower defect probability than other defects. Focusing on this point, the visual inspection method of embodiment 1 trains the second AI model based only on defect images with a high risk of overlooking defects. This increases the weighting of products with a reduced defect probability, enabling the generation of a learning model that fully covers the surrounding area, thereby enabling the aforementioned overlooking area to be reduced effectively and quickly.

[0021] If the second AI model is not provided and the first AI model is made to re-learn defect images with a high risk of being overlooked, the NG area to be covered will be large, resulting in partial overlooked areas and making it impossible to obtain the same effect as in embodiment 1. As described above, the visual inspection method of embodiment 1 has the significant effect of eliminating the need for a worker to visually double-check.

[0022] In addition, in embodiment 1, a second AI model is provided for each defect type with a high risk of being overlooked, and an oversight judgment is performed for each defect type, making it possible to generate a learning model with higher sensitivity. Furthermore, in embodiment 1, supervised learning is performed in which the second AI model is taught NG products and OK products, so that feature amounts of defect images with a high risk of being overlooked can be extracted and similar samples can be detected more accurately than in unsupervised learning in which only OK products are taught.

[0023] Additionally, in the first embodiment, the second AI model is trained by optimizing learning hyperparameters, such as the reduction rate, the number of iterative learning iterations, and the batch size. The learning hyperparameters are parameters used to improve the degree of agreement between the correct answer and the inference result, and by applying these to the visual inspection method of the first embodiment, an AI model with a high degree of separation between OK and NG can be created. Therefore, a more accurate second AI model can be created, further reducing the proportion of false alarms among discarded products.

[0024] In the first embodiment, a pseudo defect image is generated by superimposing pseudo defects similar to defects with a high risk of being overlooked at multiple locations on the image of the crown surface 5a, and the pseudo defect image is used as learning data for the second AI model. That is, by learning using a pseudo defect image in which pseudo defects of defects with a high risk of being overlooked are arranged over the entire crown surface 5a, the occurrence of location-dependent oversight can be reduced and the comprehensiveness of detection can be improved. Therefore, the risk of oversight can be reduced at an earlier stage after mass production.

[0025] [Embodiment 2] The basic configuration of embodiment 2 is the same as embodiment 1, and therefore only the differences from embodiment 1 will be described. In embodiment 2, as in embodiment 1, learning is performed on defects with a high risk of being overlooked, and in the first AI model after re-learning following a false alarm determination, defects for which the increase in the defect probability before and after re-learning is greater than a predetermined value (see FIG. 7 ) are determined to be defects with a high risk of false alarm, and the system is caused to learn on defects with a high risk of false alarm.

[0026] Good products with similar features to good products whose defect probability has increased after re-learning have a defect probability closer to the threshold than before re-learning, so there is a risk that they will be judged as NG and produce a false report. Therefore, by teaching good products with an increased defect probability, i.e., good products with a high risk of false reports, as learning data, it is possible to generate a second AI model with a low false report rate.

[0027] [Embodiment 3] The basic configuration of embodiment 3 is the same as embodiment 1, so only the differences from embodiment 1 will be described. Fig. 8 is a flowchart showing the flow of an appearance inspection method by a CPU 7 according to embodiment 3. Note that steps that perform the same processing as in the flowchart of Fig. 2 are given the same step numbers, and descriptions thereof will be omitted.

[0028] In step S11, external information is used to determine whether the NG judgment by the second AI model is a false defect. Here, the external information may be information from at least a source that performs pass / fail judgment using logic different from that of the second AI model. In step S12, if step S11 determines that the defect is a false defect, the process proceeds to step S13, and if it determines that the defect is not a false defect, the process ends. In step S13, the image determined to be a false defect or a pseudo-defect image is used as a false defect sample, and the first AI model is re-trained using the false defect sample (second re-training step).

[0029] As described above, in the visual inspection method of embodiment 3, the risk of oversight in the first AI model that performs primary judgment can be reduced by re-learning the first AI model using oversight samples, thereby improving the coverage of NG samples and generating a first AI model with higher detection accuracy.

[0030] [Embodiment 4] The basic configuration of embodiment 4 is the same as embodiment 3, so only the differences from embodiment 3 will be described. Fig. 9 is a flowchart showing the flow of an appearance inspection method by a CPU 7 of embodiment 4. Note that steps that perform the same processing as in the flowchart of Fig. 8 are assigned the same step numbers and their descriptions will be omitted. In step S21, an image determined to be a false alarm or a pseudo-defect image is set as a false alarm sample, and the second AI model is re-learned using the false alarm sample (third re-learning step).

[0031] As described above, in the visual inspection method of embodiment 4, by re-learning the second AI model using false alarm samples, it is possible to reduce the false alarm rate in the second AI model, thereby obtaining a visual inspection method with high accuracy and high yield.

[0032] [Other Embodiments] Although the embodiments for carrying out the present invention have been described above, the specific configuration of the present invention is not limited to the configuration of the embodiments, and design changes and the like that do not deviate from the gist of the invention are also included in the present invention.

[0033] For example, the object to be inspected is not limited to a piston. Also, machine learning is not limited to deep learning, which is a multilayered neural network. Furthermore, the method for calculating the defect probability is arbitrary.

[0034] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0035] This application claims priority to Japanese Patent Application No. 2024-108585, filed July 5, 2024. The entire disclosure of Japanese Patent Application No. 2024-108585, filed July 5, 2024, including the specification, claims, drawings, and abstract, is incorporated herein by reference in its entirety.

[0036] 2 Camera (imaging unit), 5 Piston (inspection object), 7 CPU (processing unit)

Claims

1. A visual inspection method comprising: an imaging step of obtaining an image of an object to be inspected by imaging the object to be inspected; a pass / fail judgment step of calculating a defect probability for the image of the object to be inspected using a first AI model and making a pass / fail judgment based on the defect probability; a verification step of verifying whether the pass / fail judgment made by the first AI model is a false alarm using external information; a first relearning step of re-training the first AI model when the pass / fail judgment made by the first AI model is determined to be a false alarm by the verification step; and a re-judgment step of judging, in the first AI model after the first relearning step, that defects for which the decrease in the defect probability before and after the relearning is greater than a predetermined amount are to be defects with a high risk of being overlooked, having a second AI model train on the defects with a high risk of being overlooked, and making a pass / fail judgment again for the image of the object to be inspected using the second AI model.

2. A visual inspection method according to claim 1, wherein the second AI model is provided for each defect type, and the re-evaluation step performs a pass / fail judgment for each defect type.

3. A visual inspection method according to claim 1 or 2, wherein the first re-learning step determines that a defect for which the increase in the defect probability before and after re-learning is greater than a predetermined value is a defect with a high risk of false alarm, and causes the second AI model to learn about the defects with a high risk of false alarm.

4. A visual inspection method according to any one of claims 1 to 3, wherein the first re-learning step uses, as learning data for the second AI model, pseudo-defect images in which pseudo-defects similar to defects with a high risk of being overlooked are synthesized at multiple locations on the image of the object to be inspected.

5. A visual inspection method according to any one of claims 1 to 4, comprising a second re-learning step of re-learning the first AI model using image data of missed samples extracted from samples judged as NG by the re-evaluation step.

6. A visual inspection method according to claim 5, further comprising a third re-learning step of re-learning the second AI model using image data of false alarm samples extracted from samples judged as NG by the verification step.

7. A visual inspection method according to any one of claims 1 to 6, wherein the re-evaluation step performs supervised learning to teach the second AI model which products are acceptable and which are not.

8. A visual inspection method according to claim 7, wherein the re-evaluation step optimizes learning hyperparameters, represented by a reduction rate, a number of learning iterations, and a batch size, for the second AI model and causes the model to learn.

9. An appearance inspection device comprising: an imaging unit that obtains an image of an object to be inspected by imaging an object to be inspected; and a processing unit that processes the image of the object to be inspected, which determines a defect probability for the image of the object to be inspected using a first AI model and makes a pass / fail judgment based on the defect probability, verifies using external information whether the pass / fail judgment by the first AI model is a false alarm, and when the pass / fail judgment by the first AI model is judged to be a false alarm, re-trains the first AI model, and in the first AI model after re-training, judges defects for which the decrease in the defect probability before and after re-training is greater than a predetermined amount as defects with a high risk of being overlooked, trains a second AI model with regard to the defects with a high risk of being overlooked, and re-judges the image of the object to be pass / fail using the second AI model.

Images