Appearance inspection device and method for generating identifier for appearance inspection

Abstract

The invention provides a visual inspection device and a method for generating a recognizer for visual inspection, wherein the determination accuracy of inspection is improved. The appearance inspection apparatus includes: a storage unit that stores a boundary learning result that generates at least one simulation image that approaches a determination criterion for a defective product and a qualified product in appearance inspection on the basis of a defective product image including a defect determined as a defective product in appearance inspection; a machine learning unit that performs machine learning on the boundary between the defective product and the defective product using a pair of the defective product image and the analog image or a pair of two analog images; and an inspection unit that inspects the surface of the object to be inspected on the basis of the boundary learning result.

Description

Technical Field

[0001] The present invention relates to a visual inspection device and a method for generating an identifier for visual inspection. Background Art

[0002] Patent Document 1 discloses a visual inspection device that generates a single simulated image to determine whether an inspection object is acceptable.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2011-214903 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, in the appearance inspection device of Patent Document 1, it is necessary for a human to judge whether the extracted simulated image is good or bad, and there is a problem that the feature amount is very difficult to apply to the deep learning of the black box.

[0008] One of the objects of the present invention is to provide a visual inspection device and a method for generating a visual inspection identifier that improve the inspection judgment accuracy.

[0009] Means for solving problems

[0010] An appearance inspection device according to one embodiment of the present invention comprises: a storage unit for storing boundary learning results, wherein the boundary learning results are generated based on a defective product image including defects judged as defective products in the appearance inspection, at least one simulation image close to the judgment criteria of defective products and qualified products in the appearance inspection, and the boundary between qualified products and defective products is machine-learned using a pair of the defective product image and the simulation image, or a pair of two simulation images; and an inspection unit for inspecting the surface of the inspected object based on the boundary learning results.

[0011] Therefore, in one embodiment of the present invention, both efficient learning time and high detection accuracy can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is an overall view of the appearance inspection device according to the first embodiment.

[0013] Figure 2 This is a flowchart showing the process of generating boundary machine learning results in implementation mode 1.

[0014] Figure 3 This is the first timing diagram illustrating the actions produced by the boundary machine learning results in Implementation Example 1.

[0015] Figure 4 This is a second timing diagram illustrating the actions produced by the boundary machine learning results in implementation mode 1.

[0016] Figure 5 This is the third timing diagram illustrating the actions produced based on the boundary machine learning results in implementation mode 1.

[0017] Figure 6 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in implementation mode 1.

[0018] Figure 7 This is a flowchart showing the process of producing boundary machine learning results in Implementation Method 2.

[0019] Figure 8 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in Implementation Method 2.

[0020] Fig. 9 This is a flowchart showing the process of producing boundary machine learning results in Implementation Method 3.

[0021] Fig.10 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in Implementation Example 3.

[0022] Fig.11 This is an overall view of a visual inspection device according to a fourth embodiment.

[0023] Fig.12 This is a flowchart showing the process of generating boundary machine learning results in Implementation Method 4.

[0024] Fig.13 This is the third timing diagram illustrating the actions produced based on the boundary machine learning results in Implementation Example 4.

[0025] Fig.14 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in Implementation Example 4. DETAILED DESCRIPTION

[0026] [Implementation method 1]

[0027] Figure 1 This is an overall view of the appearance inspection device according to the first embodiment.

[0028] The appearance inspection device 1 according to the first embodiment includes a camera 2 , a robot 3 , and a computer 4 .

[0029] The camera 2 captures an image of the top surface 5 a of the piston (inspection object) 5 .

[0030] In addition, the top surface 5 a of the piston 5 has a machined surface and a cast surface having greatly different surface properties.

[0031] The robot 3 changes the relative posture of the piston 5 with respect to the camera 2 .

[0032] The computer 4 is, for example, a personal computer and includes a CPU 6 .

[0033] CPU 6 comprises: an inspection image acquisition unit 7; a simulation image generation unit 8; a boundary machine learning unit 9; a first identifier 10 for inspecting the piston top surface (an identifier for appearance inspection), comprising a first storage unit (storage unit) 10a and a first inspection unit (inspection unit) 10b; and a second identifier 11 for boundary machine learning, comprising a second storage unit 11a and a second inspection unit 11b.

[0034] This machine learning is, for example, learning using a neural network, and in the first embodiment, deep learning, which is a multilayered version of a neural network, is adopted.

[0035] Figure 2 This is a flowchart showing the process of generating boundary machine learning results in implementation mode 1.

[0036] In step S1 , a plurality of images of conforming products and one or more images of defective products including a plurality of defects to be detected are acquired from the inspection image acquisition unit 7 (image acquisition step).

[0037] In step S2, boundary machine learning is performed in the boundary machine learning unit 9 based on multiple qualified product images and one or more unqualified product images to be detected, and a threshold (judgment criterion) a (= 0.6) is set based on the boundary machine learning results for product inspection (the boundary b1 of the feature quantity) (threshold setting step).

[0038] In step S3, boundary machine learning is performed using the boundary machine learning unit 9 based on multiple qualified product images and unqualified product images including a defect to be detected, and the boundary machine learning results specific to the defect to be detected are stored in the second storage unit 11a of the second identifier 11 (defect-specific boundary machine learning step).

[0039] In step S4, the feature value of the defect image to be detected is changed to produce a plurality of simulated shapes, and a plurality of simulated defect images are produced based on the produced simulated shapes. At positions that at least partially overlap with the location where the defect to be detected is located, or at multiple positions where the defect to be detected is located, the plurality of simulated defect images are arranged on the qualified product image by the simulated image generating unit 8 to produce a plurality of simulated images (simulated defect image production and acquisition step, simulated image production step).

[0040] In step S5, the plurality of simulated images prepared are inspected in the second inspection unit 11b based on the boundary machine learning result dedicated to the defect to be detected stored in the second storage unit 11a (simulated conforming product image extraction step).

[0041] In step S6, one or more simulated conforming product images near a threshold value a of conforming products / defective products are extracted based on the inspection results (simulated conforming product image extraction step).

[0042] In step S7, it is confirmed whether there are other defects that need to be detected.

[0043] If it exists, return to step S1, if not, proceed to step 8.

[0044] As a result, when there are new defects that need to be detected (specified triggers), the boundary machine learning results can be updated in a timely manner.

[0045] In step S8, the defective product image and the simulated qualified product image near the threshold a are paired, and boundary machine learning for product inspection is performed again in the boundary machine learning unit 9 to correct the boundary learning result, and the corrected boundary learning result (the boundary b of the feature quantity) is stored in the first storage unit 10a and the second storage unit 11a (step of obtaining the boundary learning result again).

[0046] Figure 3 is a first sequence diagram illustrating the actions produced by the boundary machine learning results in Implementation Method 1, Figure 4 is a second timing diagram illustrating the actions produced by the boundary machine learning results in Implementation Method 1, Figure 5 is a third timing diagram illustrating the actions produced by the boundary machine learning results in Implementation Method 1, Figure 6 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in implementation mode 1.

[0047] In the image acquisition step, a plurality of images of conforming products and one or more images of defective products including a plurality of defects to be detected are acquired from the inspection image acquisition unit 7 .

[0048] In the threshold (determination criterion) setting step, boundary machine learning is performed based on multiple qualified product images acquired from the inspection image acquisition unit 7 and one or more unqualified product images including multiple defects to be detected, and the boundary machine learning result is acquired.

[0049] Furthermore, based on the obtained boundary machine learning results, scores (similarity) are used to sort out qualified products and unqualified products, and a threshold a between qualified products and unqualified products is set, for example, to 0.6.

[0050] In the defect-specific boundary machine learning step, boundary machine learning is performed using the boundary machine learning unit 9 based on multiple qualified product images and unqualified product images including a defect to be detected, and the boundary machine learning results specific to the defect to be detected are stored in the second storage unit 11a of the second identifier 11.

[0051] Figure 5 This is the third timing diagram illustrating the actions produced based on the boundary machine learning results in implementation mode 1.

[0052] In the simulated defect image step, the characteristic quantities of the defect (angle A, defect brightness B, major axis L, minor axis S) of the defect image to be detected are changed to create simulated defect images of a plurality of simulated shapes.

[0053] In addition, Figure 5 Only 6 kinds are described in the literature, but about 20 kinds are produced.

[0054] In the simulation image preparation step, the simulation image generation unit 8 arranges multiple simulated defect images on the qualified product image at a position or multiple positions that at least partially overlap with the location where the defect of the defect image to be detected is located to prepare multiple simulation images.

[0055] Since multiple simulated shapes are created by changing the characteristic quantities of the defect (angle A, defect brightness B, major axis L, minor axis S) of the defect image to be detected, multiple simulated images based on multiple simulated shapes can be easily created.

[0056] In addition, since multiple simulated defect images are arranged on a qualified product image to produce multiple simulated images at a position or multiple positions that at least partially overlap with the location where the defect of the defect image to be detected is located, it is possible to extract only the feature value of the defective part after removing the background and position information, and find a higher precision pair.

[0057] Figure 6 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in implementation mode 1.

[0058] In the simulated qualified product image extraction step which is a step of generating at least one simulated image close to the threshold value a (=0.6) between qualified products and defective products, the multiple simulated images produced are inspected in the second inspection unit 11b based on the boundary machine learning result dedicated to the defect to be detected stored in the second storage unit 11a, and based on the inspection result, the qualified products and defective products are sorted by scores (similarity), and one or more simulated qualified product images near the threshold value a (=0.6) between qualified products and defective products are extracted.

[0059] In the step of obtaining the boundary learning result again, which is a step for obtaining an identifier for appearance inspection, the defect image and the simulated qualified product image near the threshold a (=0.6) are set as a pair, and boundary machine learning for product inspection is performed again in the boundary machine learning unit 9, and the boundary learning result is corrected. The corrected boundary learning result (the boundary b of the feature value) is stored in the first storage unit 10a and the second storage unit 11a.

[0060] Thus, the target boundary b0 can be made roughly consistent with the boundary b based on the boundary learning result. By storing the corrected boundary learning result in the first storage unit 10a, the boundary learning result can be efficiently constructed, and the inspection accuracy of qualified / defective products of the top surface 5a of the piston 5 performed by the first identifier 10 can be highly accurate.

[0061] Next, the effects of the first embodiment will be described.

[0062] In the first embodiment, the following effects are achieved.

[0063] (1) The first storage unit 10a of the first identifier 10 of the appearance inspection device 1 stores a boundary learning result (boundary b of the feature quantity), which is obtained by pairing a defect image with a simulated qualified product image near a threshold a and performing boundary machine learning in the boundary machine learning unit 9. The inspection unit 10b of the first identifier 10 inspects the top surface 5a of the piston 5 based on the stored boundary learning result (boundary b of the feature quantity).

[0064] Therefore, the target boundary b0 can be made roughly consistent with the boundary b based on the boundary learning result, the boundary learning result can be efficiently constructed, and the inspection accuracy of the top surface 5a of the piston 5 by the first identifier 10 for good / bad products can be improved.

[0065] (2) When there are other defects that you want to detect, perform boundary machine learning again.

[0066] Therefore, the boundary machine learning results can be updated in a timely manner.

[0067] (3) The characteristic quantities of the defect (angle A, defect brightness B, major axis L, minor axis S) of the defect image to be detected are changed to create a plurality of simulated defect images of simulated shapes.

[0068] Therefore, a plurality of simulated defect images prepared based on a plurality of simulated shapes can be easily prepared.

[0069] (4) A plurality of simulated defect images created based on a plurality of simulated shapes are arranged on a conforming product image at a position or positions that at least partially overlap with a location where a defect of a defect image to be detected is located, thereby creating a plurality of simulated images.

[0070] Therefore, it is possible to extract only the feature amount of the defect after removing the background and position information, and to find a pair with higher accuracy.

[0071] [Implementation method 2]

[0072] Figure 7 is a flowchart showing the process of generating the boundary machine learning results in Implementation Method 2, Figure 8 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in Implementation Method 2.

[0073] In implementation mode 1, the defect image and the simulated qualified product image near threshold a are paired, boundary machine learning for product inspection is performed again in the boundary machine learning unit 9, the boundary learning result is corrected, and the corrected boundary learning result (the boundary b of the feature value) is stored in the first storage unit 10a and the second storage unit 11a. However, in implementation mode 2, the simulated defect image near threshold a and the simulated qualified product image near threshold a are paired, boundary machine learning for product inspection is performed again in the boundary machine learning unit 9, the boundary learning result is corrected, and the corrected boundary learning result (the boundary b of the feature value) is stored in the first storage unit 10a and the second storage unit 11a.

[0074] That is, Figure 7 , Figure 8 As shown, the simulated conforming product image and simulated defective product image extraction step is substituted for the simulated conforming product image extraction step of the first embodiment.

[0075] The other structures are the same as those in Implementation Example 1, and therefore the same structures are denoted by the same reference numerals and the description thereof is omitted.

[0076] Therefore, in the second embodiment, the same functions and effects as those of the first embodiment are achieved.

[0077] [Implementation method 3]

[0078] Fig. 9 is a flowchart showing the process of generating the boundary machine learning results in Implementation Method 3, Fig.10 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in Implementation Example 3.

[0079] In implementation mode 1, a defective product image and a simulated defective product image near threshold a are paired, boundary machine learning for product inspection is performed again in the boundary machine learning unit 9, the boundary learning result is corrected, and the corrected boundary learning result (boundary b of the feature value) is stored in the first storage unit 10a and the second storage unit 11a. However, in implementation mode 3, a defective product image and a simulated defective product image near threshold a are paired, boundary machine learning for product inspection is performed again in the boundary machine learning unit 9, the boundary learning result is corrected, and the corrected boundary learning result (boundary b of the feature value) is stored in the first storage unit 10a and the second storage unit 11a.

[0080] That is, Fig. 9 , Fig.10 As shown, the simulated non-conforming product image extraction step is replaced by the simulated non-conforming product image extraction step in the first embodiment.

[0081] The other structures are the same as those in Implementation Example 1, and therefore the same structures are denoted by the same reference numerals and the description thereof is omitted.

[0082] Therefore, in the third embodiment, the same functions and effects as those of the first embodiment are achieved.

[0083] [Implementation method 4]

[0084] Fig.11 This is an overall view of a visual inspection device according to a fourth embodiment.

[0085] The appearance inspection device according to the fourth embodiment further includes a similarity calculation unit 12 in addition to the appearance inspection device according to the first embodiment.

[0086] Fig.12 is a flowchart showing the process of generating the boundary machine learning results in Implementation Method 4, Fig.13 is a third timing diagram illustrating the actions produced by the boundary machine learning results in Implementation Method 4, Fig.14 This is the fourth timing diagram illustrating the actions produced based on the boundary machine learning results in Implementation Example 4.

[0087] like Fig.12 As shown in the simulated defect image preparation step and the simulated image preparation step, i.e., step S4a, the score estimation formula is calculated using the least squares method in the similarity calculation unit 12 to prepare multiple simulated qualified product images near the threshold value a obtained by changing the feature value of the defect image to be detected.

[0088] Specifically, if Fig.13As shown, the feature values ​​(angle A, defect brightness B, major diameter L, minor diameter S) are changed and a score estimation formula is used to generate multiple simulated qualified product images near the threshold value a, and a simulated image generating unit 8 is used to generate multiple simulated images by configuring qualified product images at positions or multiple positions that at least partially overlap with the location where the defect of the unqualified product image is located.

[0089] In addition, Fig.13 Only 4 kinds are described in the literature, but about 20 kinds are produced, for example.

[0090] That is, Fig.14 As shown, in the first embodiment, a defective product image is paired with a simulated conforming product image near the threshold a, but in the fourth embodiment, one defective product image is paired with a plurality of simulated conforming product images near the threshold a.

[0091] The other structures are the same as those in Implementation Example 1, and therefore the same structures are denoted by the same reference numerals and the description thereof is omitted.

[0092] Next, the effects of the fourth embodiment will be described.

[0093] In the fourth embodiment, in addition to the effects of the first embodiment, the following effects are achieved.

[0094] (1) A defective product image and a plurality of simulated good product images near a threshold are paired.

[0095] Therefore, the setting accuracy of the boundary learning result (boundary b of the feature amount) can be improved.

[0096] [Other implementation methods]

[0097] Although the embodiments for implementing the present invention have been described above, the specific configuration of the present invention is not limited to the configuration of the embodiments, and design changes and the like that do not depart from the gist of the invention are also included in the present invention.

[0098] For example, the objects to be inspected are not limited to pistons, and machine learning is not limited to deep learning, which is a multi-layered version of a neural network.

[0099] Furthermore, the posture control unit changes the posture of the piston using the robot, but may also change the position of the camera.

[0100] In the fourth embodiment, one defective product image is paired with a plurality of simulated conforming product images near a threshold value. However, the present invention can also be applied to the simulated images of the second and third embodiments.

[0101] In addition, the present invention is not limited to the above-mentioned embodiments, and includes various modifications. For example, the above-mentioned embodiments are described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to having all the structures described. In addition, a part of the structure of a certain embodiment can be replaced with the structure of other embodiments, and in addition, the structure of other embodiments can be added to the structure of a certain embodiment. In addition, for a part of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0102] This application claims the benefit of priority based on Japanese Patent Application No. 2022-186211 filed on November 22, 2022. The entire disclosure of Japanese Patent Application No. 2022-186211 filed on November 22, 2022, including the specification, claims, drawings, and abstract, is incorporated herein by reference in its entirety.

[0103] Description of Reference Numerals

[0104] 1 appearance inspection device; 10 first identifier (identifier for appearance inspection); 10a first storage unit (storage unit); 10b first inspection unit (inspection unit); 12 similarity calculation unit; a threshold value (judgment standard); b boundary (boundary learning result).

Claims

1. A visual inspection device for inspecting the surface of an object to be inspected to identify qualified products and unqualified products, the visual inspection device comprising: a storage unit storing a boundary learning result obtained by: generating at least one simulated image close to a determination criterion between a defective product and a conforming product in the appearance inspection based on a defective product image including a defect determined as a defective product in the appearance inspection, and performing machine learning on a boundary between a conforming product and a defective product using a pair of the defective product image and the simulated image, or a pair of two simulated images; and The inspection unit inspects the surface of the inspection object based on the boundary learning result.

2. The visual inspection device according to claim 1, wherein: The simulated image is a simulated image of a conforming product that is close to the criteria for determining a defective product and a conforming product in the appearance inspection and is generated in a manner that the product is determined to be a conforming product in the appearance inspection. The boundary learning result is obtained through machine learning using the defective product image and the simulated conforming product image.

3. The visual inspection device according to claim 1, wherein: The simulated image is a simulated defective product image generated so as to be closer to the determination reference than the defective product image but determined as a defective product in the appearance inspection, and a simulated conforming product image generated so as to be determined as a conforming product in the appearance inspection.

4. The visual inspection device according to claim 1, wherein: The appearance inspection device further includes a similarity calculation unit, which generates a plurality of the simulation images, extracts features of the defects based on the plurality of simulation images, and calculates similarities between the plurality of simulation images and a determination criterion for determining defective products and qualified products in the appearance inspection. The appearance inspection device performs machine learning using a specific simulated image selected from the plurality of simulated images based on the similarity.

5. The visual inspection device according to claim 1, wherein: The appearance inspection device is capable of updating the boundary machine learning result according to a specified trigger.

6. The visual inspection device according to claim 1, wherein: The simulated image is generated by changing the feature amount of the defect to generate a simulated shape and arranging the simulated shape in an image.

7. The visual inspection device according to claim 6, wherein: The simulated image is generated by placing the simulated shape at a position at least partially overlapping with a location where the defect is located in a region of the defective product image.

8. The visual inspection device according to claim 7, wherein: The simulation image is generated by arranging the simulation shape at a plurality of positions where the defect is located.

9. A method for generating a visual inspection identifier, wherein the visual inspection identifier inspects the surface of an object to be inspected to identify qualified products and unqualified products, the method comprising the following steps: generating at least one simulated image close to a determination standard between defective products and conforming products in the appearance inspection based on a defective product image including a defect determined as a defective product in the appearance inspection; and A visual inspection identifier is used to identify whether the inspected object is a qualified product based on an inspection image obtained by photographing the inspected object. The visual inspection identifier is obtained by performing machine learning on the boundary between qualified products and unqualified products using a pair of the defect image and the simulated image, or a pair of two simulated images.

Images