Appearance inspection device and appearance inspection method

Abstract

The appearance inspection device inspects the surface of the piston on the basis of a captured image of the surface of the piston and a learning result obtained by machine learning using a plurality of defective sample images. The plurality of defective product sample images are generated by combining a defect element image generated by overlapping a plurality of predetermined shapes with a surface image of a sample equivalent to a piston.

Description

Technical Field

[0001] The present invention relates to a visual inspection device and a visual inspection method. Background Art

[0002] Patent Document 1 discloses a technique for inspecting the surface of an object to be inspected based on a captured image of the surface of the object to be inspected and a learning result obtained by machine learning using a plurality of defective product sample images.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2022-135248 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] In the above-mentioned conventional appearance inspection device, the construction of a machine learning model requires a large number of sample images of defective products. However, since the frequency of occurrence of defective products is low, there is a concern that the collection process will become cumbersome.

[0008] One object of the present invention is to provide a visual inspection device and a visual inspection method that can reduce the workload of collecting defective product samples.

[0009] Means for solving problems

[0010] In one embodiment of the present invention, a visual inspection device inspects the surface of an object to be inspected based on a captured image of the surface of the object to be inspected and a learning result obtained by machine learning using multiple artificial defect images, wherein the multiple artificial defect images are generated by synthesizing defect element images made by overlapping multiple specified shapes and a surface image of a sample equivalent to the object to be inspected.

[0011] According to one embodiment of the present invention, since the collection of defective product samples is not required, the collection architecture of defective product samples can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram of the appearance inspection device 1 according to the first embodiment.

[0013] Figure 2 This is an example of the initial shape E1 and the final shape E2 of an ellipse.

[0014] Figure 3 It is used Figure 2 An example of a generated image of an elliptical blow hole defect element is shown.

[0015] Figure 4 This is an example of a patch image (partial image, image block) of a defect element such as a pore.

[0016] Figure 5 It is used Figure 2 The elliptical defect elements shown are other examples of generated images of casting holes and flaws.

[0017] Figure 6 This is an example of a patch image of a defect element for a casting hole and a defect element for a flaw.

[0018] Figure 7 are other examples of the initial shape E1 and the final shape E2 of an ellipse.

[0019] Figure 8 It is used Figure 7 An example of a generated image of an elliptical defect element facing a casting hole is shown.

[0020] Figure 9 This is another example of a patch image of a defect element facing a casting hole.

[0021] Figure 10 A diagram showing a method for generating an artificial defect image and an artificial defect patch image.

[0022] Figure 11 This is a flowchart showing the process of optimizing defect factors using statistical analysis.

[0023] Figure 12 This is a diagram showing the correlation between the major axis and brightness in a captured image of an actual defect.

[0024] Figure 13 This is an example of a defect element generated such that the brightness decreases as the defect becomes larger.

[0025] Figure 14 This is a flowchart showing the process of optimizing defect factors using an optimization technique. DETAILED DESCRIPTION

[0026] (Implementation Method 1)

[0027] Figure 1 This is a schematic diagram of the appearance inspection device 1 according to the first embodiment.

[0028] The visual inspection apparatus 1 of the first embodiment includes a camera (imaging unit) 2, a robot 3, and a computer 4. The camera 2 images the surface of an engine piston blank (raw material) or a finished engine piston (hereinafter referred to as a piston) 5, which is an inspection object. The robot 3 is a multi-jointed robot having a hand 3a that holds the piston 5.

[0029] The computer 4 is, for example, a personal computer and includes a CPU 6 and a memory (storage unit) 7. The CPU 6 includes an inspection unit 8 and a learning unit 9. The inspection unit 8 includes an acquisition unit 8a and a defect determination unit 8b. The acquisition unit 8a acquires an image captured by the camera 2. The defect determination unit 8b inspects the surface of the piston 5 for scratches or defects based on the image acquired by the acquisition unit 8a and the learning results stored in the memory 7 (products with scratches or defects are marked as NG, and products without scratches or defects are marked as OK).

[0030] The learning unit 9 includes a defect element generation unit 9a, an artificial defect image generation unit 9b, and a CNN learning unit 9c. The defect element generation unit 9a overlaps multiple predetermined shapes to generate a defect element image (a first image generation step). The artificial defect image generation unit 9b combines the defect element image with a surface image of a sample corresponding to the piston 5 to generate an artificial defect image that simulates a defective product sample image (a second image generation step). The CNN learning unit 9c performs machine learning using the multiple artificial defect images. Machine learning is learning using neural networks. In the first embodiment, deep learning based on a CNN (Convolutional Neural Network) model is employed.

[0031] The memory 7 stores a plurality of defect element images generated by the defect element generating unit 9 a , a plurality of artificial defect images generated by the artificial defect image generating unit 9 b , and learning results by the CNN learning unit 9 c .

[0032] Next, a method for generating a defect element image by the defect element generating unit 9 a will be described.

[0033] The defect element generating unit 9a generates a defect element image by overlapping a plurality of basic graphics having different sizes and brightness (second parameters). In the first embodiment, an ellipse is used as the basic graphic, and a defect element image is generated by changing the input parameters. Figure 2 This is an example of the initial shape E1 and the final shape E2 of an ellipse. Figure 2 The initial shape E1 of (a) is the major diameter d1 and the aspect ratio a1. Figure 2 The final shape E2 of (b) has a major diameter d2 and an aspect ratio a2.

[0034] For the defect element, one or more intermediate shapes are generated to interpolate (complement) between the initial shape E1 and the final shape E2, and the shapes are overlapped to generate the defect element. Figure 3 It is used Figure 2 The following is an example of generating an image of a defect element facing a pore, with input parameters including the major diameter (first parameter) d, brightness (second parameter) L, drawing path (third parameter) p(t), rotation angle θ, and the number of ellipses n. The drawing path p(t) is mathematically formulated using a straight line, and the number of ellipses n is set to 3. Figure 4 Patch image of defect element facing the pore as shown.

[0035] Figure 5 It is used Figure 2 The image of the defect element generated by the ellipse facing the casting hole and the scar is shown. The input parameters are the major diameter d, brightness L, drawing path p(t), random rotation angle dθ, random coordinate offset dP, and the number of ellipses n. The drawing path p(t) is mathematically formulated using a straight line, and the number of ellipses n is set to 4. By randomly rotating / translating the basic shape, it is possible to generate Figure 6 (a) Defective elements facing the casting hole, Figure 6 (b) A patch image of a defect element facing a flaw.

[0036] Figure 7 is another example of an initial ellipse shape E1 and a final shape E2, in which Figure 7 (a) The initial shape E1 has a major diameter d1 and an aspect ratio a1. Figure 7 The major diameter d2 and the aspect ratio a2 of the final shape E2 in (b) are substantially the same. Figure 8 It is used Figure 7 The input parameters for generating the defect element of the ellipse facing the casting hole are the major diameter d, brightness L, drawing path p(t), and the number of ellipses n. The drawing path p(t) is mathematically formulated using a quadratic function or a Bezier curve, and the number of ellipses n is set to 5. Figure 9 Patch image of defect element facing the casting hole as shown.

[0037] Figure 10 1 is a diagram showing a method for generating an artificial defect image by the artificial defect image generating unit 9b. Figure 10 (a) is the method for generating artificial defect images. Figure 10 (b) Method for generating patch images of artificial defects.

[0038] like Figure 10 As shown in (a), the entire piston 5 can be Figure 11As a background image, a plurality of patch images 10 of defect elements are synthesized to generate a plurality of artificial defect images 12. Figure 10 As shown in (b), a plurality of patch images 14 of artificial defects can be generated by synthesizing the patch image 13 of the piston 5 with the patch images 10 of the plurality of defect elements as a background image.

[0039] Next, the optimization of defect factors will be described.

[0040] The CNN learning unit 9c causes the neural network to learn defect images based on the multiple artificial defect images and multiple patch images of artificial defects generated by the artificial defect image generation unit 9b. Learning artificial defect images whose visual characteristics differ from those of actual defects reduces the accuracy of the CNN. Therefore, defect factors that can improve the CNN's recognition accuracy are needed—defect factors that are similar to actual defects but dissimilar to those of qualified products. In other words, optimizing the defect factors (their generation parameters) is necessary to achieve improved CNN accuracy. Two methods for optimizing defect factors are described below.

[0041] The first method is a method of generating defect elements based on statistical information of actual defects, thereby generating defect elements that are close to the actual defects. Figure 11 This is a flowchart showing the process of optimizing defect factors using statistical analysis.

[0042] In step S1, statistical analysis is performed on past samples of actual defects for each defect type (casting hole, flaw). Specifically, for example, the correlation between the major diameter and brightness is extracted.

[0043] In step S2, based on the statistical analysis results in step S1, defect elements are designed / generated for each defect type. Figure 12 In the case where there is a correlation that the larger the defect, the lower the brightness, Figure 13 As shown in FIG. 1 , the parameters are set so that the brightness becomes lower (darker) as the defect becomes larger, and the defect element is generated.

[0044] In step S3, an artificial defect image is generated based on the defect element generated in step S2.

[0045] In step S4 , the artificial defect image generated in step S3 is labeled for each defect type to generate a machine learning model (CNN model).

[0046] In step S5, actual defects are checked based on the learning results of the CNN model generated in step S4.

[0047] The second method is to optimize the design variables of the defect element using optimization techniques or the like. Figure 14 This is a flowchart showing the process of optimizing defect factors using an optimization technique.

[0048] In step S11 , the parameters of the defect element (longer diameter d, brightness L, drawing path p(t), rotation angle θ, number of ellipses n) are designed for each defect type, and the defect element is generated using the provisional parameters.

[0049] In step S12, an artificial defect image is generated based on the defect element generated in step S11.

[0050] In step S13 , the artificial defect image generated in step S12 is labeled for each defect type to generate a machine learning model (CNN model).

[0051] In step S14, actual defect past samples are checked.

[0052] In step S15, it is determined whether the termination condition is met. If "YES", the process proceeds to step S16, and if "NO", the process proceeds to step S17. Here, the termination condition determines whether the objective function (the accuracy rate of NG and OK) is above a threshold.

[0053] In step S16, actual defect inspection is performed.

[0054] In step S17, the parameters are modified to maximize the objective function.

[0055] Next, the effects of the first embodiment will be described.

[0056] Conventional visual inspection methods use collected defective product samples to perform CNN learning on defective images, determining whether or not the inspected object has defects. Therefore, collecting defective product samples required for learning in production lines with low defect rates is labor-intensive and time-consuming.

[0057] In contrast, in the appearance inspection method of embodiment 1, the surface of the piston 5 is inspected based on a captured image of the surface of the piston 5 and a learning result of machine learning using a plurality of artificial defect images generated by synthesizing a defect element image with a surface image of a sample equivalent to the piston 5, wherein the defect element image is made by overlapping a plurality of prescribed shapes.

[0058] In other words, using artificial defect images eliminates the need to collect and store defective sample images. Generating artificial defect images takes less time than collecting defective samples, thus reducing actual processing time and improving inspection efficiency. This benefit is particularly significant in production lines with low defect rates or those producing a wide variety of products, where collecting defective samples requires time.

[0059] Furthermore, in the appearance inspection method of the first embodiment, since a defect element image is generated by superimposing a plurality of predetermined shapes, an artificial defect image can be generated more simply than a method of generating a defect element image using a 3D model.

[0060] The defect element image of embodiment 1 is generated by overlapping a plurality of shapes that are different from each other in at least one of size or brightness. By changing the parameters of the shape or brightness and overlapping, a defect element image that is closer to the actual defect can be produced. In the previous method of generating defect element images using a single graphic, only specific shapes such as ellipses and circles and monotonous changes in brightness can be expressed. On the other hand, in the appearance inspection method of embodiment 1, by changing the overlapping shapes and brightness, defects of various shapes and appearances can be produced, and artificial defect images that are more consistent with actual defects can be generated. In particular, by having multiple parameters and changing them, more complex artificial defect images can be easily generated. Furthermore, the defect changes that are desired to be detected can be parametrically managed / rationalized.

[0061] Furthermore, the basic overlapping shapes in the defect element image are ellipses. By varying the dimensional parameters of each ellipse, such as the major and minor diameters, and the rotation angle, various defect shapes can be represented, including circular or elliptical casting holes and linear flaws. Furthermore, shapes with rounded corners can be easily represented, similar to actual defects. For example, curved linear defects can be represented by reducing the ellipse size to its maximum possible size.

[0062] A defect element image is generated by overlaying an initial shape, a final shape, and one or more intermediate shapes. If only the initial and final shapes were overlaid to generate a defect element image, the size and brightness would change dramatically, making it difficult to create an artificial defect image that closely resembles the actual defect. Therefore, by providing intermediate shapes between the initial and final shapes, these dramatic changes in size and brightness can be suppressed, allowing the generation of an artificial defect image that more closely resembles the actual defect.

[0063] The defect element image has a drawing path connecting the initial shape and the final shape as a parameter, and an intermediate shape is generated at a position corresponding to the drawing path. This makes it possible to generate a more complex artificial defect image such as an arc-shaped defect.

[0064] The parameters of the defect element image are set based on statistical data obtained from past samples of actual defects. By generating defect elements based on statistical information about actual defects, various artificial defect elements can be generated with shapes that are closer to the actual defects.

[0065] (Other Implementation Methods)

[0066] While the embodiments for implementing the present invention have been described above, the specific configuration of the present invention is not limited to the configuration of the embodiments, and design changes that do not depart from the spirit of the invention are also encompassed by the present invention.

[0067] For example, the basic graphics to be superimposed when generating the defect element image are not limited to ellipses, but may also be circles or polygons.

[0068] The object to be inspected is not limited to the piston.

[0069] The learning results are not limited to neural networks or deep learning, as long as they are machine learning.

[0070] Furthermore, the present invention is not limited to the above-described embodiments and includes various variations. For example, the above-described embodiments are described in detail to facilitate understanding of the present invention and are not necessarily limited to having all of the structures described. In addition, a portion of the structure of a certain embodiment can be replaced with the structure of another embodiment, and a portion of the structure of a certain embodiment can be added to the structure of another embodiment. Furthermore, for a portion of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0071] This application claims the benefit of priority based on Japanese Patent Application No. 2023-022389 filed on February 16, 2023. The entire disclosure of Japanese Patent Application No. 2023-022389 filed on February 16, 2023 including the specification, claims, drawings, and abstract is incorporated herein by reference in its entirety.

[0072] Description of Reference Numerals

[0073] 1 Appearance inspection device; 2 Camera (photographing unit); 5 Piston (inspected object); 7 Memory (storage unit); 8 Inspection unit.

Claims

1. A visual inspection device for inspecting the surface of an object to be inspected, comprising: a photographing unit for photographing the surface of the object to be inspected; a storage unit storing learning results obtained by performing machine learning using a plurality of artificial defect images, the plurality of artificial defect images being generated by synthesizing defect element images generated by overlapping a plurality of predetermined shapes with a surface image of a sample corresponding to the object to be inspected; as well as The inspection unit inspects the surface of the inspection object based on the image captured by the imaging unit and the learning result.

2. The appearance inspection device according to claim 1, wherein: Each of the plurality of predetermined shapes has a characteristic value of the predetermined shape as a first parameter, The defect element image is generated by overlapping a plurality of the predetermined shapes in which the first parameters are respectively changed.

3. The appearance inspection device according to claim 2, wherein: The first parameter is the size or brightness of the prescribed shape.

4. The appearance inspection device according to claim 2, wherein: The predetermined shape is an ellipse, and the first parameter is at least one of a major axis and a minor axis of the ellipse.

5. The appearance inspection device according to claim 2, wherein: Each of the plurality of predetermined shapes has a characteristic value different from the first parameter as a second parameter, The defect element image is generated by overlapping a plurality of the predetermined shapes in which the first parameter and the second parameter are changed.

6. The appearance inspection device according to claim 5, wherein: The first parameter is the size of the prescribed shape, The second parameter is the brightness of the prescribed shape.

7. The appearance inspection device according to claim 5, wherein: The prescribed shape is an ellipse, The first parameter is at least one of the major axis and the minor axis of the ellipse, The second parameter is the brightness of the prescribed shape.

8. The appearance inspection device according to claim 2, wherein: The plurality of prescribed shapes include: Initial and final shapes; and an intermediate shape, generated by changing the first parameter from the initial shape or the final shape, and performing interpolation between the initial shape and the final shape; The defect element image is generated by overlapping the initial shape, the final shape, and the intermediate shape.

9. The appearance inspection device according to claim 8, wherein: A path connecting the initial shape and the final shape can be specified as a third parameter, and the intermediate shape is generated at a position corresponding to the third parameter.

10. The appearance inspection device according to claim 5, wherein: The first parameter and the second parameter are set based on statistical data obtained from samples of actual defects.

11. An image generation method for generating an artificial defect image for inspecting a surface of an object to be inspected, comprising: A first image generation step of overlapping a plurality of predetermined shapes to generate a defect element image; and The second image generation step is to synthesize the defect element image with a surface image of a sample corresponding to the inspection object to generate a plurality of artificial defect images.

Images