Appearance inspection method and model generation device

Abstract

Provided is an appearance inspection method for inspecting an appearance of an inspection object using a trained model obtained through supervised learning by setting a normal appearance image, which is an appearance image of an inspection object without any abnormality, as a normal image and a pseudo defective image, which is created by incorporating a defective image feature into a normal appearance image, as an abnormal image, the appearance inspection method including: a defective image feature incorporation step of generating a pseudo defective image using a normal appearance image and a defective image feature; a first generation step of generating a first trained model by performing first machine learning using the pseudo defective image generated by the defective image feature incorporation step as an abnormal image; a first appearance inspection step of inputting a first input image, which is an appearance image of an inspection object, into the first trained model generated by the first generation step to inspect the appearance of the inspection object; an over-detection specification step of specifying over-detection related to the inspection object on the basis of a result of the first appearance inspection step and specifying a causal image feature being a pseudo defective image that has caused over-detection; a pseudo defect adjustment step of adjusting, for the causal image feature, a parameter related to incorporation of the defective image feature; and a second generation step of generating a second trained model by performing second machine learning using the pseudo defective image adjusted by the pseudo defect adjustment step as an abnormal image.

Description

Visual inspection method and model generation device

[0001] The present invention relates to a visual inspection method and a model generation device.

[0002] When applying machine learning to visual inspection, it is necessary to prepare a large number of images of both defective and normal products to be inspected. Furthermore, to improve the accuracy of visual inspection, the number of images of normal products and the number of images of defective products must be approximately the same. However, the number of defective products is generally far fewer than the number of normal products. This poses a problem of a shortage of images of defective products compared to images of normal products. Patent Literature 1 discloses a training data generation device including: a defective part data storage unit that stores defective part data, which is image data of defective part images; a good product data storage unit that stores good product data, which is image data of good product images; a training data generation unit that generates training data, which is image data of training images obtained by combining the good product images with the defective part images, based on the defective part data, the good product data, and generation parameters; a training data storage unit that stores the training data; and a generation parameter setting unit that sets the generation parameters.

[0003] Japanese Patent Application Publication No. 2020-027424

[0004] The invention described in Patent Document 1 cannot prevent overdetection.

[0005] A visual inspection method according to a first aspect of the present invention is a visual inspection method for performing a visual inspection of an object to be inspected using a learning model obtained by supervised learning, the method comprising: a defect image element synthesis step of generating the pseudo-defect image using the normal appearance image and the defect image element; a first generation step of performing first machine learning using the pseudo-defect image generated by the defect image element synthesis step as the abnormality-containing image to generate a first learning model; and the first learning model generated by inputting a first input image, which is an appearance image of the inspection object, into the first learning model generated by the above method, thereby performing an appearance inspection of the inspection object; an overdetection identification step, which identifies an overdetection of the inspection object based on the result of the first appearance inspection step and identifies a causal image element, which is the pseudo-defect image that caused the overdetection; a pseudo-defect adjustment step, which adjusts parameters related to the synthesis of the defect image element for the causal image element; and a second generation step, which performs second machine learning using the pseudo-defect image adjusted in the pseudo-defect adjustment step as the image with an abnormality, thereby generating a second learning model.A model generating device according to a second aspect of the present invention is a model generating device that sets a normal appearance image, which is an appearance image of an object to be inspected without an abnormality, as an abnormality-free image, sets a pseudo-defect image obtained by combining the normal appearance image with a defect image element as an abnormality-containing image, and generates a learning model through supervised learning, the model generating device comprising: a defect image element synthesis program that generates the pseudo-defect image using the normal appearance image and the defect image element; a first generation program that performs first machine learning using the pseudo-defect image generated by the defect image element synthesis program as the abnormality-containing image to generate a first learning model; and The system comprises a first visual inspection program that inputs a first input image, which is an appearance image of the inspection object, into a learning model and performs a visual inspection of the inspection object; an overdetection identification program that identifies overdetection of the inspection object based on the results of the first visual inspection program and identifies causal image elements, which are the pseudo-defect images that caused the overdetection; a pseudo-defect adjustment program that adjusts parameters related to the synthesis of the defect image elements for the causal image elements; and a second generation program that performs second machine learning using the pseudo-defect image adjusted by the pseudo-defect adjustment program as the image with an abnormality to generate a second learning model.

[0006] According to the present invention, overdetection can be suppressed.

[0007] FIG. 1 is a diagram illustrating various images in this embodiment; FIG. 2 is an external view of a piston, which is an object to be inspected; FIG. 3 is a diagram illustrating an example of a pseudo defect image; FIG. 4 is a diagram illustrating the generation of a pseudo defect image by a defect image element synthesis program; FIG. 5 is a diagram illustrating the generation of a segmented pseudo defect image by a defect image element synthesis program;

[0008] -First Embodiment- A first embodiment of a model generating device will be described below with reference to FIGS.

[0009] 1 is a configuration diagram of a model generation device 1. The model generation device 1 includes a CPU 41, which is a central processing unit, a ROM 42, which is a read-only storage device, a RAM 43, which is a read / write storage device, an image library 16, a model library 17, and an imaging unit 18. The model generation device 1 also includes an input / output device 44, which is a user interface, and a communication device 45. The CPU 41 loads and executes programs stored in the ROM 42 into the RAM 43. A defect image element synthesis program 11, a learning model generation program 12, a visual inspection program 15, an overdetection identification program 13, and a defect adjustment program 14 are loaded into the RAM 43.

[0010] The model generating device 1 may be realized by a field programmable gate array (FPGA), which is a rewritable logic circuit, or an application specific integrated circuit (ASIC), which is an application specific integrated circuit, instead of the combination of the CPU 41, ROM 42, and RAM 43. The model generating device 1 may also be realized by a combination of different configurations, for example, a combination of the CPU 41, ROM 42, RAM 43, and an FPGA, instead of the combination of the CPU 41, ROM 42, and RAM 43. Before describing the configuration shown in FIG. 1 , various images in this embodiment will be described.

[0011] FIG. 2 is a diagram illustrating various images in this embodiment. A normal appearance image 21 is obtained by the photographing unit 18. A pseudo-defect image 23 is generated using the normal appearance image 21 and the defect image element 22. As will be described in detail later, a plurality of pseudo-defect images 23 are generated using a set of the normal appearance image 21 and the defect image element 22. The pseudo-defect image 23 is divided into a plurality of regions, and each divided image is called a subdivided pseudo-defect image 24. For convenience of illustration, FIG. 2 illustrates one normal appearance image 21, one defect image element 22, one pseudo-defect image 23, and one subdivided pseudo-defect image 24, but the numbers are different as described above. That is, a plurality of pseudo-defect images 23 are generated using a set of the normal appearance image 21 and the defect image element 22, and a plurality of subdivided pseudo-defect images 24 are generated for each pseudo-defect image 23. Returning to FIG. 1 , the description will continue.

[0012] The model library 17 stores a first learning model 171 and a second learning model 172. Both the first learning model 171 and the second learning model 172 are generated by the learning model generation program 12. The photographing unit 18 photographs the inspection object and outputs a photographed image, which is an image obtained by photographing. This photographed image becomes a normal appearance image 21 and an input image to the appearance inspection program 15. The photographed image by the photographing unit 18 may be temporarily stored in the image library 16. In this case, a photographed image of an inspection object that has been confirmed in advance to be free of defects may be stored as the normal appearance image 21.

[0013] The defect image element synthesis program 11 synthesizes a defect image element 22 with a normal appearance image 21, which is an appearance image of an object to be inspected without defects, to generate a pseudo defect image 23 and a subdivided pseudo defect image 24. The learning model generation program 12 performs machine learning using the subdivided pseudo defect image 24 to generate a first learning model 171 and a second learning model 172. The learning model generated by the defect image element synthesis program 11 using the subdivided pseudo defect image 24 without adjustment by the defect adjustment program 14 is the first learning model 171. The learning model generated by the defect image element synthesis program 11 using the subdivided pseudo defect image 24 after adjustment by the defect adjustment program 14 is the second learning model 172. For learning by the learning model generation program 12, machine learning such as a deep neural network or a support vector machine is preferably used, using a learning image as input.

[0014] The visual inspection program 15 inputs the captured images output by the imaging unit 18 into a learning model to perform a visual inspection of the object to be inspected. Hereinafter, the visual inspection using the first learning model 171 will be referred to as the "first visual inspection," and the visual inspection using the second learning model 172 will be referred to as the "second visual inspection." The overdetection identification program 13 identifies overdetection based on the output results of the first visual inspection. The method for identifying overdetection will be described later. The defect adjustment program 14 adjusts parameters related to the synthesis of defect image elements 22 for specific elements (hereinafter referred to as "causal image elements") of the pseudo defect image 23 that caused the overdetection, identified by the overdetection identification program 13. Specifically, the defect adjustment program 14 adjusts parameters such as the brightness, size, and angle of the causal image element so as to reduce the similarity of the causal image element to the overdetected portion 402. The causal image element is one of the defect image elements 22.

[0015] The first learning model 171 and the second learning model 172 receive a captured image as input and output the inspection judgment result and coordinates where defects are detected. The normal appearance image 21 is read into the learning model generation program 12 along with information indicating that there is no abnormality. The subdivided pseudo-defect image 24, along with information indicating that there is an abnormality, and the coordinates of the defect image elements 22 are read into the learning model generation program 12. The learning model generation program 12 learns the parameters of the first learning model 171 and the second learning model 172 so that when the normal appearance image 21 is input, a message indicating that there is no abnormality is output. The learning model generation program 12 learns the parameters of the first learning model 171 and the second learning model 172 so that when the subdivided pseudo-defect image 24 is input, a message indicating that there is an abnormality and the coordinates of the defect image elements 22 are output.

[0016] The processing executed by the defect image element synthesis program 11 is also called a "defect image element synthesis step." The generation of the first learning model 171 by the learning model generation program 12 is also called a "first generation step," and the generation of the second learning model 172 is also called a "second generation step." The processing executed by the appearance inspection program 15 is also called a "appearance inspection step." The processing executed by the overdetection identification program 13 is also called an "overdetection identification step." The processing executed by the defect adjustment program 14 is also called a "pseudo defect adjustment step." The learning model generation program 12 is divided into a first generation program that generates the first learning model 171 and a second generation program that generates the second learning model 172.

[0017] 3 is an external view of a piston 201 for an automobile engine, which is the object of inspection in this embodiment. The piston 201 is a generally cylindrical cast part. A crown surface 202, which is the upper side of the piston 201 in the figure, shows a cast surface 203 and a machined surface 204 obtained by machining the cast surface 203.

[0018] FIG. 4 shows an example of a pseudo-defect image 23. The pseudo-defect image 23 in this figure is an image obtained by combining 12 defect image elements 22 with a normal appearance image 21 obtained by photographing an abnormality-free crown surface 202 from directly above. The defect image elements 22 may be images obtained by extracting defective portions from a defective product image, or may be artificial images created to resemble defects. As shown in the enlarged view at the bottom of FIG. 4 , in the following description, the dimension of the defect image elements 22 is referred to as the element dimension Dl, and the spacing between the defect image elements 22 arranged in a grid pattern is referred to as the defect spacing Dp. For simplicity's sake, in this embodiment, the vertical direction in the figure also has the same dimension and spacing. However, the vertical direction and the horizontal direction in the figure do not have to have the same dimension and spacing. In other words, although the defect image elements 22 are described as being square in this embodiment, they may also be rectangular. Here, "abnormal" means that there are no defects during inspection, and includes the presence of minute irregularities that are not recognized as defects.

[0019] FIG. 5 is a diagram showing the generation of a pseudo defect image 23 by the defect image element synthesis program 11. The defect image element synthesis program 11 first defines a coordinate system with a reference point 20, which is an arbitrary point in the normal appearance image 21, as the origin. The reference point 20 and the coordinate system are common to the normal appearance image 21 and all the pseudo defect images 23. In this embodiment, changing the starting position of the synthesis position of the defect image elements 22 is expressed by the concept of "phase." The defect image element synthesis program 11 generates a pseudo defect image 23 with a phase (0, 0) as the first pseudo defect image 23. In the pseudo defect image 23 with a phase (0, 0), the first defect image element 22 is arranged at the origin. The defect image element synthesis program 11 then arranges defect image elements 22 one after another at positions spaced apart by a defect arrangement interval Dp in the X-axis and Y-axis directions from this first defect image element 22 as the starting point.

[0020] In the pseudo defect image 23 of phase (1, 0), the first defect image element 22 is arranged at a position shifted by the phase interval Pp from the reference point 20 only in the X direction. The arrangement of the second and subsequent defect image elements 22 is the same as above, and therefore will not be described again below. In the pseudo defect image 23 of phase (0, 1), the first defect image element 22 is arranged at a position shifted by the phase interval Pp from the reference point 20 only in the Y direction. In the pseudo defect image 23 of phase (1, 1), the first defect image element 22 is arranged at a position shifted by the phase interval Pp from the reference point 20 in both the X and Y directions. Generalizing, in the pseudo defect image 23 of phase (n, m), the first defect image element 22 is arranged at a position shifted by n times the phase interval Pp from the reference point 20 in the X direction and m times the phase interval Pp in the Y direction from the reference point 20.

[0021] FIG. 6 is a diagram showing the generation of the subdivided pseudo defect image 24 by the defect image element synthesis program 11. As described above, the defect image element synthesis program 11 generates the subdivided pseudo defect image 24 by subdividing the pseudo defect image 23. The defect image element synthesis program 11 always uses the reference point 20 as a reference and generates each of the regions separated at intervals of a predetermined cutout dimension Ls as the subdivided pseudo defect image 24. This grid is, for example, parallel to the X-axis and Y-axis. However, the defect image element synthesis program 11 does not have to actually create the subdivided pseudo defect image 24. In other words, the defect image element synthesis program 11 may treat each region of the pseudo defect image 23 as the subdivided pseudo defect image 24.

[0022] The element dimension Dl, cutout dimension Ls, phase interval Pp, and defect arrangement interval Dp described above satisfy the following relational expressions.

[0023] Dp>Ls+Dl (Formula 1)

[0024] Dp<Ls+Dl+Pp (Equation 2)

[0025] Since the equality of the formulas 1 and 2 is established, only one defect image element 22 exists in each subdivided pseudo defect image 24, and the defect image element 22 exists at the same position in each subdivided pseudo defect image 24.

[0026] 7 is a flowchart showing an outline of the processing of the model generating device 1. In the model generating device 1, first, in step S101, the defect image element synthesis program 11 creates a pseudo defect image 23 and a subdivided pseudo defect image 24. Next, the learning model generation program 12 generates a first learning model 171 using the subdivided pseudo defect image 24. In the following step S103, the appearance inspection program 15 performs a first appearance inspection using the first learning model 171. In the following step S104, the overdetection identification program 13 identifies overdetection in the first appearance inspection.

[0027] In the following step S105, the defect adjustment program 14 adjusts the defective pixel elements, and the defect image element synthesis program 11 newly creates the subdivided pseudo defect image 24. In the following step S106, the learning model generation program 12 generates a second learning model 172 using the subdivided pseudo defect image 24 generated in step S105. In the following step S107, the appearance inspection program 15 performs a second appearance inspection using the second learning model 172.

[0028] FIG. 8 is a diagram showing an example of overdetection and causative image elements. The left side of FIG. 8 shows an overdetection image 401 in which overdetection has occurred, and the right side of FIG. 8 shows a pseudo defect image 23-1. In the example shown in FIG. 8, an overdetection point 402 is present approximately in the center, to the left, of the overdetection image 401. It can be confirmed by any of the following methods that the overdetection point 402 is an overdetection, in other words, that the overdetection point 402 is not an abnormality that should actually be detected by the visual inspection program 15.

[0029] The first method is a method in which all images input to the first learning model 171 are normal images. For example, a captured image obtained by photographing an inspection object that has passed inspection is input to the first learning model 171, or a captured image that has been previously confirmed by a human to be free of abnormalities is input to the first learning model 171. The second method is a method in which the image is input to another visual inspection method unrelated to the model library 17 for confirmation. In other words, a captured image in which the first learning model 171 has detected a defect is input to another visual inspection method unrelated to the model library 17, and if this visual inspection method does not detect an abnormality, it is determined to be an overdetection.

[0030] When the overdetection identification program 13 detects overdetection, it collates the coordinates of the overdetected point 402 on the pseudo defect image 23-1 to determine the corresponding coordinates 403. Then, it identifies the defect image element 22 closest to the corresponding coordinates 403 as the causative image element 22C that is the cause of the overdetection. As described above, since a plurality of pseudo defect images 23 are generated, the causative image element 22C is identified in each of the plurality of pseudo defect images 23.

[0031] FIG. 9 is a diagram showing a first example of parameter adjustment by the defect adjustment program 14. In this example, the defect adjustment program 14 adjusts the luminance of the cause image element 22C. In FIG. 9, the horizontal axis represents the luminance of the background of the defect image element 22 (hereinafter also referred to as "background luminance"), and the vertical axis represents the luminance of the defect image element 22 (hereinafter also referred to as "defect luminance"). For example, the background luminance is the luminance of the brightest point in the defect image element 22, and the defect luminance is the luminance of the darkest point in the defect image element 22. Also, for example, the background luminance is the average value of the luminance of the defect image element 22 surrounding the defect image element 22, and the defect luminance is the average value of the luminance of the defect image element 22.

[0032] In Figure 9, for each defect image element 22, which is a defect detected in the past, points corresponding to its background luminance and defect luminance are plotted as white circles. In each plot, the defect luminance generally monotonically increases relative to the background luminance, and the regression line 504 is shown as a solid line. The dotted lines above and below the regression line 504 indicate a possible defect range 505, which is a range in which a defect is possible. In Figure 9, the causative image element 22C identified by the overdetection identification program 13 is also plotted in a similar manner. However, since the overdetected portion 402 is an overdetection, the relationship between the background luminance of the portion erroneously determined to be a defect and the luminance of the portion erroneously determined to be a defect is shown.

[0033] The defect adjustment program 14 then adjusts the parameters of the cause image element 22C so as to reduce the similarity with the overdetected portion 402. In Fig. 9, the defect brightness of the cause image element 22C is increased so as to increase the distance between the cause image element 22C and the overdetected portion 402. However, extreme changes are not preferable, and adjustment is made within a range in which the defect brightness after adjustment falls within the defect possibility range 505. Note that it is desirable to adjust the cause image element 22C not only so that the distance between the overdetected portion 402 and the cause image element 22C is increased, but also so that the difference in the distance to the regression line 504 from the overdetected portion 402 is increased.

[0034] 10 is a diagram showing a second example of parameter adjustment by the defect adjustment program 14. In the histogram shown in FIG. 10, the horizontal axis represents defect size, and the vertical axis represents the occurrence frequency of each defect size. The histogram plots each defect detected in the past based on its size and occurrence frequency. The dotted lines on the left and right of the histogram indicate a defect possibility range 1104, which is a range in which a defect is likely to exist, based on the defects detected so far.

[0035] The defect adjustment program 14 plots the overdetected portion 402 identified by the overdetection identification program 13 on a histogram based on its size. The defect adjustment program 14 also plots the cause image element 22C identified as the cause of the overdetection in the same way. The defect adjustment program 14 adjusts the parameters of the cause image element 22C so that the similarity between the overdetected portion 402 and the defect adjustment program 14 reduces, resulting in an adjusted cause image element 22D. That is, in the example of FIG. 10 , the cause image element 22C is enlarged so as to move away from the plot position of the overdetected portion 402, resulting in an adjusted cause image element 22D. The amount of this adjustment is set to the range of the defect possibility range 1104.

[0036] The first embodiment described above provides the following advantageous effects. (1) The model generation device 1 defines a normal appearance image 21, which is an appearance image of an object without an abnormality, as an abnormality-free image, defines a pseudo-defect image 23, which is obtained by combining the normal appearance image 21 with a defect image element 22, as an abnormality-containing image, and performs an appearance inspection of the object using a learning model obtained by supervised learning. The defect image element synthesis program 11 executes a defect image element synthesis step of generating the pseudo-defect image 23 using the normal appearance image 21 and the defect image element 22. The learning model generation program 12 executes a first generation step of performing first machine learning using the pseudo-defect image 23 generated by the defect image element synthesis step as an abnormality-containing image to generate a first learning model 171. The appearance inspection program 15 executes a first appearance inspection step of inputting a first input image, which is an appearance image of the object, into the first learning model 171 generated by the first generation step and performing an appearance inspection of the object. The overdetection identification program 13 executes an overdetection identification step in which an overdetection of the inspection object is identified based on the results of the first visual inspection step, and a cause image element 22C, which is a pseudo defect image 23 that caused the overdetection, is identified. The defect adjustment program 14 executes a pseudo defect adjustment step in which parameters related to the synthesis of the defect image element 22 are adjusted for the cause image element 22C. The learning model generation program 12 executes a second generation step in which a second machine learning is performed using the pseudo defect image 23 adjusted by the defect adjustment program 14 as an image with an abnormality, to generate a second learning model 172. Therefore, the second learning model 172 is generated after correcting the learning data so that the overdetection that occurred in the first learning model 171 does not occur. Therefore, overdetection can be suppressed by using the second learning model 172.

[0037] (2) The defect adjustment program 14 adjusts at least one parameter of the brightness, size, and angle of the defect image element 22 so as to reduce the similarity of the cause image element 22C to the overdetected portion 402. Therefore, the learning data can be corrected in terms of brightness, size, and angle to prevent overdetection.

[0038] (3) In the first generation step and the second generation step, the first learning model 171 and the second learning model 172 are generated using subdivided pseudo defect images 24 obtained by dividing the pseudo defect image 23 into a grid pattern. In the defect image element synthesis step, the defect image elements 22 are arranged in a grid pattern so that only one defect image element 22 exists in each subdivided pseudo defect image 24 and the defect image elements 22 exist in the same positions in the subdivided pseudo defect image 24, and are synthesized with the normal appearance image 21. Therefore, efficient learning can be achieved by preparing learning images that conform to the specifications of the learning model generation program 12.

[0039] 9 and 10 , the defect adjustment program 14 adjusted the brightness or size of the defect image element 22 so as to reduce the similarity of the cause image element 22C to the overdetected portion 402. However, the defect adjustment program 14 may adjust at least one parameter of the brightness, size, and angle of the defect image element 22 so as to reduce the similarity of the cause image element 22C to the overdetected portion 402. For example, the defect adjustment program 14 may adjust the angle of the cause image element 22C, or may adjust both the brightness and size of the cause image element 22C.

[0040] - Second embodiment - A second embodiment of a model generation device will be described with reference to Fig. 11. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and differences will be mainly described. Points that are not specifically described are the same as those in the first embodiment. This embodiment differs from the first embodiment mainly in terms of handling rotation of captured images.

[0041] FIG. 11 is a diagram illustrating the identification of the causative image element 22C in the second embodiment. The processing described below is an additional processing executed by the overdetection identification program 13. A captured image when the object to be inspected is not rotated is shown in the upper part of FIG. 11. The overdetection identification program 13 detects rotation based on the left end 708 and right end 709 of the machined surface 204 of this captured image 704. A captured image 701 shown in the lower left of FIG. 11 includes a detection point 702 detected by the overdetection identification program 13. The captured image 701 was captured with the object to be inspected slightly rotated. The overdetection identification program 13 calculates an orientation error 706 from the deviation amounts of the left end 711 and right end 712 in the captured image 701. The overdetection identification program 13 then uses this orientation error 706 to accurately calculate corresponding coordinates 713 in the pseudo defect image 23-1. The processing after calculating the corresponding coordinates 713 is the same as in the first embodiment, and therefore will not be described again.

[0042] According to the second embodiment described above, the following advantageous effects can be obtained. (4) The pseudo defect image 23 is captured with the object to be inspected in a reference orientation. In the overdetection identification step, an orientation error, which is a change from the reference orientation of the object to be inspected in the first input image, is detected, and the first input image is transformed based on the orientation error to identify the causative image element 22C. In other words, when the positioning accuracy of the object to be inspected is poor, the captured image 701 is transformed, i.e., rotated, based on the calculated orientation error to identify the causative image element 22C. Therefore, the overdetection identification program 13 can accurately identify the causative image element 22C.

[0043] -Third embodiment- A third embodiment of a model generation device will be described with reference to Figures 12 and 13. In the following description, the same components as in the first embodiment are designated by the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as in the first embodiment. This embodiment differs from the first embodiment mainly in that normal appearance images 21 are captured from multiple angles and defect image elements 22 are deformed according to the angles.

[0044] 12 is a diagram showing the photographing unit 18 in the third embodiment. The photographing unit 18 includes a jig 802 and an imaging device 803. The jig 802 is a jig having two rotation axes that can change the posture of the inspection object 801. The photographing unit 18 photographs normal appearance images 21, which are appearance images of a defect-free inspection object, at a plurality of postures by cooperative operation with the jig 802. That is, in this embodiment, a plurality of appearance images of a single defect-free inspection object are photographed by changing the relative position and relative angle between the inspection object 801 and the imaging device 803.

[0045] FIG. 13 is a diagram showing the processing of the defect image element synthesis program 11 in the third embodiment. In this embodiment, the defect image element synthesis program 11 adjusts the position and size of the defect image element 22 on the machined surface 204 so that they do not change even if the position or orientation changes. That is, the defect image element synthesis program 11 synthesizes the defect image element 22 for each normal appearance image 21, taking into account the relative position and relative angle between the inspection object 801 and the imaging device 803. The pseudo-defect image 23-2, which is based on an image captured from directly above, has a shape that is squashed vertically, while the pseudo-defect image 23-3, which is based on an image captured from an oblique angle, has a shape that is squashed vertically. Even in this case, the defect image element 22 is synthesized at a position that has been subjected to coordinate transformation based on the relative angle, so there is no contradiction between the two images. In other words, the defect is expressed as if it actually exists on the machined surface 204.

[0046] For example, if the inspection object 801 is disposed at an angle with respect to the imaging device 803, the defect image element synthesis program 11 distorts the defect image element 22 in accordance with the angle and then synthesizes it into the normal appearance image 21. The relative position and relative angle between the inspection object 801 and the imaging device 803 may be calculated by the defect image element synthesis program 11 through image processing. Furthermore, when the normal appearance image 21 is captured, a recording device (not shown) may record the posture of the jig 802 and link it to the normal appearance image 21.

[0047] The defect image element synthesis program 11 may use three-dimensional data of the piston 201, which is the object to be inspected. For example, three-dimensional data showing the three-dimensional shape of the machined surface 204 may be obtained from a design drawing. The defect image elements 22 may be attached to this three-dimensional data, and two-dimensional images may be written from multiple angles, or so-called rendering may be performed.

[0048] According to the third embodiment described above, the following advantageous effects can be obtained. (5) The normal appearance images 21 are a plurality of images obtained by photographing the inspection object from a plurality of angles. In the defect image element synthesis step, the defect image elements 22 are synthesized at positions obtained by performing coordinate transformation on each of the normal appearance images 21 based on the angle. Therefore, since the defect image elements 22 are synthesized at positions corresponding to each other in a plurality of orientations, the causative image elements 22C that cause overdetection are always present at positions corresponding to each other in a plurality of orientations, and therefore, by identifying these elements 22C collectively and adjusting them individually, efficient adjustment can be achieved.

[0049] In the above-described embodiments and modifications, the functional block configurations are merely examples. Some functional configurations shown as separate functional blocks may be integrated, or a configuration shown in a single functional block diagram may be divided into two or more functions. Furthermore, some of the functions of each functional block may be provided by other functional blocks.

[0050] In the above-described embodiments and modifications, the program is stored in ROM 42, but the program may be stored in a non-volatile storage device. Furthermore, the model generating device 1 may be provided with an input / output interface (not shown), and the program may be loaded from another device as needed via the input / output interface and a medium available to the model generating device 1. Here, the medium refers to, for example, a storage medium detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, or optical network, or a carrier wave or digital signal propagating through the network. Furthermore, some or all of the functions realized by the program may be realized by a hardware circuit or FPGA.

[0051] The above-described embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention.

[0052] 1: Model generation device 11: Defect image element synthesis program 12: Learning model generation program 13: Overdetection identification program 14: Defect adjustment program 15: Appearance inspection program 21: Normal appearance image 22: Defect image element 22C: Cause image element 22D: Adjusted cause image element 23: Pseudo defect image 24: Segmented pseudo defect image 171: First learning model 172: Second learning model

Claims

1. A visual inspection method for performing visual inspection of an object to be inspected using a learning model obtained by supervised learning, comprising: a defect image element synthesis step of generating the pseudo-defect image using the normal appearance image and the defect image element; a first generation step of performing first machine learning using the pseudo-defect image generated by the defect image element synthesis step as the image with an abnormality to generate a first learning model; a first visual inspection step of inputting a first input image, which is an appearance image of the object to be inspected, into the first learning model generated by the first generation step to perform visual inspection of the object to be inspected; an overdetection identification step of identifying an overdetection related to the object to be inspected based on the result of the first visual inspection step, and identifying a cause image element, which is the pseudo-defect image that caused the overdetection; and a pseudo-defect adjustment step of adjusting a parameter related to the synthesis of the defect image element for the cause image element. A second generation step of performing second machine learning using the pseudo defect image adjusted in the pseudo defect adjustment step as the image with an abnormality to generate a second learning model.

2. An appearance inspection method as described in claim 1, wherein the pseudo-defect adjustment step adjusts at least one parameter of the brightness, size, and angle of the defect image element so as to reduce the similarity of the cause image element to the overdetected location.

3. An appearance inspection method as described in claim 1, wherein in the first generation step and the second generation step, the first learning model and the second learning model are generated using subdivided pseudo-defect images obtained by dividing the pseudo-defect image into a grid pattern, and in the defect image element synthesis step, the defect image elements are arranged in a grid pattern so that only one defect image element exists in each of the subdivided pseudo-defect images and the defect image elements exist in the same positions in the subdivided pseudo-defect images, and are synthesized into the normal appearance image.

4. A visual inspection method as described in claim 1, wherein the pseudo defect image is captured with the object being inspected in a reference posture, and the overdetection identification step detects a posture error, which is a change from the reference posture of the object being inspected in the first input image, and transforms the first input image based on the posture error to identify the causative image element.

5. An appearance inspection method as described in claim 1, wherein the normal appearance images are a plurality of images obtained by photographing the object to be inspected from a plurality of angles, and in the defect image element synthesis step, the defect image elements are synthesized at positions obtained by applying coordinate transformation to each of the normal appearance images based on the angles.

6. A model generation device which sets a normal appearance image, which is an appearance image of an object to be inspected without an abnormality, as an abnormality-free image, sets a pseudo-defect image obtained by synthesizing a defect image element with the normal appearance image as an abnormality-containing image, and generates a learning model through supervised learning, comprising: a defect image element synthesis program which generates the pseudo-defect image using the normal appearance image and the defect image element; a first generation program which performs a first machine learning to generate a first learning model using the pseudo-defect image generated by the defect image element synthesis program as the abnormality-containing image; a first appearance inspection program which inputs a first input image, which is an appearance image of the object to be inspected, into the first learning model generated by the first generation program, and inspects the object; an overdetection identification program which identifies an overdetection related to the object to be inspected based on the results of the first appearance inspection program, and identifies a cause image element which is the pseudo-defect image that caused the overdetection; and a pseudo-defect adjustment program which adjusts a parameter related to the synthesis of the defect image element for the cause image element. a second generation program that performs second machine learning using the pseudo defect image adjusted by the pseudo defect adjustment program as the image with an abnormality to generate a second learning model.

Images