Visual inspection method and model generation device

Abstract

To provide a visual inspection method and a model generation device which suppress excessive detection.SOLUTION: A visual inspection method by a model generation device includes the steps of: synthesizing a pseudo defect image using a normal visual image and a defect image element; performing first machine learning using the synthesized pseudo defect image as an abnormal image, and generating a first learning model; inputting a first input image which is a visual image of an inspection object to the generated first learning model, and performing visual inspection of the inspection object; specifying excessive detection relating to the inspection object on the basis of the result, and specifying a cause image element which is a pseudo defect image causing the excessive detection; adjusting a parameter relating to the synthesis of the defect image element, for the cause image element; and performing second machine learning using the pseudo defect image adjusted by the pseudo defect adjustment step as an abnormal image, and generating a second learning model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a visual inspection method and a model generating device. [Background technology]

[0002] When applying machine learning to visual inspection, it is necessary to prepare a large number of images of defective and normal products as inspection targets. In addition, in order to improve the accuracy of visual inspection, the number of images of normal products and images of defective products must be approximately the same. However, in general, the number of defective products is far less than the number of good products. For this reason, there is a problem that the images of defective products are insufficient compared to images of normal products. Patent Document 1 discloses a learning data generation device including a defective part data storage unit that stores defective part data, which is image data of a defective part image, a good product data storage unit that stores good product data, which is image data of a good product image, a learning data generation unit that generates learning data, which is image data of a learning image in which the good product image is synthesized with the defective part image, based on the defective part data, the good product data, and a generation parameter, a learning data storage unit that stores the learning data, and a generation parameter setting unit that sets the generation parameter. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-027424 A Summary of the Invention [Problem to be solved by the invention]

[0004] The invention described in Patent Document 1 cannot suppress overdetection. [Means for solving the problem]

[0005] A visual inspection method according to a first aspect of the present invention is a visual inspection method for performing visual inspection of an object to be inspected using a learning model obtained by supervised learning, the method comprising: a defect image element synthesis step of generating the pseudo-defect image using the normal appearance image and the defect image elements; a first generation step of performing first machine learning using the pseudo-defect image generated by the defect image element synthesis step as the image with an abnormality to generate a first learning model; and the first learning model generated by the above process includes a first visual inspection step of inputting a first input image, which is an appearance image of the inspection object, into the first learning model generated by the above process, thereby performing a visual inspection of the inspection object; an overdetection identification step of identifying an overdetection regarding the inspection object based on a result of the first visual inspection step, and identifying a causative image element, which is the pseudo-defect image that caused the overdetection; a pseudo-defect adjustment step of adjusting parameters regarding the synthesis of the defect image element for the causative image element; and a second generation step of performing a second machine learning using the pseudo-defect image adjusted in the pseudo-defect adjustment step as the image with an abnormality to generate a second learning model. A model generating device according to a second aspect of the present invention is a model generating device which sets a normal appearance image, which is an appearance image of an object to be inspected without an abnormality, as an abnormality-free image, sets a pseudo-defect image obtained by combining the normal appearance image with a defect image element as an abnormality-containing image, and generates a learning model through supervised learning, the model generating device comprising: a defect image element synthesis program which generates the pseudo-defect image using the normal appearance image and the defect image element; a first generation program which performs first machine learning using the pseudo-defect image generated by the defect image element synthesis program as the abnormality-containing image to generate a first learning model; The system comprises a first visual inspection program that inputs a first input image, which is an appearance image of the inspection object, into a learning model to perform a visual inspection of the inspection object; an overdetection identification program that identifies overdetection regarding the inspection object based on a result of the first visual inspection program and identifies a causative image element, which is the pseudo-defect image that caused the overdetection; a pseudo-defect adjustment program that adjusts parameters related to the synthesis of the defect image element for the causative image element; and a second generation program that performs second machine learning using the pseudo-defect image adjusted by the pseudo-defect adjustment program as the image with an abnormality to generate a second learning model. Effect of the Invention

[0006] According to the present invention, overdetection can be suppressed. [Brief description of the drawings]

[0007] [Figure 1] Configuration diagram of model generation device [Diagram 2] FIG. 1 is a diagram for explaining various images according to the present embodiment. [Diagram 3] External view of the piston being inspected [Figure 4] FIG. 13 is a diagram showing an example of a pseudo defect image; [Diagram 5] A diagram showing the generation of pseudo-defect images by a defect image element synthesis program. [Figure 6] A diagram showing the generation of segmented pseudo defect images by a defect image element synthesis program. [Figure 7]1 is a flowchart showing an outline of the process of the model generating device. [Figure 8] Diagram showing examples of overdetection and causal image elements [Figure 9] FIG. 1 shows a first example of parameter adjustment by a defect adjustment program. [Figure 10] FIG. 2 shows a second example of parameter adjustment by a defect adjustment program. [Figure 11] FIG. 13 is a diagram showing identification of a causal image element in the second embodiment; [Figure 12] FIG. 13 is a diagram showing an imaging unit according to the third embodiment. [Figure 13] FIG. 13 is a diagram showing the process of a defect image element synthesis program in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] --First embodiment-- Hereinafter, a first embodiment of a model generating device will be described with reference to FIGS.

[0009] 1 is a configuration diagram of a model generating device 1. The model generating device 1 includes a CPU 41 which is a central processing unit, a ROM 42 which is a read-only storage device, a RAM 43 which is a readable and writable storage device, an image library 16, a model library 17, and an imaging unit 18. The device includes an input / output device 44 which is a user interface, and a communication device 45. The CPU 41 loads a program stored in the ROM 42 into the RAM 43 and executes it. A defect image element synthesis program 11, a learning model generation program 12, an appearance inspection program 15, an overdetection identification program 13, and a defect adjustment program 14 are loaded into the RAM 43.

[0010] The model generating device 1 may be realized by a field programmable gate array (FPGA) which is a rewritable logic circuit or an application specific integrated circuit (ASIC) which is an integrated circuit for a specific application, instead of the combination of the CPU 41, the ROM 42, and the RAM 43. The model generating device 1 may also be realized by a combination of different configurations, for example, a combination of the CPU 41, the ROM 42, the RAM 43, and an FPGA, instead of the combination of the CPU 41, the ROM 42, and the RAM 43. Before describing the configuration shown in FIG. 1, various images in this embodiment will be described.

[0011] FIG. 2 is a diagram for explaining various images in the present embodiment. The normal appearance image 21 is obtained by the photographing unit 18. A pseudo-defect image 23 is generated using the normal appearance image 21 and the defect image element 22. Although details will be described later, a plurality of pseudo-defect images 23 are generated using a pair of the normal appearance image 21 and the defect image element 22. The pseudo-defect image 23 is divided into a plurality of regions, and each divided image is called a subdivided pseudo-defect image 24. In FIG. 2, for convenience of drawing, the normal appearance image 21, the defect image element 22, the pseudo-defect image 23, and the subdivided pseudo-defect image 24 are illustrated one by one, but the quantities are different as described above. That is, a plurality of pseudo-defect images 23 are generated using a pair of the normal appearance image 21 and the defect image element 22, and a plurality of subdivided pseudo-defect images 24 are generated for each pseudo-defect image 23. Returning to FIG. 1, the explanation will be continued.

[0012] The model library 17 stores a first learning model 171 and a second learning model 172. Both the first learning model 171 and the second learning model 172 are generated by the learning model generation program 12. The photographing unit 18 photographs the inspection object and outputs a photographed image obtained by photographing. This photographed image becomes a normal appearance image 21 and an input image to the appearance inspection program 15. The photographed image of the photographing unit 18 may be temporarily stored in the image library 16. In this case, a photographed image of an inspection object that has been confirmed in advance to be free of defects may be stored as the normal appearance image 21.

[0013] The defect image element synthesis program 11 synthesizes a defect image element 22 with a normal appearance image 21, which is an appearance image of an inspection object without defects, to generate a pseudo defect image 23 and a subdivided pseudo defect image 24. The learning model generation program 12 performs machine learning using the subdivided pseudo defect image 24 to generate a first learning model 171 and a second learning model 172. The learning model generated by the defect image element synthesis program 11 using the subdivided pseudo defect image 24 without adjustment by the defect adjustment program 14 is the first learning model 171. The learning model generated by the defect image element synthesis program 11 using the subdivided pseudo defect image 24 after adjustment by the defect adjustment program 14 is the second learning model 172. For the learning by the learning model generation program 12, it is preferable to use machine learning such as a Deep Neural Network or a Support Vector Machine, using a learning image as an input.

[0014] The appearance inspection program 15 inputs the captured image output by the photographing unit 18 into a learning model to perform an appearance inspection of the object to be inspected. Hereinafter, the appearance inspection using the first learning model 171 is referred to as the "first appearance inspection", and the appearance inspection using the second learning model 172 is referred to as the "second appearance inspection". The overdetection identification program 13 identifies overdetection using the output result of the first appearance inspection. A method for identifying overdetection will be described later. The defect adjustment program 14 adjusts parameters related to the synthesis of the defect image element 22 for a specific element (hereinafter referred to as the "cause image element") of the pseudo defect image 23 that caused the overdetection, identified by the overdetection identification program 13. Specifically, the defect adjustment program 14 adjusts parameters such as the brightness, size, angle, etc. of the cause image element so as to reduce the similarity of the cause image element to the overdetected portion 402. The cause image element is one of the defect image elements 22.

[0015] The first learning model 171 and the second learning model 172 receive a captured image as input and output the inspection judgment result and coordinates where a defect is detected. The normal appearance image 21 is read into the learning model generation program 12 together with information that there is no abnormality. The subdivided pseudo defect image 24 is read into the learning model generation program 12 together with information that there is an abnormality, and the coordinates of the defect image element 22. The learning model generation program 12 learns the parameters of the first learning model 171 and the second learning model 172 so that when the normal appearance image 21 is input, a message that there is no abnormality is output. The learning model generation program 12 learns the parameters of the first learning model 171 and the second learning model 172 so that when the subdivided pseudo defect image 24 is input, a message that there is an abnormality and the coordinates of the defect image element 22 are output.

[0016] The processing executed by the defect image element synthesis program 11 is also called a "defect image element synthesis step." The generation of the first learning model 171 by the learning model generation program 12 is also called a "first generation step," and the generation of the second learning model 172 is also called a "second generation step." The processing executed by the appearance inspection program 15 is also called a "appearance inspection step." The processing executed by the overdetection identification program 13 is also called an "overdetection identification step." The processing executed by the defect adjustment program 14 is also called a "pseudo defect adjustment step." The learning model generation program 12 is divided into a first generation program that generates the first learning model 171 and a second generation program that generates the second learning model 172.

[0017] 3 is an external view of a piston 201 of an automobile engine, which is the inspection object in this embodiment. The piston 201 is a roughly cylindrical cast part. A crown surface 202, which is the upper side of the piston 201 in the figure, shows a cast surface 203 and a machined surface 204 obtained by machining the cast surface 203.

[0018] FIG. 4 is a diagram showing an example of a pseudo defect image 23. The pseudo defect image 23 in this figure is an image obtained by combining 12 defect image elements 22 with a normal appearance image 21 obtained by photographing a crown surface 202 without an abnormality from directly above. The defect image elements 22 may be images obtained by cutting out defective parts of a defective product image, or may be artificial images created to resemble defects. As shown in the enlarged view at the bottom of FIG. 4, in the following description, the size of the defect image elements 22 is called the element size Dl, and the interval between the defect image elements 22 arranged in a lattice pattern is called the defect arrangement interval Dp. In this embodiment, in order to simplify the description, the same size and interval are also assumed in the vertical direction shown in the figure. However, the size and interval in the vertical direction shown in the figure do not have to be the same as those in the horizontal direction shown in the figure. In other words, in this embodiment, the defect image elements 22 are described as being square, but the defect image elements 22 may be rectangular. Here, "without anomalies" means that there are no defects in the inspection, and includes the presence of minute irregularities that are not recognized as defects.

[0019] FIG. 5 is a diagram showing the generation of the pseudo defect image 23 by the defect image element synthesis program 11. The defect image element synthesis program 11 first defines a coordinate system with an arbitrary point of the normal appearance image 21 as the reference point 20 as the origin. The reference point 20 and the coordinate system are common to the normal appearance image 21 and all the pseudo defect images 23. In this embodiment, changing the starting position of the synthesis position of the defect image element 22 is expressed by the concept of "phase". The defect image element synthesis program 11 generates a pseudo defect image 23 with a phase (0, 0) as the first pseudo defect image 23. In the pseudo defect image 23 with a phase (0, 0), the first defect image element 22 is arranged at the origin. Then, the defect image element synthesis program 11 sequentially arranges the defect image elements 22 at positions spaced apart by the defect arrangement interval Dp in the X-axis direction and the Y-axis direction from this first defect image element 22 as the starting point.

[0020] In the pseudo defect image 23 with phase (1, 0), the first defect image element 22 is arranged at a position shifted by only the phase interval Pp in the X direction from the reference point 20. Since the arrangement of the second and subsequent defect image elements 22 is the same as before, the description will be omitted in the following explanation. In the pseudo defect image 23 with phase (0, 1), the first defect image element 22 is arranged at a position shifted by only the phase interval Pp in the Y direction from the reference point 20. In the pseudo defect image 23 with phase (1, 1), the first defect image element 22 is arranged at a position shifted by the phase interval Pp in the X direction and the Y direction respectively from the reference point 20. Generally speaking, in the pseudo defect image 23 with phase (n, m), the first defect image element 22 is arranged at a position shifted by n times the phase interval Pp in the X direction and m times the phase interval Pp in the Y direction from the reference point 20.

[0021] Figure 6 is a diagram showing the generation of the subdivided pseudo defect image 24 by the defect image element synthesis program 11. As described above, the defect image element synthesis program 11 generates the subdivided pseudo defect image 24 by subdividing the pseudo defect image 23. The defect image element synthesis program 11 always generates each of the regions delimited at intervals of the predetermined cutout dimension Ls with the reference point 20 as a reference as the subdivided pseudo defect image 24. This grid is parallel to the X-axis and the Y-axis, for example. However, the defect image element synthesis program 11 does not necessarily have to actually create the subdivided pseudo defect image 24. That is, the defect image element synthesis program 11 may treat each region of the pseudo defect image 23 as the subdivided pseudo defect image 24.

[0022] The following relational expressions hold for the above-described element dimension Dl, cutout dimension Ls, phase interval Pp, and defect arrangement interval Dp.

[0023] Dp > Ls + Dl ··· (Equation 1)

[0024] Dp < Ls + Dl + Pp ··· (Equation 2)

[0025] Since the equality of the formula 1 and the formula 2 is established, only one defect image element 22 exists in each of the subdivided pseudo defect images 24, and the defect image element 22 exists at the same position in each of the subdivided pseudo defect images 24.

[0026] 7 is a flowchart showing an outline of the processing of the model generating device 1. In the model generating device 1, first, in step S101, the defect image element synthesis program 11 creates a pseudo defect image 23 and a subdivided pseudo defect image 24. Next, the learning model generation program 12 generates a first learning model 171 using the subdivided pseudo defect image 24. In the following step S103, the appearance inspection program 15 performs a first appearance inspection using the first learning model 171. In the following step S104, the overdetection identification program 13 identifies overdetection in the first appearance inspection.

[0027] In the following step S105, the defect adjustment program 14 adjusts the defective pixel elements, and the defect image element synthesis program 11 newly creates the subdivided pseudo defect image 24. In the following step S106, the learning model generation program 12 generates a second learning model 172 using the subdivided pseudo defect image 24 generated in step S105. In the following step S107, the appearance inspection program 15 performs a second appearance inspection using the second learning model 172.

[0028] Fig. 8 is a diagram showing an example of overdetection and causal image elements. An overdetected image 401 in which overdetection has occurred is shown on the left side of Fig. 8, and a pseudo defect image 23-1 is shown on the right side of Fig. 8. In the example shown in Fig. 8, an overdetected point 402 is present approximately in the center, toward the left, of the overdetected image 401. It can be confirmed by any of the following methods that the overdetected point 402 is an overdetection, in other words, that the overdetected point 402 is not an abnormality that should actually be detected by the visual inspection program 15.

[0029] The first method is a method in which all images input to the first learning model 171 are normal images. For example, a captured image obtained by photographing an inspection object that has passed inspection is input to the first learning model 171, or a captured image that has been previously confirmed by a human to be free of abnormalities is input to the first learning model 171. The second method is a method in which the image is input to another appearance inspection method unrelated to the model library 17 for confirmation. That is, a captured image in which the first learning model 171 has detected a defect is input to another appearance inspection method unrelated to the model library 17, and if this appearance inspection method does not detect an abnormality, it is determined to be an overdetection.

[0030] When the overdetection identification program 13 detects an overdetection, it collates the coordinates of the overdetected point 402 on the pseudo defect image 23-1 to determine the corresponding coordinates 403. Then, it identifies the defect image element 22 closest to the corresponding coordinates 403 as the causative image element 22C that is the cause of the overdetection. Since a plurality of pseudo defect images 23 are generated as described above, the causative image element 22C is identified in each of the plurality of pseudo defect images 23.

[0031] FIG. 9 is a diagram showing a first example of parameter adjustment by the defect adjustment program 14. In this example, the defect adjustment program 14 adjusts the luminance of the cause image element 22C. In FIG. 9, the horizontal axis represents the luminance of the background of the defect image element 22 (hereinafter also referred to as "background luminance"), and the vertical axis represents the luminance of the defect image element 22 (hereinafter also referred to as "defect luminance"). For example, the background luminance is the luminance of the brightest point in the defect image element 22, and the missing luminance is the luminance of the darkest point in the defect image element 22. Also, for example, the background luminance is the average value of the luminance of the surroundings of the defect image element 22 in the defect image element 22, and the missing luminance is the average value of the luminance in the defect image element 22.

[0032] In Fig. 9, for each defect image element 22 that is a defect detected in the past, points corresponding to its background luminance and defect luminance are plotted as white circles. In each plot, the defect luminance generally monotonically increases relative to the background luminance, and the regression line 504 is shown as a solid line. The dotted lines above and below the regression line 504 indicate a possible defect range 505, which is a range in which a defect is possible. In Fig. 9, the causative image element 22C identified by the overdetection identification program 13 is also plotted in the same way. However, since the overdetected portion 402 is an overdetection, the relationship between the background luminance of the portion that was erroneously determined to be a defect and the luminance of the portion that was erroneously determined to be a defect is shown.

[0033] Then, the defect adjustment program 14 adjusts the parameters of the cause image element 22C so as to reduce the similarity with the overdetected portion 402. In Fig. 9, the defect luminance of the cause image element 22C is increased so as to increase the distance between the cause image element 22C and the overdetected portion 402. However, extreme changes are not preferable, and adjustment is made within a range in which the defect luminance after adjustment falls within a defect possibility range 505. Note that it is preferable to adjust the cause image element 22C not only so that the distance between the overdetected portion 402 and the cause image element 22C is increased, but also so that the difference in the distance to the regression line 504 from the overdetected portion 402 is increased.

[0034] Fig. 10 is a diagram showing a second example of parameter adjustment by the defect adjustment program 14. In the histogram shown in Fig. 10, the horizontal axis represents defect size, and the vertical axis represents the occurrence frequency for each defect size. In the histogram, each defect detected in the past is plotted based on its size and occurrence frequency. The dotted lines on the left and right of the histogram indicate a defect possibility range 1104, which is a range where there is a possibility of a defect based on the defects detected so far.

[0035] The defect adjustment program 14 plots the overdetected portion 402 identified by the overdetection identification program 13 on a histogram based on its size. The defect adjustment program 14 also plots the cause image element 22C identified as the cause of the overdetection in the same way. The defect adjustment program 14 adjusts the parameters of the cause image element 22C so that the similarity between the overdetected portion 402 and the cause image element 22C decreases, and sets the cause image element 22C as the adjusted cause image element 22D. That is, in the example of FIG. 10, the cause image element 22C is enlarged so as to move away from the plot position of the overdetected portion 402, and sets the cause image element 22C as the adjusted cause image element 22D. The amount of this adjustment is set to be within the defect possibility range 1104.

[0036] According to the above-described first embodiment, the following advantageous effects can be obtained. (1) The model generating device 1 regards a normal appearance image 21, which is an appearance image of an inspection object without an abnormality, as an abnormality-free image, and regards a pseudo-defect image 23, which is obtained by combining the normal appearance image 21 with a defect image element 22, as an abnormality-containing image, and performs an appearance inspection of the inspection object using a learning model obtained by supervised learning. The defect image element synthesis program 11 executes a defect image element synthesis step of generating a pseudo-defect image 23 using the normal appearance image 21 and the defect image element 22. The learning model generating program 12 executes a first generation step of performing a first machine learning using the pseudo-defect image 23 generated by the defect image element synthesis step as an abnormality-containing image to generate a first learning model 171. The appearance inspection program 15 executes a first appearance inspection step of inputting a first input image, which is an appearance image of the inspection object, into the first learning model 171 generated by the first generation step to perform an appearance inspection of the inspection object. The overdetection identification program 13 executes an overdetection identification step of identifying an overdetection of the inspection object based on the result of the first appearance inspection step, and identifying a cause image element 22C, which is the pseudo-defect image 23 that caused the overdetection. The defect adjustment program 14 executes a pseudo-defect adjustment step of adjusting parameters related to synthesis of the defect image element 22 for the cause image element 22C. The learning model generation program 12 executes a second generation step of performing a second machine learning using the pseudo-defect image 23 adjusted by the defect adjustment program 14 as an image with an abnormality to generate a second learning model 172. Therefore, the second learning model 172 is generated after correcting the learning data so that overdetection that occurred in the first learning model 171 does not occur, so that overdetection can be suppressed by using the second learning model 172.

[0037] (2) The defect adjustment program 14 adjusts at least one parameter of the brightness, size, and angle of the defect image element 22 so as to reduce the similarity of the cause image element 22C to the overdetected portion 402. Therefore, the learning data can be corrected in terms of brightness, size, and angle to prevent overdetection.

[0038] (3) In the first generation step and the second generation step, the first learning model 171 and the second learning model 172 are generated using the subdivided pseudo defect images 24 obtained by dividing the pseudo defect image 23 into a grid shape. In the defect image element synthesis step, the defect image elements 22 are arranged in a grid shape so that only one defect image element 22 exists in each of the subdivided pseudo defect images 24 and the defect image elements 22 exist in the same positions in the subdivided pseudo defect images 24, and are synthesized into the normal appearance image 21. Therefore, efficient learning can be achieved by preparing learning images that meet the specifications of the learning model generation program 12.

[0039] (Variation 1) In the above-described first embodiment, the defect adjustment program 14 adjusted the brightness or size of the defect image element 22 as shown in Figures 9 and 10 so as to reduce the similarity of the cause image element 22C to the overdetected portion 402. However, the defect adjustment program 14 may adjust at least one parameter of the brightness, size, and angle of the defect image element 22 so as to reduce the similarity of the cause image element 22C to the overdetected portion 402. For example, the defect adjustment program 14 may adjust the angle of the cause image element 22C, or may adjust both the brightness and size of the cause image element 22C.

[0040] --Second embodiment-- A second embodiment of a model generating device will be described with reference to FIG. 11. In the following description, the same components as those in the first embodiment are given the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as those in the first embodiment. This embodiment differs from the first embodiment mainly in terms of handling rotation of a captured image.

[0041] FIG. 11 is a diagram showing the identification of the causative image element 22C in the second embodiment. The process described below is an additional process executed by the overdetection identification program 13. The captured image when there is no rotation of the inspection object is shown in the upper part of FIG. 11. The overdetection identification program 13 detects the rotation based on the left end 708 and the right end 709 of the machined surface 204 of this captured image 704. The captured image 701 shown in the lower left of FIG. 11 includes the detection point 702 detected by the overdetection identification program 13. The captured image 701 is captured with the inspection object rotated slightly. The overdetection identification program 13 calculates the orientation error 706 from the deviation amount of the left end 711 and the right end 712 in the captured image 701. Then, the overdetection identification program 13 uses this orientation error 706 to calculate the corresponding coordinates 713 in the pseudo defect image 23-1 with high accuracy. The process after calculating the corresponding coordinates 713 is the same as that in the first embodiment, so the description will be omitted.

[0042] According to the above-described second embodiment, the following advantageous effects can be obtained. (4) The pseudo defect image 23 is captured with the object in a reference orientation. In the overdetection identification step, an orientation error, which is a change from the reference orientation of the object in the first input image, is detected, and the first input image is transformed based on the orientation error to identify the causative image element 22C. In other words, when the positioning accuracy of the object is poor, the captured image 701 is transformed, i.e., rotated, based on the calculated orientation error to identify the causative image element 22C. Therefore, the overdetection identification program 13 can accurately identify the causative image element 22C.

[0043] --Third embodiment-- A third embodiment of a model generating device will be described with reference to Figs. 12 and 13. In the following description, the same components as in the first embodiment are given the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as in the first embodiment. This embodiment differs from the first embodiment mainly in that normal appearance image 21 is captured from multiple angles and defect image element 22 is deformed according to the angle.

[0044] 12 is a diagram showing the photographing unit 18 in the third embodiment. The photographing unit 18 includes a jig 802 and an imaging device 803. The jig 802 is a jig having two rotation axes capable of changing the posture of the inspection object 801. The photographing unit 18 photographs normal appearance images 21, which are appearance images of the inspection object without defects, in a plurality of postures by a cooperative operation with the jig 802. That is, in this embodiment, a plurality of appearance images of one inspection object without defects are photographed by changing the relative position and relative angle between the inspection object 801 and the imaging device 803.

[0045] FIG. 13 is a diagram showing the processing of the defect image element synthesis program 11 in the third embodiment. In this embodiment, the defect image element synthesis program 11 adjusts so that the position and size of the defect image element 22 on the machined surface 204 do not change even if the position and posture change. That is, the defect image element synthesis program 11 synthesizes the defect image element 22 for each normal appearance image 21, taking into consideration the relative position and relative angle between the inspection object 801 and the imaging device 803. The pseudo defect image 23-3 based on the image taken obliquely has a shape that is squashed up and down, compared to the pseudo defect image 23-2 based on the image taken from directly above. Even in this case, the defect image element 22 is synthesized at a position that has been subjected to coordinate transformation based on the relative angle, so there is no contradiction between the two images. In other words, it is expressed as if a defect actually exists on the machined surface 204.

[0046] For example, when the inspection object 801 is disposed at an angle with respect to the imaging device 803, the defect image element synthesis program 11 distorts the defect image element 22 according to the angle and then synthesizes it into the normal appearance image 21. The relative position and relative angle between the inspection object 801 and the imaging device 803 may be calculated by the defect image element synthesis program 11 through image processing. Furthermore, when the normal appearance image 21 is captured, a recording device (not shown) may record the attitude of the jig 802 and link it to the normal appearance image 21.

[0047] The defect image element synthesis program 11 may use three-dimensional data of the piston 201, which is the object to be inspected. For example, three-dimensional data showing the three-dimensional shape of the machined surface 204 may be obtained from a design drawing. The defect image elements 22 may be attached to this three-dimensional data, and two-dimensional images may be written from multiple angles, i.e., so-called rendering may be performed.

[0048] According to the above-described third embodiment, the following advantageous effects can be obtained. (5) The normal appearance images 21 are a plurality of images obtained by photographing the object to be inspected from a plurality of angles. In the defect image element synthesis step, the defect image elements 22 are synthesized at positions obtained by performing coordinate transformation based on the angle for each of the normal appearance images 21. Therefore, since the defect image elements 22 are synthesized at positions corresponding to each other in a plurality of orientations, the causative image elements 22C that are the cause of overdetection are always present at corresponding positions in a plurality of orientations, and therefore can be efficiently adjusted by identifying them all at once and adjusting each of them.

[0049] In each of the above-mentioned embodiments and modifications, the configuration of the functional blocks is merely an example. Some functional configurations shown as separate functional blocks may be integrated, or a configuration shown in one functional block diagram may be divided into two or more functions. In addition, some of the functions of each functional block may be provided by other functional blocks.

[0050] In the above-mentioned embodiments and modifications, the programs are stored in the ROM 42, but the programs may be stored in a non-volatile storage device. The model generating device 1 may have an input / output interface (not shown), and the programs may be read from another device via the input / output interface and a medium available to the model generating device 1 when necessary. The medium here refers to, for example, a storage medium that is detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, or optical network, or a carrier wave or digital signal that propagates through the network. Some or all of the functions realized by the programs may be realized by a hardware circuit or an FPGA.

[0051] The above-mentioned embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other aspects that are conceivable within the scope of the technical idea of ​​the present invention are also included in the scope of the present invention. [Explanation of symbols]

[0052] 1: Model generation device 11: Defect image element synthesis program 12: Learning model generation program 13: Overdetection identification program 14: Fault correction program 15: Visual inspection program 21: Normal appearance image 22: Defective image element 22C:Cause image element 22D: Adjusted cause image element 23: Pseudo defect image 24: Subdivision pseudo defect image 171: First learning model 172: Second Learning Model

Claims

1. A visual inspection method for inspecting the appearance of an object to be inspected using a learning model obtained by supervised learning, the method comprising: setting a normal appearance image, which is an appearance image of an object to be inspected without an abnormality, as an abnormality-free image; setting a pseudo-defect image, which is obtained by combining the normal appearance image with a defect image element, as an abnormality-containing image; a defect image element synthesis step of generating the pseudo defect image using the normal appearance image and the defect image element; a first generation step of performing a first machine learning process using the pseudo-defect image generated by the defect image element synthesis step as the image with an abnormality to generate a first learning model; a first appearance inspection step of inputting a first input image, which is an appearance image of the inspection object, into the first learning model generated by the first generation step, and performing an appearance inspection of the inspection object; an overdetection identifying step of identifying an overdetection related to the inspection object based on a result of the first appearance inspection step, and identifying a cause image element which is the false defect image that caused the overdetection; a pseudo defect adjustment step of adjusting a parameter related to synthesis of the defect image element for the cause image element; A second generation step of performing second machine learning using the pseudo defect image adjusted in the pseudo defect adjustment step as the image with an abnormality to generate a second learning model.

2. 2. The appearance inspection method according to claim 1, The appearance inspection method, wherein the pseudo defect adjustment step adjusts at least one parameter of the brightness, size, and angle of the defect image element so as to reduce the similarity of the cause image element to the overdetected portion.

3. 2. The appearance inspection method according to claim 1, In the first generation step and the second generation step, the first learning model and the second learning model are generated using a segmented pseudo defect image obtained by dividing the pseudo defect image into a lattice shape, In the defect image element synthesis step, the defect image elements are arranged in a grid pattern so that each of the subdivided pseudo defect images has only one defect image element, and the defect image elements are arranged in the same positions in the subdivided pseudo defect images, and then synthesized into the normal appearance image.

4. 2. The appearance inspection method according to claim 1, The pseudo defect image is captured with the inspection object in a reference posture, An appearance inspection method in which the overdetection identification step detects an orientation error, which is a change from the reference orientation of the object in the first input image, and transforms the first input image based on the orientation error to identify the causative image element.

5. 2. The appearance inspection method according to claim 1, The normal appearance images are a plurality of images obtained by photographing the inspection object from a plurality of angles, The defect image element synthesizing step synthesizes the defect image element at a position obtained by performing coordinate transformation on each of the normal appearance images based on the angle.

6. A model generating device which generates a learning model in supervised learning, comprising: a normal appearance image, which is an appearance image of an inspection object without an abnormality, is defined as an abnormality-free image; a pseudo-defect image, which is obtained by combining the normal appearance image with a defect image element, is defined as an abnormality-containing image; a defect image element synthesis program for generating the pseudo-defect image by using the normal appearance image and the defect image element; a first generation program that performs first machine learning using the pseudo-defect image generated by the defect image element synthesis program as the image with an abnormality to generate a first learning model; a first appearance inspection program for inputting a first input image, which is an appearance image of the inspection object, into the first learning model generated by the first generation program to perform an appearance inspection of the inspection object; an overdetection identification program for identifying an overdetection related to the inspection object based on a result of the first visual inspection program, and identifying a causative image element that is the false defect image that caused the overdetection; a pseudo-defect adjustment program for adjusting parameters related to synthesis of the defect image element for the cause image element; a second generation program that performs second machine learning using the pseudo defect image adjusted by the pseudo defect adjustment program as the image with an abnormality to generate a second learning model.

Images