Inspection device

Abstract

The present invention provides an inspection device having a function of automatically confirming the validity of appearance inspection results without manual work. The inspection device for performing appearance inspection using multiple images obtained by photographing an inspection object is provided with a determination unit, which estimates the inspection results related to the multiple images respectively based on a predefined relationship model between the images and the inspection results, and calculates a comprehensive inspection result based on the reliability of each of the inspection results related to the multiple images.

Description

Technical Field

[0001] The present invention relates to an inspection device, and in particular to an inspection device having a function of automatically confirming the validity of appearance inspection results without relying on manual labor. Background Art

[0002] In the appearance inspection process of the inspection object (workpiece after processing, etc.), inspection has been performed by human visual inspection in the past. In recent years, inspection by taking an image of the inspection object with a camera and processing the image has been widely performed.

[0003] Recently, machine learning, such as deep learning, has been used in appearance inspections. By using machine learning, it is possible to accurately determine scratches and dents, which cannot be determined in detail using numerical criteria.

[0004] Japanese Patent Application Publication No. 2011-007553 describes that optical conditions are changed for each defect type, and different scattered light is detected by multiple detection units, and the detected light is compared with learning data to perform abnormality detection. Japanese Patent Application Publication No. 2011-007553 also discloses that when detecting abnormalities, the results of multiple identifiers can be comprehensively determined by OR (logical OR) and majority decision.

[0005] Japanese Patent Application Laid-Open No. 2009-103508 describes that a plurality of detectors detect scattered light at different angles, and compare the detected scattered light with a classifier obtained by learning. If there is a detector that exceeds a threshold value at even one position, the detection is determined to be a defect.

[0006] However, it is currently necessary to manually check whether the inspection results using machine learning are correct. For example, the inspection results of machine learning are compared with the existing inspection results based on human visual inspection to check the inspection results of machine learning. The inspection results of machine learning are affected by shooting conditions, etc. Therefore, it is necessary to manually evaluate the reliability of the inspection results of machine learning while changing the shooting conditions multiple times by trial and error. Summary of the invention

[0007] The present invention is to solve such a problem, and an object of the present invention is to provide an inspection device having a function of automatically confirming the validity of appearance inspection results without relying on manual work.

[0008] An inspection device according to one embodiment of the present invention is an inspection device that performs appearance inspection using multiple images obtained by photographing an inspection object, and includes a learning unit that estimates inspection results respectively associated with the multiple images based on a pre-defined relationship model between the images and the inspection results, and calculates a comprehensive inspection result based on the reliability of each of the inspection results associated with the multiple images.

[0009] In the inspection device of one embodiment of the present invention, when the reliability of the comprehensive inspection result does not satisfy a certain condition, the learning unit outputs an instruction to re-take the image, and recalculates the comprehensive inspection result using the re-taken image.

[0010] In an inspection device of one embodiment of the present invention, when the reliability of the above-mentioned comprehensive inspection result does not meet certain conditions, the above-mentioned learning unit outputs an instruction to re-take the above-mentioned image, and uses the above-mentioned image taken again and the above-mentioned images taken until the last time to recalculate the above-mentioned comprehensive inspection result.

[0011] The inspection device according to one embodiment of the present invention further includes a photographing condition determination unit that determines photographing conditions of the image, wherein the photographing condition determination unit changes the photographing conditions for the re-photographing with respect to the photographing conditions for the previous photographing.

[0012] In the inspection device according to one embodiment of the present invention, the imaging condition determination unit increases the number of images in the re-imaging relative to the number of images in the previous imaging for an inspection portion whose reliability of the inspection result does not satisfy a certain condition.

[0013] In the inspection device according to one embodiment of the present invention, the imaging condition determination unit increases the number of images to be captured in the re-image relative to the number of images to be captured in the previous image capture, with respect to the inspection portion for which the inspection result is unacceptable.

[0014] In the inspection device according to one embodiment of the present invention, the learning unit includes a plurality of the relationship models, and estimates the inspection result using different relationship models based on the plurality of images.

[0015] In the inspection device according to one embodiment of the present invention, the learning unit includes a plurality of the relationship models, and estimates the inspection result using the relationship model different from that used for the image captured last time for the image captured again.

[0016] According to the present invention, it is possible to provide an inspection device having a function of automatically confirming the validity of appearance inspection results without relying on human labor. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects and features of the present invention will become clear from the following description of the embodiments with reference to the accompanying drawings.

[0018] Figure 1 This shows the hardware structure of the inspection device.

[0019] Figure 2 This is one way of showing the functional structure of the inspection device.

[0020] Figure 3 This is one way of showing the functional structure of the inspection device.

[0021] Figure 4 This is one way of showing the functional structure of the inspection device. DETAILED DESCRIPTION

[0022] Figure 1 1 is a schematic hardware structure diagram showing the main parts of an inspection device 1 according to an embodiment of the present invention. The inspection device 1 is, for example, a device for performing appearance inspection of a processed workpiece. The inspection device 1 includes a CPU 11, a ROM 12, a RAM 13, a nonvolatile memory 14, an interface 18, an interface 19, a bus 10, an input / output device 60, and a camera 70.

[0023] The CPU 11 is a processor that controls the entire inspection device 1. The CPU 11 reads a system program stored in the ROM 12 via the bus 10, and controls the entire inspection device 1 in accordance with the system program.

[0024] The ROM 12 stores in advance a system program for executing various controls of the machine.

[0025] The RAM 13 temporarily stores calculation data, input / output data, and the like.

[0026] The nonvolatile memory 14 is supported by, for example, a battery (not shown), and maintains the storage state even when the power supply of the inspection device 1 is turned off. The nonvolatile memory 14 stores data input from the input / output device 60, image data input from the imaging device 70 via the interface 19, and the like. The programs and data stored in the nonvolatile memory 14 can be expanded in the RAM 13 when executed or used.

[0027] The input / output device 60 is a data input / output device including a display, a keyboard, etc. The input / output device 60 displays information received from the CPU 11 via the interface 18 on the display. The input / output device 60 transmits commands and data input from the keyboard, etc. to the CPU 11 via the interface 18 .

[0028] The photographing device 70 is a device that photographs the appearance of the inspection object and generates image data. The photographing device 70 includes, for example, a light source that irradiates light to the inspection object, a camera that photographs the inspection object under predetermined photographing conditions, and an output unit that outputs the photographing result of the camera as image data. The photographing device 70 may also include a plurality of light sources and cameras. The photographing device 70 photographs according to instructions received from the CPU 11 via the interface 19. The photographing device 70 transmits the image data to the CPU 11 via the interface 19.

[0029] Figure 2 1 is a schematic functional block diagram of the inspection device 1 according to the first embodiment. The inspection device 1 includes an imaging condition determination unit 101, an imaging unit 102, and a determination unit 103. The various functions of the imaging condition determination unit 101, the imaging unit 102, and the determination unit 103 may be realized as one function of the CPU 11, or may be realized by the CPU 11 executing a program stored in the ROM 12, the RAM 13, and the nonvolatile memory 14.

[0030] The shooting condition determination unit 101 determines shooting conditions for the appearance of the inspection object. The shooting conditions include, for example, the angle of the light source or camera relative to the inspection object, the positional relationship between the camera and the light source, the type of light source (color temperature, brightness, etc.), and the number of pixels of the camera. In addition, when an optical filter or software-based filtering is used, the shooting conditions include information such as its type, exposure time, dynamic range, the number of cameras used for shooting, the number of shots, and which part of the inspection object is shot.

[0031] The imaging unit 102 captures an image of the appearance of the inspection object according to the imaging conditions determined by the imaging condition determination unit 101 , and generates and outputs one or more image data.

[0032] The determination unit 103 includes a learning unit 1031 and a comprehensive determination unit 1032. The learning unit 1031 uses the image data output by the photographing unit 102 as input and performs appearance inspection, that is, pass / fail determination, through machine learning. Various known techniques can be used for appearance inspection based on machine learning. The comprehensive determination unit 1032 determines the reliability of the inspection result of machine learning. The comprehensive determination unit 1032 performs photographing again based on the reliability determination result.

[0033] Hereinafter, an example of the operation of the inspection device 1 will be described as an embodiment.

[0034] <First shot of the inspection object>

[0035] The photographing unit 102 photographs the inspection object using the photographing conditions predetermined by the photographing condition determination unit 101. In the present embodiment, the photographing unit 102 can change the photographing conditions to perform multiple photographing. For example, the photographing unit 102 can change the brightness of the light source, the angle of the light source or the camera relative to the inspection object (such as changing the angle by 120 degrees each time and photographing 3 images). The photographing unit 102 outputs a plurality of image data obtained in this way with different photographing conditions.

[0036] <Inspection and comprehensive judgment based on machine learning>

[0037] The determination unit 103 inputs a plurality of pieces of image data having different imaging conditions output by the imaging unit 102 to the learning unit 1031 .

[0038] The learning unit 1031 has a learning model constructed by a known machine learning method (representatively supervised learning, unsupervised learning, etc.). The learning model is obtained by learning the correlation between image data (or any feature quantity based on image data) and inspection results (values ​​indicating the result of the pass / fail judgment) to a certain degree of confidence. That is, the learning model can take image data as input and output an inspection result that is presumed to be highly correlated with the image data.

[0039] In this embodiment, the learning unit 1031 automatically extracts appropriate feature quantities from the image data through deep learning, which is a well-known technology, and outputs an inspection result that has a strong correlation with the feature quantity. The learning unit 1031 outputs a numerical value indicating the degree of non-conformity (hereinafter referred to as NG degree) as the inspection result. The NG degree, for example, is an indicator that represents the degree of non-conformity with a numerical value between 0 and 100, where the case where the inspection object can be said to be completely qualified (qualified product) is set to 0, and the case where it can be said to be completely unqualified (unqualified product) is set to 100. The closer the NG degree is to 0 or 100, the higher the possibility of passing or failing, and the farther it is from 0 and 100, the lower the possibility of passing or failing. That is, the farther the NG is from 0 and 100, the less it can be clearly determined whether it is passed or failed. That is, the NG degree is an indicator indicating the reliability of the inspection result. The closer the NG degree is to 0 or 100, the higher the reliability of the inspection result, and the farther it is from 0 and 100, the lower the reliability of the inspection result.

[0040] An example of a method for calculating the NG degree. For example, assume that a cluster of feature quantities of image data that failed inspection is formed during the construction phase of the learning model. At this time, the NG degree is set to 100 when the distance D between the input node (feature quantity of the image data of the inspection object) and the cluster centroid is 0. When the distance D between the input node and the cluster centroid is sufficiently far away, specifically, when D ≥ a predetermined threshold value Dth, the NG degree is set to 0. When the distance D between the input node and the cluster centroid is 0<D<Dth, the NG degree can be calculated by (1-(D / (distance between the cluster centroid and Dth)))×100.

[0041] The learning unit 1031 inputs the plurality of image data with different shooting conditions output by the shooting unit 102 to the learning model, and obtains the inspection results corresponding to the respective image data. Since the shooting conditions of the plurality of image data are different, for example, the appearance of the scars may be different. Therefore, as the inspection results based on the plurality of image data, different NG degrees may be output.

[0042] Here, the learning unit 1031 may input all of the multiple image data into the same learning model, or Figure 3 For example, when a plurality of learning models optimized for specific imaging conditions are constructed, the inspection can be performed using a learning model suitable for the imaging conditions of the input image data.

[0043] The comprehensive determination unit 1032 obtains multiple inspection results (inspection results related to the same inspection object but based on multiple image data with different shooting conditions) output by the learning unit 1031, and determines the final inspection result according to the content. For example, the comprehensive determination unit 1032 classifies the inspection results into three types according to the NG degree: "unqualified" (NG degree is from 81 to 100), "unclear" (NG degree is from 21 to 80), and "qualified" (NG degree is from 0 to 20). In addition, as long as one "unqualified" (NG degree is from 81 to 100) is included in the multiple inspection results output by the learning unit 1031, the comprehensive inspection result is determined to be "unqualified". Alternatively, a majority decision is adopted for the multiple inspection results output by the learning unit 1031, and the inspection result with the most "unqualified", "unclear" and "qualified" is used as the comprehensive inspection result. Alternatively, the inspection result with the highest reliability among the multiple inspection results output by the learning unit 1031, that is, the inspection result with the NG degree closest to 0 or 100 is used as the comprehensive inspection result. Alternatively, the statistics (e.g., average value, median value, mode value, etc.) of the multiple inspection results output by the learning unit 1031 are calculated, and the statistics are judged as "unqualified", "unclear", or "qualified" as the final inspection result. Alternatively, a learning model that takes the multiple inspection results output by the learning unit 1031 as input and can output a comprehensive inspection result can be used to infer the comprehensive inspection result. The learning model is obtained by pre-learning the correlation between the comprehensive inspection result determined by various methods and the multiple inspection results output by the learning unit 1031 that serve as the basis thereof through a known machine learning method.

[0044] When the comprehensive judgment unit 1032 outputs "failed" or "passed" as the comprehensive inspection result, the user can accept it as the final inspection result. The final inspection result is a result obtained by comprehensively considering the inspection results of multiple image data based on different shooting conditions, and the possibility of being evaluated as "failed" or "passed" is quite high, so the reliability is high.

[0045] On the other hand, when the comprehensive determination unit 1032 outputs "unknown" as the comprehensive inspection result, it means that a highly reliable inspection result cannot be obtained comprehensively in the shooting corresponding to the output. Therefore, the comprehensive determination unit 1032 instructs the shooting condition determination unit 101 and the shooting unit 102 to change the shooting conditions and try shooting again.

[0046] <Re-shoot>

[0047] When receiving an instruction to shoot again, the shooting condition determination unit 101 changes the above-mentioned various shooting conditions used in the last shooting, and notifies the shooting unit 102 of the changed shooting conditions. The shooting condition determination unit 101 can change the shooting conditions randomly. Alternatively, the shooting condition determination unit 101 can change the predetermined shooting conditions according to a predetermined rule. For example, the shooting condition determination unit 101 decides to change the relative positions of the inspection object, the camera, and the light source when shooting again. The shooting condition determination unit 101 changes the shooting conditions in the following manner: rotating the camera or the light source around the inspection object so that the angle of the camera or the light source relative to the inspection object is offset by a predetermined angle (for example, 60 degrees) relative to the last shooting. The shooting condition determination unit 101 can change the angle of the camera or the light source relative to the inspection object by rotating the inspection object on the workbench, or the shooting condition determination unit 101 can increase the number of shots (for example, 3) when shooting again compared to the number of shots (for example, 1) when shooting for the first time.

[0048] In addition, the shooting condition determination unit 101 can evaluate the NG degree for each shooting part of the inspection object (the above-mentioned comprehensive inspection result evaluation method can be applied), and change the shooting conditions for each shooting part according to the NG degree. For example, the shooting condition determination unit 101 can adjust the shooting parts with low reliability (NG degree is 21 to 80, etc.) so as to increase the number of shots, etc., to increase the judgment material of the inspection result. The shooting condition determination unit 101 can also increase the number of shots for the shooting parts judged as "unqualified" to obtain a more accurate inspection result.

[0049] After re-shooting, the determination unit 103 uses the re-shot image data to re-evaluate the comprehensive inspection result through the above steps. In this way, the inspection device 1 repeatedly performs re-shooting and re-evaluation of the comprehensive inspection result until a comprehensive inspection result with high reliability is obtained.

[0050] Here, the learning unit 1031 may use the same learning model in the first evaluation and the second and subsequent evaluations, or may use different learning models as shown in FIG. Figure 4 For example, if a learning model optimized for reshooting is constructed, it is appropriate to use the learning model optimized for reshooting in the second and subsequent evaluations.

[0051] Alternatively, the inspection device 1 may add image data by re-shooting, and repeatedly perform re-evaluation of the comprehensive inspection result until a comprehensive inspection result with high reliability is obtained.

[0052] At this time, the learning unit 1031 outputs the inspection result using the image data obtained by re-shooting. The comprehensive judgment unit 1032 calculates the comprehensive inspection result using both the inspection result estimated from the image data obtained by the previous shooting and the inspection result estimated from the image data newly obtained by re-shooting. In this way, the base number of image data used in the final judgment can be increased, and a high-precision inspection result can be output.

[0053] According to the present embodiment, the judgment of the validity of the inspection result, which was conventionally performed manually, can be automated, and therefore automatic appearance inspection (for example, appearance inspection based on machine learning) that does not rely on manual labor can be introduced more easily.

[0054] As mentioned above, although embodiment of this invention was described, this invention is not limited to the example of the said embodiment, By adding appropriate changes, it is possible to implement in various forms.

[0055] For example, in the above embodiment, the learning unit 1031 performs appearance inspection using a learning model constructed by machine learning, but the present invention is not limited thereto. The learning unit 1031 may perform appearance inspection using any other automated method such as an expert system constructed based on rules.

[0056] In addition, in the above embodiment, it is described that the inspection device 1 is equipped with the imaging unit 102, but the present invention is not limited to this, and the function equivalent to the imaging unit 102 may be provided outside the inspection device 1. For example, the imaging condition determination unit 101 of the inspection device 1 may control a robot equipped with a camera and a light source, thereby obtaining image data of the inspection object. Alternatively, the imaging condition determination unit 101 of the inspection device 1 may control an external imaging unit equipped with a camera and a light source and having an information processing function, thereby obtaining image data of the inspection object.

[0057] The inspection device 1 described in the above embodiment can be installed in various information processing devices. For example, the inspection device 1 may be an inspection machine, a robot controller, a numerical control device for a machine tool, a PC connected with a camera and a light source, or the like.

Claims

1. An inspection device that uses a plurality of images obtained by photographing an inspection object to perform appearance inspection on the inspection object, characterized in that: For each of the plurality of images, a numerical value indicating a degree of failure is estimated based on a learning model that predetermines a relationship between the image and a pass / fail inspection result, and used as a pass / fail inspection result associated with each image. The reliability of the pass / fail inspection result related to each image is represented by the numerical value representing the degree of unqualifiedness. The closer the numerical value representing the degree of unqualifiedness is to the numerical value representing that the inspection object is completely qualified or the numerical value representing that the inspection object is completely unqualified, the higher the reliability of the pass / fail inspection result. The farther the numerical value representing the degree of unqualifiedness is from the numerical value representing that the inspection object is completely qualified or the numerical value representing that the inspection object is completely unqualified, the lower the reliability of the pass / fail inspection result. The inspection device includes a comprehensive determination unit that calculates a comprehensive pass / fail inspection result based on the reliability of each of the pass / fail inspection results related to the plurality of images.

2. The inspection device according to claim 1, characterized in that: The comprehensive determination unit outputs an instruction to re-photograph the image when the reliability of the comprehensive pass / fail inspection result does not satisfy a certain condition, and recalculates the comprehensive pass / fail inspection result using the re-photographed image.

3. The inspection device according to claim 1, characterized in that: When the reliability of the comprehensive pass / fail inspection result does not satisfy certain conditions, the comprehensive judgment unit outputs an instruction to re-shoot the image, and recalculates the comprehensive pass / fail inspection result using the re-shot image and the images shot until the last time.

4. The inspection device according to claim 2 or 3, characterized in that: The inspection device further includes a photographing condition determination unit, which determines photographing conditions of the image. The imaging condition determination unit changes the imaging condition for the re-imaging with respect to the imaging condition for the previous imaging.

5. The inspection device according to claim 4, characterized in that: The imaging condition determination unit increases the number of images in the re-imaging relative to the number of images in the previous imaging for the inspection portion whose reliability of the pass / fail inspection result does not satisfy a certain condition.

6. The inspection device according to claim 4, characterized in that: The imaging condition determination unit increases the number of images in the re-imaging relative to the number of images in the previous imaging for the inspection portion for which the pass / fail inspection result is unacceptable.

7. The inspection device according to claim 1, characterized in that: The inspection device includes a plurality of the learning models, and estimates the pass / fail inspection result based on the plurality of images using different learning models.

8. The inspection device according to claim 2 or 3, characterized in that: The inspection device includes a plurality of the learning models, and estimates the pass / fail inspection result for the image captured again using the learning model different from that for the image captured last time.

Images