Defect inspection apparatus and defect inspection method
By acquiring multiple images and analyzing features like luminance and gradient changes, the method improves defect inspection accuracy by distinguishing defects from dirt, addressing false detection issues in composite image-based methods.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- ASTEMO LTD
- Filing Date
- 2022-06-20
- Publication Date
- 2026-07-23
AI Technical Summary
Existing surface defect inspection methods face issues with false defect detection due to the creation of composite images, which compromises inspection accuracy.
The method involves acquiring multiple images at varying relative orientations, extracting features such as luminance, luminance gradient, geometric properties, and color intensity, and using waveform data analysis with machine learning to determine defects based on these features.
This approach enhances inspection accuracy by analyzing the patterns of feature changes to differentiate between defects and dirt, improving the reliability of defect detection.
Smart Images
Figure 0007894249000001 
Figure 0007894249000002 
Figure 0007894249000003
Abstract
Description
Technical Field
[0001] The present invention relates to a defect inspection apparatus and a defect inspection method.
Background Art
[0002] In Patent Document 1, while relatively moving a light and dark pattern by an illumination device with respect to a workpiece that is a detection target for surface defects, a plurality of images of the measurement site of the workpiece are acquired, and by performing binarization processing and applying a threshold value to each image, feature points of the image are extracted, a multidimensional feature amount is obtained for each feature point, and a provisional defect candidate is extracted. Among the plurality of images in which provisional defect candidates are extracted, if there are a preset threshold value or more of the images corresponding to the provisional defect candidates, the provisional defect candidates are determined as defect candidates, and a plurality of images including the determined defect candidates are combined to create a composite image, and a surface defect detection apparatus for a workpiece that performs defect detection based on the created composite image is disclosed.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] However, in the surface defect inspection apparatus of Patent Document 1, since a composite image is created by combining a plurality of images including the determined defect candidates, and defect detection is performed based on the created single composite image, there is a risk of false detection of defects. One object of the present invention is to provide a defect inspection apparatus and a defect inspection method with improved inspection determination accuracy.
Means for Solving the Problems
[0005] An embodiment of the defect inspection method includes an image acquisition step in which the defect inspection apparatus changes the relative orientation between the object to be inspected and the imaging unit, takes images for each relative orientation, and acquires multiple images, and scans each of the multiple images to obtain It is one of the following: luminance, luminance gradient, geometric properties, luminance variability, and color intensity. First feature and a second feature which is one of the following features other than the first feature: luminance, luminance gradient, geometric properties, luminance variability, and color intensity. The feature acquisition step obtains the first features, and the first waveform data is obtained by arranging the first features in the order of relative posture changes. The second waveform data, obtained by arranging the second features in order of relative attitude change, is then generated. The waveform data acquisition step and the first waveform data Using both waveform data of the first and second waveform data It includes a defect determination step for determining defects. [Effects of the Invention]
[0006] Therefore, in the present invention, It is one of the following: luminance, luminance gradient, geometric properties, luminance variability, and color intensity. First waveform data with the first feature arranged in order of relative attitude change. The second waveform data is obtained by arranging the second feature (one of the following features other than the first feature: luminance, luminance gradient, geometric properties, luminance variation, and color intensity) in order of change in relative orientation. By determining whether something is dirty or defective based on these factors, the accuracy of the inspection can be improved. [Brief explanation of the drawing]
[0007] [Figure 1] This is an overall view of the defect inspection apparatus of Embodiment 1. [Figure 2] This is a flowchart showing the flow of the defect inspection method in Embodiment 1. [Figure 3] This figure shows the characteristics of the luminance difference (first feature quantity) in Embodiment 1. [Figure 4] This is the first time chart illustrating the operation of the defect inspection method in Embodiment 1. [Figure 5] This is a second time chart illustrating the operation of the defect inspection method in Embodiment 1. [Figure 6] This figure shows the characteristics of the luminance gradient (first feature quantity) in Embodiment 2. [Figure 7] This is the first time chart illustrating the operation of the defect inspection method in Embodiment 2. [Figure 8] This is a second time chart illustrating the operation of the defect inspection method in Embodiment 2. [Figure 9]It is a diagram showing the characteristics of the luminance variation (the first feature amount) in Embodiment 3. [Figure 10] It is a time chart explaining the operation of the defect inspection method in Embodiment 3. [Figure 11] It is a time chart explaining the geometric characteristics (the first feature amount) and the operation of the defect inspection method in Embodiment 4. [Figure 12] It is a diagram showing the characteristics of the color intensity (the first feature amount) in Embodiment 5. [Figure 13] It is a time chart explaining the operation of the defect inspection method in Embodiment 5. [Figure 14] It is a flowchart showing the flow of the defect inspection method in Embodiment 6. [Figure 15] It is the first time chart explaining the operation of the defect inspection method in Embodiment 6. [Figure 16] It is the second time chart explaining the operation of the defect inspection method in Embodiment 6. [Figure 17] It is a flowchart showing the flow of the defect inspection method in Embodiment 7. [Figure 18] It is a time chart explaining the operation of the defect inspection method in Embodiment 7. [Figure 19] It is a flowchart showing the flow of the defect inspection method in Embodiment 8.
Modes for Carrying Out the Invention
[0008] 〔Embodiment 1〕 FIG. 1 is an overall view of the defect inspection apparatus of Embodiment 1.
[0009] [ The inspection apparatus 1 of Embodiment 1 includes a camera (imaging unit) 2, a robot 3, and a computer 4. The camera 2 captures an image of the crown surface 5a of the piston (test object) 5. Note that the crown surface 5a of the piston5 has a machined surface and a casting surface with significantly different surface properties. Robot 3 changes the relative orientation (angle) of piston 5 with respect to camera 2. Computer 4 is, for example, a personal computer and is equipped with a CPU 6. The CPU 6 includes an image acquisition unit 7, a feature acquisition unit 8 having a region extraction unit 8a, a waveform data acquisition unit 9, and a defect determination unit 10. The image acquisition unit 7 acquires multiple images of the crown surface 5a of the piston 5 captured by the camera 2 (image acquisition step). The region extraction unit 8a extracts and identifies candidate defect regions using machine learning or image processing (candidate defect region identification step). Machine learning, for example, involves learning using neural networks, and in Embodiment 1, a Convolutional Neural Network (CNN) is employed. The feature acquisition unit 8 acquires images of multiple defect candidate regions extracted and identified by the region extraction unit 8a using machine learning and image processing, scans the entire pixel region of each region, and acquires the brightness difference (first feature) (feature acquisition step). The luminance difference is the difference between the maximum and minimum luminance values obtained. The waveform data acquisition unit 9 acquires first waveform data by arranging the acquired first features in order of change in relative attitude (angle) (waveform data acquisition step). For angle numbers that have not been extracted or identified as potential defect areas, the brightness difference is set to 0. The defect detection unit 10 uses machine learning to determine defects from the shape of the first waveform data (defect detection step). This machine learning method, for example, is learning using a neural network, and in Embodiment 1, an RNN (Recurrent Neural Network) is employed.
[0010] Figure 2 is a flowchart showing the flow of the defect inspection method in Embodiment 1.
[0011] In step S1, the robot 3 changes the relative posture (angle) of the piston 5, and the image acquisition unit 7 acquires multiple images of the crown surface 5a of the piston 5 captured by the camera 2 (image acquisition step). Specifically, the total number of relative angles is 25. In step S2, the region extraction unit 8a extracts and identifies candidate defect regions using machine learning and image processing (candidate defect region identification step). In step S3, the feature acquisition unit 8 acquires images of multiple defect candidate regions extracted and identified by the region extraction unit 8a using machine learning and image processing, and scans the entire pixel region of each region to obtain the brightness difference (feature acquisition step). In step S4, the waveform data acquisition unit 9 acquires first waveform data in which the acquired brightness differences are arranged in order of change in relative attitude (angle) (waveform data acquisition step). For angle numbers that have not been extracted or identified as potential defect areas, the brightness difference is set to 0. In step S5, the defect detection unit 10 uses machine learning to determine a defect from the shape of the first waveform data (defect detection step).
[0012] Figure 3 shows the characteristics of the luminance difference (first feature quantity) in Embodiment 1. In other words, as shown in Figure 3(a), the characteristics of the brightness difference are that dirt has a light black color so the brightness difference is small, and defects have a dark black color so the brightness difference is large. Furthermore, as shown in Figure 3(b), the characteristics of the brightness difference are such that even with the same brightness difference, dirt occurs randomly, while defects occur continuously in bright-field imaging.
[0013] Figure 4 is a first time chart illustrating the operation of the defect inspection method in Embodiment 1. In other words, this shows the operation of the defect inspection method for the machined surfaces of piston 5.
[0014] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies the images with angle numbers 14 and 18-20 as defect candidate areas on the machined surface 1, and the images with angle numbers 14-17 as defect candidate areas on the machined surface 2. In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires images of angle numbers 14, 18-20 of the processed surface 1 and images of angle numbers 14-17 of the processed surface 2, which have been extracted and identified as defect candidate regions by the region extraction unit 8a. The unit scans the entire pixel region of each to acquire the brightness difference, and the waveform data acquisition unit 9 acquires first waveform data arranged in order of change in relative orientation (angle). In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired first waveform data of the brightness difference. In other words, the brightness difference a1 in the images of angle numbers 14 and 18-20 on processed surface 1 is small, so it is judged as OK (dirt), and the brightness difference a2 in the images of angle numbers 14-17 on processed surface 2 is large, so it is judged as NG (defect).
[0015] Figure 5 is a second time chart illustrating the operation of the defect inspection method in Embodiment 1. In other words, this illustrates the operation of the defect inspection method for the cast and machined surfaces of piston 5.
[0016] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies the images with angle numbers 10, 12, and 15 as defect candidate areas on the cast surface, and the images with angle numbers 14-17 as defect candidate areas on the machined surface. In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires images of the cast surface with angle numbers 10, 12, and 15 and the processed surface with angle numbers 14-17, which have been extracted and identified as defect candidate regions by the region extraction unit 8a, scans the entire pixel region of each to obtain the brightness difference, and the waveform data acquisition unit 9 acquires first waveform data arranged in order of change in relative orientation (angle). In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired first waveform data of brightness. In other words, there is no difference in brightness a3 between the images of the cast surface at angles 10, 12, and 15 and the images of the machined surface at angles 14-17. However, the brightness differences on the cast surface occur randomly, while the brightness differences on the machined surface occur continuously. Therefore, the images of the cast surface at angles 10, 12, and 15 are judged as OK (dirty), while the images of the machined surface at angles 14-17 are judged as NG (defective).
[0017] Next, the effects of Embodiment 1 will be explained. Embodiment 1 provides the following effects and benefits.
[0018] (1) The defect inspection device 1 includes an image acquisition unit 7 that changes the relative posture of the piston 5 and the camera 2 and takes images for each relative posture to acquire multiple images, a feature acquisition unit 8 that acquires the brightness difference as a first feature obtained by scanning the entire pixel area of each of the multiple images, a waveform data acquisition unit 9 that acquires first waveform data in which the brightness differences are arranged in the order of the change in relative posture, and a defect determination unit 10 that determines whether a defect exists according to the magnitude of the brightness difference based on the first waveform data. Therefore, the accuracy of the NG (defect) and OK (stain) inspections can be improved.
[0019] (2) The brightness difference is defined as the difference between the maximum and minimum brightness values obtained by scanning the entire pixel area of each of the multiple images. Therefore, the greater the difference between the maximum and minimum brightness values, the more abrupt the brightness change, which can further improve the accuracy of the inspection.
[0020] (3) The region extraction unit 8a of the feature acquisition unit 8 uses a machine learning method called CNN (Convolutinal Neural Network) to extract and identify candidate defect regions. Therefore, it becomes easier to identify defects.
[0021] (4) Defects are determined based on the magnitude of the brightness difference and the circumstances under which the brightness difference occurs (random or continuous). Therefore, the accuracy of the NG (defect) and OK (stain) inspections can be further improved.
[0022] [Embodiment 2] Figure 6 shows the characteristics of the luminance gradient (first feature quantity) in Embodiment 2. In other words, as shown in Figure 6(a), the characteristics of the luminance gradient are that dirt has a small maximum value of the luminance gradient, while defects have a large maximum value of the luminance gradient. Furthermore, the characteristics of the luminance gradient, similar to those of the luminance difference, are such that, as shown in Figure 6(b), dirt occurs randomly and defects occur continuously in bright-field imaging.
[0023] Figure 7 is a first time chart illustrating the operation of the defect inspection method in Embodiment 2. In other words, this shows the operation of the defect inspection method for the machined surfaces of piston 5.
[0024] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies the images with angle numbers 14 and 18-20 as defect candidate areas on the machined surface 1, and the images with angle numbers 21-24 as defect candidate areas on the machined surface 2. In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires images of angle numbers 14, 18-20 of the processed surface 1 and images of angle numbers 21-24 of the processed surface 2, which have been extracted and identified as defect candidate regions by the region extraction unit 8a. The unit scans the entire pixel region of each to acquire the brightness gradient, and the waveform data acquisition unit 9 acquires first waveform data arranged in order of change in relative orientation (angle). For angle numbers that have not been extracted or identified as potential defect regions, the luminance gradient is set to 0. In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired first waveform data of the brightness gradient. In other words, the brightness gradient of the images at angles 14 and 18-20 on processed surface 1 is small, so it is judged as OK (dirt), and the brightness gradient of the images at angles 21-24 on processed surface 2 is large, so it is judged as NG (defect).
[0025] Figure 8 is a second time chart illustrating the operation of the defect inspection method in Embodiment 2. In other words, this illustrates the operation of the defect inspection method for the cast and machined surfaces of piston 5.
[0026] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies images with angle numbers 14, 16, and 18 as defect candidate areas on the cast surface, and images with angle numbers 21-24 as defect candidate areas on the machined surface. In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires images of the cast surface with angle numbers 14, 16, and 18, and images of the processed surface with angle numbers 21-24, which have been extracted and identified as defect candidate regions by the region extraction unit 8a. The unit scans the entire pixel region of each to acquire the brightness gradient, and the waveform data acquisition unit 9 acquires first waveform data arranged in order of change in relative orientation (angle). For angle numbers that have not been extracted or identified as potential defect regions, the luminance gradient is set to 0. In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired first waveform data of brightness. In other words, the brightness gradients in the images of angle numbers 14, 16, and 18 on the cast surface are small and occur randomly, so they are judged as OK (dirt), while the brightness gradients in the images of angle numbers 21-24 on the machined surface are large and occur continuously, so they are judged as NG (defect). Thus, in Embodiment 1, the luminance difference was used as the first feature, but in Embodiment 2, the luminance gradient was used as the first feature. Since the other components are the same as those in Embodiment 1, the same reference numerals are used for the same components, and their descriptions are omitted.
[0027] Next, the effects of Embodiment 2 will be explained. Embodiment 2 provides the same effects and benefits as Embodiment 1.
[0028] (1) The defect inspection device 1 includes an image acquisition unit 7 that changes the relative posture of the piston 5 and the camera 2 and takes images for each relative posture to acquire multiple images, a feature acquisition unit 8 that acquires a brightness gradient as a first feature obtained by scanning the entire pixel area of each of the multiple images, a waveform data acquisition unit 9 that acquires first waveform data in which the brightness gradients are arranged in the order of the change in relative posture, and a defect determination unit 10 that determines whether a defect exists according to the magnitude of the brightness gradient obtained from the first waveform data. Therefore, the accuracy of the NG (defect) and OK (stain) inspections can be improved.
[0029] (2) The brightness gradient was calculated using the maximum value of the brightness gradient obtained by scanning multiple images individually. Therefore, although multiple brightness gradients can be obtained, using the maximum value can further improve the accuracy of the NG (defect) and OK (stain) inspections.
[0030] (3) The region extraction unit 8a of the feature acquisition unit 8 uses a machine learning method called CNN (Convolutinal Neural Network) to extract and identify candidate defect regions. Therefore, it becomes easier to identify defects.
[0031] (4) Defect detection is performed according to the magnitude of the brightness gradient and the circumstances under which the brightness gradient occurs (random or continuous). Therefore, the accuracy of the NG (defect) and OK (stain) inspections can be further improved.
[0032] [Embodiment 3] Figure 9 shows the characteristics of the luminance variation (first feature quantity) in Embodiment 3. In other words, as shown in Figure 9(a), the characteristics of the luminance variation (luminance distribution standard deviation) σ are that dirt has a large luminance variation (luminance distribution standard deviation) σ, while defects have a small luminance variation (luminance distribution standard deviation) σ.
[0033] Figure 10 is a time chart illustrating the operation of the defect inspection method in Embodiment 3. In other words, this illustrates the operation of the defect inspection method for the cast and machined surfaces of piston 5.
[0034] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies the images with angle numbers 14, 16, and 18 as defect candidate areas on the cast surface, and the images with angle numbers 22-24 as defect candidate areas on the machined surface. In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires images of the cast surface with angle numbers 14, 16, and 18, and images of the processed surface with angle numbers 22-24, which have been extracted and identified as defect candidate regions by the region extraction unit 8a. The unit scans the entire pixel region of each to acquire the luminance variation (luminance distribution standard deviation) σ, and the waveform data acquisition unit 9 acquires first waveform data arranged in order of change in relative orientation (angle). For angle numbers that have not been extracted or identified as potential defect areas, the luminance variation (luminance distribution standard deviation) is set to 0. In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired brightness waveform data. In other words, the variation in brightness (standard deviation of brightness distribution) σ of the images at angles 14, 16, and 18 of the cast surface is large, so it is judged as OK (dirty), while the variation in brightness (standard deviation of brightness distribution) σ of the images at angles 22-24 of the machined surface is small, so it is judged as NG (defective). Thus, in Embodiment 1, the luminance difference was used as the first feature, but in Embodiment 3, the luminance variation (luminance distribution standard deviation) σ was used as the first feature. Since the other components are the same as those in Embodiment 1, the same reference numerals are used for the same components, and their descriptions are omitted. Therefore, Embodiment 3 provides the same effects and advantages as Embodiment 1.
[0035] [Embodiment 4] Figure 11 is a time chart illustrating the geometric characteristics (first feature quantity) and the operation of the defect inspection method in Embodiment 4. In other words, this shows the operation of the defect inspection method for the machined surfaces of piston 5.
[0036] In terms of geometric characteristics, dirt has a small width and area, while defects have a large width and area. In the defect candidate area identification step, the area extraction unit 8a extracts and identifies images of defect candidate areas on the processed surface 1 with a predetermined brightness or lower, and images of defect candidate areas on the processed surface 2 with a predetermined brightness or lower (only those with a specific angle number are shown in the drawing). In the feature acquisition step and waveform data acquisition step, the feature acquisition unit 8 acquires multiple images extracted and identified by the area extraction unit 8a as defect candidate areas, scans the entire pixel area of each to acquire its geometric characteristics, and the waveform data acquisition unit 9 acquires first waveform data arranged in order of change in relative orientation (angle). For angle numbers that have not been extracted or identified as potential defect regions, the geometric properties are set to 0. In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired geometric characteristic waveform data. In other words, the width and area of the geometric characteristics of the image of processed surface 1 are large, so it is judged as NG (defective), and the width and area of the geometric characteristics of the image of processed surface 2 are small, so it is judged as OK (dirty). Thus, in Embodiment 1, luminance difference was used as the first feature quantity, but in Embodiment 4, geometric characteristics were used as the first feature quantity. Since the other components are the same as those in Embodiment 1, the same reference numerals are used for the same components, and their descriptions are omitted. Therefore, Embodiment 4 provides the same effects and advantages as Embodiment 1.
[0037] [Embodiment 5] Figure 12 shows the characteristics of color intensity (first feature quantity) in Embodiment 5. For example, if a defect has a large difference in brightness between G (green) and B (blue), and a stain has a large difference in brightness between G (green) and R (red), then it becomes possible to determine whether it is a defect or a stain based on the magnitude of the brightness difference between B (blue) and R (red).
[0038] Figure 13 is a time chart illustrating the operation of the defect inspection method in Embodiment 5. In other words, this shows the operation of the defect inspection method for the machined surface and cast surface of piston 5.
[0039] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies images of defect candidate areas on the machined surface and images of defect candidate areas on the cast surface (only those with specific angle numbers are shown in the drawing). In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires multiple images that the region extraction unit 8a has extracted and identified as defect candidate regions, scans the entire pixel region of each image to acquire the respective color intensity (luminance difference), and the waveform data acquisition unit 9 acquires first waveform data arranged in order of change in relative orientation (angle). For angle numbers that have not been extracted or identified as potential defect areas, the color intensity (luminance difference) is set to 0. In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired waveform data of intensity (luminance difference) for each color (RGB). In other words, if the color intensity (luminance difference) of the processed surface image is large for B (blue) and small for R (red), it is judged as NG (defective), while if the color intensity (luminance difference) of the cast surface image is small for B (blue) and large for R (red), it is judged as OK (dirty). Thus, in Embodiment 1, luminance difference was used as the first feature quantity, but in Embodiment 5, color intensity (luminance difference) is used as the first feature quantity. Since the other components are the same as those in Embodiment 1, the same reference numerals are used for the same components, and their descriptions are omitted. Therefore, Embodiment 5 provides the same effects and advantages as Embodiment 1.
[0040] [Embodiment 6] Figure 14 is a flowchart showing the flow of the defect inspection method in Embodiment 6.
[0041] In Embodiment 1, only the first feature was used, but in Embodiment 6, in addition to the first feature, a second feature is also used. In other words, in step S3a, the feature acquisition unit 8 acquires images of multiple defect candidate regions extracted and identified by the region extraction unit 8a, scans the entire pixel region of each region to acquire brightness difference (first feature) and brightness gradient (second feature). In step S4a, the waveform data acquisition unit 9 acquires first waveform data and second waveform data, which are obtained by arranging the brightness difference (first feature) and brightness gradient (second feature) in the order of change in relative orientation (angle). In step S5a, the defect determination unit 10 uses machine learning to determine defects from the shapes of the first waveform data and second waveform data. Since the other components are the same as those in Embodiment 1, the same reference numerals are used for the same components, and their descriptions are omitted.
[0042] Figure 15 is a first time chart illustrating the operation of the defect inspection method in Embodiment 6. In other words, this illustrates the operation of the defect inspection method for the cast and machined surfaces of piston 5.
[0043] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies the images with angle numbers 18, 20, and 23 as defect candidate areas on the cast surface, and the images with angle numbers 22-25 as defect candidate areas on the machined surface. In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires images of the cast surface with angle numbers 18, 20, and 23 and the processed surface with angle numbers 22-25, which have been extracted and identified as defect candidate regions by the region extraction unit 8a. It scans the entire pixel region of each to acquire the brightness difference and brightness gradient, and the waveform data acquisition unit 9 acquires first and second waveform data, which are obtained by arranging the brightness difference and brightness gradient, respectively, in order of change in relative orientation (angle). The drawing shows, as a representative example, the first and second waveform data for angle number 18 on the cast surface and angle number 22 on the machined surface. Furthermore, for angle numbers that have not been extracted or identified as potential defect regions, the luminance difference and luminance gradient are set to 0. In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired first waveform data of brightness. In other words, the brightness gradient b1 of the image at angle number 18 of the cast surface and the brightness gradient b1 of the image at angle number 22 of the machined surface are large, but the brightness difference a1 of the cast surface is small, while the brightness difference a2 of the machined surface is large. Therefore, the image of angle number 18 on the cast surface is judged as OK (dirty), and the image of angle number 22 on the machined surface is judged as NG (defective). The same process is applied to images with other extracted and acquired angle numbers.
[0044] Figure 16 is a second time chart illustrating the operation of the defect inspection method in Embodiment 6. In other words, this illustrates the operation of the defect inspection method for the cast and machined surfaces of piston 5.
[0045] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies the images with angle numbers 18, 20, and 23 as defect candidate areas on the cast surface, and the images with angle numbers 22-25 as defect candidate areas on the machined surface. In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires images of the cast surface with angle numbers 18, 20, and 23 and the processed surface with angle numbers 22-25, which have been extracted and identified as defect candidate regions by the region extraction unit 8a. The unit scans the entire pixel region of each to acquire the brightness difference and brightness gradient, and the waveform data acquisition unit 9 acquires first waveform data and second waveform data, respectively, by arranging the brightness difference and brightness gradient in order of change in relative orientation (angle). The drawing shows, as a representative example, the first and second waveform data for angle number 18 on the cast surface and angle number 22 on the machined surface. Furthermore, for angle numbers that have not been extracted or identified as potential defect regions, the luminance difference and luminance gradient are set to 0. In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired first and second waveform data of brightness. In other words, the brightness gradient b1 of the image at angle number 18 of the cast surface is small, the brightness gradient b2 of the image at angle number 22 of the machined surface is large, and the brightness difference a1 between the cast surface and the machined surface is large. Therefore, the image of the cast surface at angle number 18 is judged as OK (dirty) because the brightness difference a1 is large but the brightness gradient b1 is small, while the image of the machined surface at angle number 22 is judged as NG (defective) because both the brightness difference a1 and the brightness gradient b2 are large. The same process is applied to images with other extracted and acquired angle numbers.
[0046] Next, the effects of Embodiment 6 will be explained. In addition to the effects of Embodiment 1, the following effects are also achieved.
[0047] (1) As features, we used both brightness difference (first feature) and brightness gradient (second feature). Therefore, increasing the amount of information further improves the accuracy of the judgment.
[0048] [Embodiment 7] Figure 17 is a flowchart showing the flow of the defect inspection method in Embodiment 7.
[0049] In Embodiment 6, the first and second features were used, but in Embodiment 7, in addition to the first and second features, a third feature is used. Specifically, in step S3b, the feature acquisition unit 8 acquires images of multiple defect candidate regions extracted and identified by the region extraction unit 8a, scans the entire pixel region of each region to acquire brightness difference (first feature), brightness gradient (second feature), and brightness variation (third feature). In step S4b, the waveform data acquisition unit 9 acquires first waveform data, second waveform data, and third waveform data, which are obtained by arranging the brightness difference (first feature), brightness gradient (second feature), and brightness variation (third feature) in the order of changes in relative orientation (angle). In step S5b, the defect determination unit 10 uses machine learning to determine defects from the shapes of the first waveform data, second waveform data, and third waveform data. Since the other components are the same as those in Embodiment 6, the same reference numerals are used for the same components, and their descriptions are omitted.
[0050] Figure 18 is a time chart illustrating the operation of the defect inspection method in Embodiment 7. In other words, this illustrates the operation of the defect inspection method for the cast and machined surfaces of piston 5.
[0051] In the defect candidate area identification step, the area extraction unit 8a extracts and identifies images with angle numbers 14, 17, 18, and 20 as defect candidate areas on the cast surface, and images with angle numbers 19, 22, 23, and 25 as defect candidate areas on the machined surface. In the feature acquisition step and the waveform data acquisition step, the feature acquisition unit 8 acquires images of the cast surface with angle numbers 14, 17, 18, and 20, and the processed surface with angle numbers 19, 22, 23, and 25, which have been extracted and identified as defect candidate regions by the region extraction unit 8a. The unit scans the entire pixel region of each to acquire brightness difference, brightness gradient, and brightness variation. The waveform data acquisition unit 9 then acquires first waveform data, second waveform data, and third waveform data, which are obtained by arranging the brightness difference, brightness gradient, and brightness variation in order of change in relative orientation (angle). For angle numbers that have not been extracted or identified as potential defect regions, the brightness difference, brightness gradient, and brightness variation are set to 0. In the defect detection step, the defect detection unit 10 uses an RNN (Recurrent Neural Network) to perform defect detection on the acquired first waveform data, second waveform data, and third waveform data of brightness. In other words, the cast surface has a large brightness gradient and brightness difference, but the brightness variation is also large, so it is judged as OK (dirty), while the machined surface has a large brightness gradient and brightness difference, but the brightness variation is small, so it is judged as NG (defective).
[0052] Next, the effects of Embodiment 7 will be explained. In addition to the effects of Embodiment 6, the following effects are also achieved.
[0053] (1) As features, we used a combination of luminance difference (first feature), luminance gradient (second feature), and luminance variability (third feature). Therefore, increasing the amount of information further improves the accuracy of the judgment.
[0054] [Embodiment 8] Figure 19 is a flowchart showing the flow of the defect inspection method in Embodiment 8.
[0055] In Embodiment 1, the region extraction unit 8a used a Convolutional Neural Network (CNN) to extract and identify candidate defect regions and obtain brightness differences (first features). In Embodiment 8, however, the entire pixel region of each image from multiple angles is scanned to obtain brightness differences (first features). Since the other components are the same as those in Embodiment 1, the same reference numerals are used for the same components, and their descriptions are omitted.
[0056] Next, the effects of Embodiment 8 will be explained. In addition to the effects of Embodiment 1, the following effects are also achieved.
[0057] (1) The region extraction unit 8a that extracts and identifies candidate defect regions of computer 4 does not use a CNN (Convolutinal Neural Network). Therefore, the cost of computer 4 can be reduced.
[0058] [Other embodiments] Although embodiments for carrying out the present invention have been described above, the specific configuration of the present invention is not limited to the configuration of the embodiments, and design changes and the like that do not depart from the gist of the invention are also included in the present invention. For example, the object being inspected is not limited to a piston, and machine learning is not limited to CNN (Convolutinal Neural Network) or RNN (Recurrent Neural Network). Furthermore, while the attitude control unit currently changes the piston's attitude using a robot, this can also be done by changing the camera's position. [Explanation of symbols]
[0059] 1. Defect inspection device, 2. Camera (imaging unit), 3. Robot, 4. Computer, 5. Piston (object under inspection), 6. CPU, 7. Image acquisition unit, 8. Feature acquisition unit, 8a. Region extraction unit, 9. Waveform data acquisition unit, 10. Defect determination unit
Claims
1. A defect inspection device for determining defects in an object under inspection, An image acquisition unit that changes the relative orientation between the object to be inspected and the imaging unit, takes images for each relative orientation, and acquires multiple images, A feature acquisition unit that scans each of the aforementioned plurality of images and acquires specific features as a first feature quantity, which is one of the luminance, luminance gradient, geometric characteristics, luminance variation, and color intensity obtained by scanning the plurality of images, and a second feature quantity, which is one of the luminance, luminance gradient, geometric characteristics, luminance variation, and color intensity other than the first feature quantity. A waveform data acquisition unit acquires first waveform data obtained by arranging the first feature quantities in the order of the change in relative posture, and acquires second waveform data obtained by arranging the second feature quantities in the order of the change in relative posture, A defect determination unit that determines defects using waveform data from both the first waveform data and the second waveform data, A defect inspection device characterized by being equipped with the following features.
2. In the defect inspection apparatus according to claim 1, The feature acquisition unit acquires one of the following as a third feature, excluding the first and second features: luminance, luminance gradient, geometric characteristics, luminance variation, and color intensity. The waveform data acquisition unit acquires third waveform data in which the third feature quantities are arranged in the order of the change in relative attitude, The defect determination unit determines whether a defect exists using the three waveform data: the first waveform data, the second waveform data, and the third waveform data. A defect inspection device characterized by the following features.
3. In the defect inspection apparatus according to claim 1, The first feature quantity is a brightness difference, and the difference between the maximum and minimum brightness values obtained by scanning the image is used. A defect inspection device characterized by the following features.
4. In the defect inspection apparatus according to claim 1, The first feature is a luminance gradient, and the maximum value of the luminance gradient obtained by scanning the image is used. A defect inspection device characterized by the following features.
5. In the defect inspection apparatus according to claim 1, The first feature is the luminance difference, and the second feature is the luminance gradient. A defect inspection device characterized by the following features.
6. In the defect inspection apparatus according to claim 2, The first feature is the luminance difference, the second feature is the luminance gradient, and the third feature is the luminance variation. A defect inspection device characterized by the following features.
7. In the defect inspection apparatus according to claim 1, The feature acquisition unit includes a region extraction unit that uses machine learning to extract potential defect candidate regions from each of the multiple images. The first feature is a specific feature obtained by scanning the defect candidate region. A defect inspection device characterized by the following features.
8. In the defect inspection apparatus according to claim 1, The first feature is a specific feature obtained by scanning the entire pixel region of each of the multiple images. A defect inspection device characterized by the following features.
9. In the defect inspection apparatus according to claim 1, The first feature quantity is a geometric characteristic of a pixel region whose brightness is below a predetermined level, obtained by scanning the image. A defect inspection device characterized by the following features.
10. A defect inspection method in which a defect inspection device performs defect determination on an object to be inspected, The aforementioned defect inspection device Image acquisition step: Change the relative orientation of the object to be inspected and the imaging unit, perform imaging for each relative orientation, and acquire multiple images. A feature acquisition step in which specific features obtained by scanning each of the aforementioned plurality of images are acquired as a first feature quantity, which is one of the luminance, luminance gradient, geometric characteristics, luminance variation, and color intensity obtained by scanning the plurality of images, and a second feature quantity, which is one of the luminance, luminance gradient, geometric characteristics, luminance variation, and color intensity other than the first feature quantity. Waveform data acquisition step: Acquire first waveform data by arranging the first feature quantities in the order of the change in relative posture, and acquire second waveform data by arranging the second feature quantities in the order of the change in relative posture, A defect determination step in which defects are determined using waveform data of both the first waveform data and the second waveform data, A defect inspection method characterized by having the following features.