Inspection system, inspection device, inspection program, and inspection method

Abstract

To increase precision for inspecting an inspection object.SOLUTION: An inspection system 1 includes: a learning device 100 for learning a detector for detecting a detection object part of a tank 2 as an inspection object; and an inspection device 200 for detecting a detection object part of the tank 2 by using a detector learned by the learning device 100. The learning device 100 includes: an image acquisition part for learning that acquires a captured image of an inspection object including a detection object part; a learning data generation part for generating learning data from an image of a detection object part included in a captured image of an inspection object acquired by the image acquisition part for learning; and a learning part for learning the detector by using learning data generated by the learning data generation part. The learning data generation part divides at least one portion of images of plural detection object parts into plural portions and generates learning data from each of the divided images of the detection object part.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a technique for inspecting an inspection object, and in particular to an inspection system for inspecting an inspection object, a learning device, a learning program, and a learning method usable for the inspection system, an inspection device, an inspection program, and an inspection method usable for the inspection system. [Background technology]

[0002] When manufacturing sanitary ceramics such as toilet tanks, defects such as peeling of the glaze and cracks can occur during processes such as glazing and firing. Traditionally, skilled inspectors have visually inspected the appearance, but securing sufficient personnel is difficult and the training burden is high, so eliminating dependency on individual personnel has been a major issue.

[0003] In order to automatically detect such defects, a technique has been proposed for determining the surface condition from a captured image of an object to be inspected (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-047098 A Summary of the Invention [Problem to be solved by the invention]

[0005] If defects cannot be detected and products are shipped with defects, it will result in the hassle of having to recall the product, ship a replacement, or reprocess it, and it will also reduce the reliability of the product. Therefore, it is necessary to further improve the accuracy of automatically detecting defects.

[0006] The present disclosure has been made in consideration of such problems, and has an object to improve the accuracy of inspecting an inspection object. [Means for solving the problem]

[0007] In order to solve the above problems, an inspection system according to an embodiment of the present disclosure includes a learning device that learns a detector for detecting a detection target portion of an inspection target, and an inspection device that detects the detection target portion of the inspection target using the detector learned by the learning device. The learning device includes a learning image acquisition unit that acquires captured images of the inspection target including the detection target portion, a learning data generation unit that generates learning data from images of the detection target portion included in the captured images of the inspection target acquired by the learning image acquisition unit, and a learning unit that trains the detector using the learning data generated by the learning data generation unit. The learning data generation unit divides at least a portion of the images of the detection target portions into multiple parts and generates learning data from each of the divided images of the detection target portions, and the inspection device includes an inspection image acquisition unit that acquires the captured images of the inspection target, and a detection unit that uses a detector to detect the detection target portion from the captured images acquired by the inspection image acquisition unit.

[0008] Another aspect of the present disclosure is a learning device. The device includes a learning image acquisition unit that acquires captured images of an inspection target including a detection target portion, a learning data generation unit that generates learning data from images of the detection target portion included in the captured images of the inspection target acquired by the learning image acquisition unit, and a learning unit that uses the learning data generated by the learning data generation unit to train a detector for detecting the detection target portion of the inspection target. The learning data generation unit divides at least a portion of the images of the detection target portions into multiple parts, and generates learning data from each of the divided images of the detection target portions.

[0009] Yet another aspect of the present disclosure is a learning program that causes a computer to function as a learning image acquisition unit that acquires captured images of an inspection target including a detection target portion, a learning data generation unit that generates learning data from images of the detection target portion included in the captured images of the inspection target acquired by the learning image acquisition unit, and a learning unit that uses the learning data generated by the learning data generation unit to train a detector for detecting the detection target portion of the inspection target, wherein the learning data generation unit divides at least a portion of the images of the detection target portions into a plurality of parts and generates learning data from each of the divided images of the detection target portions.

[0010] Yet another aspect of the present disclosure is a learning method. This method causes a computer to execute the steps of acquiring an image of an inspection target including a detection target portion, generating learning data from an image of the detection target portion included in the acquired image of the inspection target, and training a detector for detecting the detection target portion of the inspection target using the generated learning data. In the learning step, at least a portion of the images of the detection target portions are divided into a plurality of parts, and learning data is generated from each of the divided images of the detection target portions.

[0011] Yet another aspect of the present disclosure is an inspection device including: an inspection image acquisition unit that acquires an image of an inspection target; and a detection unit that detects the detection target portion from the image acquired by the inspection image acquisition unit using a detector trained using training data generated by dividing an image of the detection target portion included in the image of the inspection target into multiple parts.

[0012] Yet another aspect of the present disclosure is an inspection program that causes a computer to function as an inspection image acquisition unit that acquires an image of an inspection target, and a detection unit that detects the detection target portion from the image acquired by the inspection image acquisition unit, using a detector trained using training data generated by dividing an image of the detection target portion included in the image of the inspection target into multiple parts.

[0013] Yet another aspect of the present disclosure is an inspection method, which causes a computer to execute the steps of acquiring an image of an inspection target, and detecting the detection target part from the acquired image using a detector trained using training data generated by dividing an image of the detection target part included in the image of the inspection target into a plurality of parts.

[0014] Any combination of the above components, and any transformation of the present invention into a method, device, system, recording medium, computer program, etc., are also effective as aspects of the present invention. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing a configuration of an inspection system according to an embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of a captured image of the surface of a tank. [Diagram 3] FIG. 13 is a diagram showing an example of a captured image of the surface of a tank. [Figure 4] 4 is a flowchart showing a procedure of a learning method according to an embodiment. [Diagram 5] 4 is a flowchart showing a procedure of an inspection method according to an embodiment. [Figure 6] FIG. 1 is a diagram illustrating a configuration of a learning device according to an embodiment. [Figure 7] FIG. 1 is a diagram showing a configuration of an inspection device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] As an embodiment of the present disclosure, a technique for inspecting the appearance of a tank, which is an example of sanitary ware, will be described. A learning device according to the embodiment learns a defect detector for detecting defects such as cracks that occur on the surface of the tank. An inspection device according to the embodiment detects defects from a captured image of the tank using the defect detector learned by the learning device. This improves the efficiency and accuracy of the inspection, thereby significantly reducing the burden on the inspector. In addition, the rate at which a passing product is judged to be a failing product can be reduced, thereby reducing the effort required to review the failing products. In addition, accurate inspection can be performed regardless of the experience or skill of the inspector, thereby eliminating the dependency on individual inspections, reducing the shortage of inspectors, and stabilizing the quality of products.

[0017] 1 shows the configuration of an inspection system according to an embodiment. The inspection system 1 includes a tank 2 to be inspected, an imaging device 3 for imaging the exterior of the tank 2, a learning device 100 for learning a defect detector for detecting defects occurring on the surface of the tank 2, an inspection device 200 for inspecting the surface of the tank 2 using the defect detector, and the Internet 4, which is an example of a communication network connecting these devices.

[0018] Fig. 2 shows an example of a captured image of the surface of the tank 2. Fig. 2(a) shows an image of a crack that has occurred on the surface of the tank 2. The captured image 10 includes images of four cracks 11a, 11b, 11c, and 11d. The learning device 100 assigns annotations (teacher labels) indicating that the cracks included in the captured image 10 are cracks, and trains a defect detector by supervised learning.

[0019] In order to improve the generalization performance (detection performance for unknown data) of the defect detector, it is necessary to train the defect detector using images of a wide variety of defects as training data. However, since the rate at which defects occur on the surface of the tank 2 during the manufacture of the tank 2 is very low, it is difficult to collect a large number of images of defects actually occurring on the surface of the tank 2 to generate training data. In order to solve this problem, in this embodiment, an image of one defect is divided into multiple parts, and training data is generated from each of the images of the divided defects. This makes it possible to efficiently generate a large amount of training data and train the defect detector, thereby improving the generalization performance and accuracy of the defect detector. When detecting defects using a defect detector trained in this way, even if there is one defect, there is a possibility that each of the divided defects will be detected as multiple pieces, but after detection, adjacent defects can be grouped together. This method is particularly effective when training a detector for defects such as cracks, where the shape and other characteristics of a single defect may differ depending on the position.

[0020] As shown in FIG. 2(a), the learning device 100 may add annotations 12a, 12b, 12c, and 12d to the images of the four cracks 11a, 11b, 11c, and 11d, respectively. In this embodiment, as shown in FIG. 2(b), three images of 11a, 11b, and 11c among the images of the four cracks 11a, 11b, 11c, and 11d are divided into a plurality of images, and annotations are added to each of the divided images of the plurality of cracks to generate separate learning data. For example, the image of the crack 11a is divided into three, and annotations 12a1, 12a2, and 12a3 are added to each of the images to generate three learning data. This makes it possible to increase the number of learning data to which annotations are added, thereby improving the generalization performance and accuracy of the defect detector.

[0021] The learning device 100 may receive an annotation specification from a person in charge, or may automatically or semi-automatically assign the annotation. The learning device 100 may receive a specification of the position, size, and type of the defect included in the captured image 10 from the person in charge, and may determine whether to generate learning data after dividing the image of the defect or to generate learning data without dividing it, and if dividing, the number of divisions and the size and shape of each divided area. For example, the crack 11d shown in FIG. 2(a) has a monotonous shape regardless of the position, so the annotation 12d may be assigned to the entire crack without dividing it. The cracks 11a, 11b, and 11c have a variety of shapes, so they may be divided into a number of parts according to the size and shape of the crack, and annotations may be assigned to each part.

[0022] FIG. 3 shows an example of a captured image of the surface of the tank 2. FIG. 3(a) shows an image of a crack that has occurred on the surface of the tank 2. The captured image of the surface of the tank 2 may contain brightness contrasts due to defects such as cracks, as well as brightness contrasts due to unevenness on the surface of the tank 2 and ambient light at the time of capture. If the defect detector erroneously learns such features of the background image that should not be detected, the detection accuracy may decrease. In order to solve such problems, in this embodiment, image processing is performed on the captured image before generating learning data from the captured image of the inspection target. This preprocessing may be image processing that can smooth the brightness by reducing the contrast of the image of a portion that is not a detection target portion such as a defect. For example, a blur filter may be applied to the entire image. The blur filter may be, for example, a box filter that takes the average of pixel values ​​within the range of the kernel, a Gaussian filter that changes the weight depending on the distance from the pixel of interest, a median filter that adopts the median value of all pixels within the range of the kernel, a bilateral filter that is a Gaussian filter weighted by normal distribution, or the like. The preprocessing may be image processing that reduces the resolution of the image. This makes it possible to improve the learning efficiency through simple image processing, so that a highly accurate defect detector can be generated in a short period of time.

[0023] Figure 3(b) shows the image shown in Figure 3(a) after applying a blur filter. The background is blurred and the contrast is reduced, which reduces the possibility of mislearning the background features. The contrast of the defects is also slightly reduced, but the features are still preserved, so the defect detector can learn the defect features.

[0024] The image processing may be performed under conditions where the contrast characteristics of the detection target portion are not lost and the contrast of the portion not being the detection target portion is sufficiently smoothed. The learning device 100 may obtain the luminance distribution of the detection target portion and the luminance distribution of the background not being the detection target portion, and may determine the conditions such as the type of image processing and the kernel size based on each of the luminance distributions. When inspecting a plurality of captured images of the same type of inspection target captured in a similar environment, the learning device 100 may perform image processing under the same conditions.

[0025] When the surface of the tank 2 to be inspected is imaged by the imaging device 3, the imaging may be performed in an imaging environment in which the contrast of non-defective areas is sufficiently low. For example, the tank 2 may be irradiated with light at an illuminance according to the color, reflectance, etc. of the surface of the tank 2. In addition, the focal length, resolution, aperture, etc. of the imaging device 3 may be adjusted.

[0026] FIG. 4 is a flowchart showing a procedure of a learning method according to an embodiment. The learning device 100 acquires a captured image of an inspection target having a defect (S10), and performs preprocessing such as blurring on the acquired captured image (S12). The learning device 100 divides at least a part of an image of a defect included in the captured image into a plurality of images (S14), and generates learning data by adding annotations to each of the divided plurality of defect images (S16). The learning device 100 may add different annotations for each type of defect. This allows the defect detector to learn features for each type of defect, so that the defect detector can be used to detect defects from the captured image of the inspection target and determine the type of defect at the same time. The learning device 100 uses the generated learning data to train the defect detector through supervised learning. The learning device 100 may apply any known learning algorithm when training the defect detector.

[0027] 5 is a flowchart showing the procedure of the inspection method according to the embodiment. The inspection device 200 acquires a captured image of an inspection target (S30), and performs image processing on the captured image similar to that performed by the learning device 100 when generating the learning data (S32). The inspection device 200 detects defects in the captured image using a trained defect detector (S34), and outputs the detection result (S36).

[0028] The inventor trained the defect detector using the captured images of 212 defective products of the tank 2 by the above method, and inspected 500 passing products and 23 failing products of the tank 2 using the trained defect detector. The number of cases where a failing product was erroneously determined to be a passing product was 0. This shows that the technology of this embodiment can minimize the possibility of erroneously shipping a failing product. In addition, the number of cases where a passing product was erroneously determined to be a failing product was 7, and the accuracy rate was 98.6%. This shows that the labor of inspectors checking failing products can be significantly reduced, and the rate of passing products that cannot be shipped and are lost can be significantly reduced.

[0029] 6 shows the configuration of learning device 100 according to an embodiment. Learning device 100 includes display device 112, input device 113, communication device 114, processing device 120, and storage device 130. Learning device 100 may be a server device, a device such as a personal computer, or a portable terminal such as a mobile phone terminal, a smartphone, or a tablet terminal.

[0030] Display device 112 displays a screen generated by processing device 120. Display device 112 may be a liquid crystal display device, an organic EL display device, or the like. Input device 113 transmits instructions input by a user of study device 100 to processing device 120. Input device 113 may be a mouse, a keyboard, a touch pad, or the like. Display device 112 and input device 113 may be implemented as a touch panel.

[0031] The communication device 114 controls communication with other devices. The communication device 114 may perform communication by any communication method such as wired or wireless. The communication device 114 performs communication with the imaging device 3 and the inspection device 200 via the Internet 4.

[0032] The storage device 130 stores programs, data, etc. used by the processing device 120. The storage device 130 may be a semiconductor memory, a hard disk, etc. The storage device 130 stores a learning image storage unit 131, a learning data storage unit 132, and a defect detector 133.

[0033] The processing device 120 includes a learning image acquisition unit 121, an image processing unit 122, a learning data generation unit 123, a learning unit 124, and a defect detector provision unit 125. These components are realized in terms of hardware by the CPU, memory, and other LSIs of any computer, and in terms of software by a program loaded into memory, but here, functional blocks realized by the cooperation of these are depicted. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms, such as hardware alone or a combination of hardware and software.

[0034] The learning image acquisition unit 121 acquires captured images of an inspection target having defects from the imaging device 3 and stores them in the learning image storage unit 131. The image processing unit 122 performs image processing such as blurring on the learning images stored in the learning image storage unit 131. The learning data generation unit 123 adds annotations to defects included in the learning images. The learning data generation unit 123 divides at least a part of the defect images into a plurality of images and adds annotations to each of the divided defect images. The learning data generation unit 123 may display the learning images on the display device 112 and accept designation of annotations from a person in charge via the input device 113. The learning data generation unit 123 may automatically add annotations to defects. The learning data generation unit 123 generates learning data and stores it in the learning data storage unit 132.

[0035] The learning unit 124 learns the defect detector 133 by using the learning data stored in the learning data holding unit 132. The defect detector providing unit 125 provides the inspection device 200 with the defect detector 133 that has been trained.

[0036] 7 shows a configuration of an inspection device 200 according to an embodiment. The inspection device 200 includes a display device 212, an input device 213, a communication device 214, a processing device 220, and a storage device 230. The inspection device 200 may be a server device, a device such as a personal computer, or a portable terminal such as a mobile phone terminal, a smartphone, or a tablet terminal.

[0037] The display device 212 displays a screen generated by the processing device 220. The display device 212 may be a liquid crystal display device, an organic EL display device, or the like. The input device 213 transmits an instruction input by a user of the inspection device 200 to the processing device 220. The input device 213 may be a mouse, a keyboard, a touch pad, or the like. The display device 212 and the input device 213 may be implemented as a touch panel.

[0038] The communication device 214 controls communication with other devices. The communication device 214 may perform communication by any communication method, such as wired or wireless. The communication device 214 performs communication with the imaging device 3 and the learning device 100 via the Internet 4.

[0039] The storage device 230 stores programs, data, etc. used by the processing device 220. The storage device 230 may be a semiconductor memory, a hard disk, etc. The storage device 230 stores an inspection image storage unit 231, a detection result storage unit 232, and a defect detector 233.

[0040] The processing device 220 includes an inspection image acquisition unit 221, an image processing unit 222, a defect detection unit 223, a detection result generation unit 224, and a detection result output unit 225. These configurations can also be realized in various forms, such as hardware only or a combination of hardware and software.

[0041] The test image acquisition unit 221 acquires an image of the test object captured by the imaging device 3 from the imaging device 3 and stores the image in the test image storage unit 231. The image processing unit 222 performs the same image processing on the test image as the learning device 100 performed on the learning image.

[0042] The defect detection unit 223 detects defects from the inspection images stored in the inspection image holding unit 231 using the trained defect detector 233 acquired from the learning device 100, and stores the detection results in the detection result holding unit 232. The detection result generation unit 224 generates information about the detected defects. When multiple defects detected by the defect detection unit 223 are close to each other, the detection result generation unit 224 groups the defects into one defect. The detection result generation unit 224 generates information about the position, size, number, type, etc. of the detected defects. The detection result output unit 225 outputs the detection results generated by the detection result generation unit 224 to the display device 212, etc.

[0043] Although the present invention has been described above based on the embodiments, the embodiments merely show the principles and applications of the present invention. Furthermore, the embodiments can be modified in many ways and with different arrangements without departing from the spirit of the present invention as defined in the claims.

[0044] In the above embodiment, the technology for detecting cracks occurring on the surface of sanitary ware has been mainly described, but the technology of the present embodiment can also be applied to the case of detecting other types of defects occurring on the surface of sanitary ware, such as adhesion of iron, copper, base material, foreign matter, etc., abnormalities in glaze or color, and the occurrence of air bubbles, cracks, chips, etc. Furthermore, any product other than sanitary ware may be the inspection target, and any part to be detected other than defects may be the detection target. [Explanation of symbols]

[0045] 1 inspection system, 2 tank, 3 imaging device, 4 Internet, 10 captured image, 11 crack, 12 annotation, 100 learning device, 121 learning image acquisition unit, 122 image processing unit, 123 learning data generation unit, 124 learning unit, 125 defect detector provision unit, 131 learning image storage unit, 132 learning data storage unit, 133 defect detector, 200 inspection device, 221 inspection image acquisition unit, 222 image processing unit, 223 defect detection unit, 224 detection result generation unit, 225 detection result output unit, 231 inspection image storage unit, 232 detection result storage unit, 233 defect detector.

Claims

1. a learning device that learns a detector for detecting defects occurring on the surface of the sanitary ware and determining the type of the defect; an inspection device that detects defects occurring on a surface of the sanitary ware using the detector trained by the learning device; Equipped with The learning device includes: a learning image acquisition unit that acquires an image of the sanitary ware including a defect; a learning data generating unit that generates learning data from images of defects included in the captured images of the sanitary ware acquired by the learning image acquiring unit; a learning unit that learns the detector using the learning data generated by the learning data generation unit; Equipped with the learning data generation unit divides at least a portion of the images of the plurality of defects into a plurality of images, and generates learning data from each of the divided images of the defects; The inspection device includes: an inspection image acquisition unit for acquiring an image of the sanitary ware; a detection unit that detects defects from the captured image acquired by the inspection image acquisition unit using the detector and determines a type of the defect; Equipped with When the detected defects are adjacent to each other, the detection unit detects the defects as one defect. Inspection system.

2. 2. The inspection system according to claim 1, wherein the training data generation unit generates the training data by attaching a teacher label to an image of a defect included in the captured image of the sanitary ware, and when the image of the defect is divided into a plurality of parts, generates the training data by attaching a teacher label to each of the plurality of divided images of the defect.

3. 3. The inspection system according to claim 1, further comprising an image processing unit that performs image processing on the captured image of the sanitary ware acquired by the learning image acquisition unit before generating learning data from the captured image.

4. The inspection system according to claim 3 , wherein the image processing is a process for reducing the resolution or contrast of pixel values ​​in the portion that is not the defect.

5. The inspection system according to claim 4 , wherein the image processing is a process of applying a blurring filter to the entire captured image.

6. The inspection system according to claim 1 , wherein the defect is a crack.

7. an inspection image acquisition unit for acquiring an image of the sanitary ware; a detection unit that detects defects from the captured images acquired by the inspection image acquisition unit using a detector that has been trained using learning data generated by dividing an image of a defect included in a captured image of the sanitary ware into a plurality of parts, and determines a type of the defect; Equipped with When the detected defects are adjacent to each other, the detection unit detects the defects as one defect. Inspection equipment.

8. 8. The inspection device according to claim 7, further comprising an image processing unit that performs image processing on the captured image of the sanitary ware acquired by the inspection image acquisition unit before detecting defects in the captured image.

9. 9. The inspection apparatus according to claim 8, wherein the image processing is a process for reducing the resolution or contrast of pixel values ​​in the portion that is not the defect.

10. 10. The inspection device according to claim 9, wherein the image processing is a process of applying a blurring filter to the entire captured image.

11. 11. The inspection device according to claim 7, wherein the defect is a crack.

12. Computer, an inspection image acquisition unit for acquiring an image of the sanitary ware; a detection unit that detects defects from the captured images acquired by the inspection image acquisition unit using a detector that has been trained using learning data generated by dividing an image of a defect included in a captured image of the sanitary ware into a plurality of parts, and determines a type of the defect; Functioning as a When the detected defects are adjacent to each other, the detection unit detects the defects as one defect. Inspection program.

13. On the computer, acquiring an image of the sanitary ware; a step of detecting defects from the captured image of the sanitary ware using a detector trained using training data generated after dividing an image of a defect included in the captured image of the sanitary ware into a plurality of parts, and determining the type of the defect; Run the command, In the detecting step, when the detected defects are adjacent to each other, the defects are collectively detected as one defect. Testing method.

Images