Imaging inspection system, imaging inspection method, and computer program

Abstract

To provide an image inspection system, an image inspection method, and a computer program that improve the accuracy of image inspection of castings.SOLUTION: In an image inspection system 10, a control device 50 includes an image acquisition unit that acquires an inspection image obtained by imaging the casting that is the object to be inspected OB, and a plurality of castings that have been determined to be non-defective in advance. A storage unit that stores a feature amount extracted from each of the plurality of non-defective images obtained by using the first convolutional neural network that has been trained, and extracts the feature amount from an inspection image using a first convolutional neural network, using the feature amount extracted from the inspection image and the feature amount extracted from each of a plurality of non-defective product images, a degree of abnormality of the inspection image for the plurality of non-defective product images is calculated, and the casting of an inspection target is performed based on a degree of abnormality of the inspection image, and a first determination unit that determines whether the product is good or bad.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an image inspection system, an image inspection method, and a computer program.

Background Art

[0002] Patent Document 1 discloses a method of detecting a defective portion of a casting to be inspected by comparing a reference image, which is an image of a casting determined to be a non-defective product, with an inspection image, which is an image of the casting to be inspected, by pattern matching.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In castings, there is a large variation in the appearance of the base material for each product, so there is also a variation in the magnitude of the difference in appearance between the defective portion and the base material. Therefore, in the method of comparing the reference image and the inspection image as in the above document, it is easy to detect defective portions with a large difference in appearance from the base material, but it is difficult to detect defective portions with a small difference in appearance from the base material. Therefore, a technique capable of accurately determining the quality of castings is desired.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to the first aspect of the present disclosure, an image inspection system is provided. The image inspection system includes an image acquisition unit that acquires an inspection image obtained by imaging a casting to be inspected, a storage unit that stores feature amounts extracted from each of a plurality of good product images obtained by imaging a plurality of castings previously determined to be good products using a learned first convolutional neural network, and a first determination unit that extracts a feature amount from the inspection image using the first convolutional neural network, calculates the degree of abnormality of the inspection image with respect to the plurality of good product images using the feature amount extracted from the inspection image and the feature amounts extracted from each of the plurality of good product images, and determines the quality of the casting to be inspected based on the degree of abnormality of the inspection image. According to the image inspection system of this aspect, the first determination unit extracts a feature amount from the inspection image using the learned first convolutional neural network, calculates the degree of abnormality of the inspection image with respect to the plurality of good product images using the feature amount extracted from the inspection image and the feature amounts extracted from each of the plurality of good product images, and determines the quality of the casting to be inspected based on the degree of abnormality of the inspection image. Therefore, the quality of the casting can be accurately determined. (2) In the image inspection system of the above aspect, prior to calculating the degree of abnormality of the inspection image, the first determination unit may calculate the degree of abnormality of the defective product image with respect to the plurality of good product images in each layer of the first convolutional neural network using a defective product image obtained by imaging a casting previously determined to be a defective product and the first convolutional neural network, identify the layer in which the degree of abnormality of the defective product image is maximized, and determine the quality of the casting to be inspected based on the degree of abnormality of the inspection image in the layer in which the degree of abnormality of the defective product image is maximized. According to the image inspection system of this aspect, the first determination unit determines the quality of the casting to be inspected based on the degree of abnormality of the inspection image in the layer in which the degree of abnormality of the defective product image is maximized among the layers of the first convolutional neural network. Therefore, compared with a form in which the quality of the casting to be inspected is determined based on the degree of abnormality of the inspection image in layers other than the layer in which the degree of abnormality of the defective product image is maximized, the determination accuracy of the quality of the casting can be improved. (3) The image inspection system of the above form includes an image division unit that divides the inspection image into a plurality of partial inspection images. The first determination unit extracts feature amounts from each of the plurality of partial inspection images using the first convolutional neural network, and calculates the abnormality degree of each of the plurality of partial inspection images with respect to the plurality of good product images using the feature amounts extracted from each of the plurality of partial inspection images and the feature amounts extracted from each of the plurality of good product images, and may determine the quality of the casting to be inspected based on the abnormality degree of each of the plurality of partial inspection images. According to the image inspection system of this form, even when the image size of the inspection image is larger than the image size that can be input to the input layer of the first convolutional neural network, the image size of the image input to the input layer can be made closer to the image size that can be input to the input layer. Therefore, by reducing the image size of the image input to the input layer to the image size that can be input to the input layer, it is possible to suppress the loss of pixels representing defective parts from the image input to the input layer and the decrease in the determination accuracy of the quality of the casting. (4) The image inspection system of the above form may include a learned second convolutional neural network learned using a learning dataset including a plurality of defective product images obtained by imaging a plurality of castings previously determined to be defective products and a plurality of labels representing the types of defects of the castings for each of the plurality of defective product images, and a second determination unit that determines the type of defect of the casting to be inspected determined to be a defective product by the first determination unit using the inspection image of the casting to be inspected determined to be a defective product by the first determination unit. According to the image inspection system of this form, the second determination unit determines the type of defect of the casting determined to be a defective product by the first determination unit using the learned second convolutional neural network without determining whether the casting is a good product or not. Therefore, the determination accuracy of the type of defect of the casting can be improved. (5) According to a second aspect of the present disclosure, an image inspection method is provided. This image inspection method includes an image acquisition step of acquiring an inspection image obtained by imaging a casting to be inspected, and extracting feature amounts from the inspection image using a learned convolutional neural network. The abnormality degree of the inspection image with respect to the plurality of good product images is calculated using the feature amounts extracted from the inspection image and the feature amounts extracted from each of the plurality of good product images obtained by imaging a plurality of castings previously determined to be good products, and a determination step of determining the quality of the casting to be inspected based on the abnormality degree of the inspection image. According to the image inspection method of this aspect, in the determination step, feature amounts are extracted from the inspection image using a learned convolutional neural network, the abnormality degree of the inspection image with respect to the plurality of good product images is calculated using the feature amounts extracted from the inspection image and the feature amounts extracted from each of the plurality of good product images, and the quality of the casting to be inspected is determined based on the abnormality degree of the inspection image. Therefore, the quality of the casting can be accurately determined. (6) According to a third aspect of the present disclosure, a computer program is provided. This computer program causes a computer to realize an image acquisition function of acquiring an inspection image obtained by imaging a casting to be inspected, and extracting feature amounts from the inspection image using a learned convolutional neural network. The abnormality degree of the inspection image with respect to the plurality of good product images is calculated using the feature amounts extracted from the inspection image and the feature amounts extracted from each of the plurality of good product images obtained by imaging a plurality of castings previously determined to be good products, and a determination function of determining the quality of the casting to be inspected based on the abnormality degree of the inspection image. According to the computer program of this aspect, feature amounts can be extracted from the inspection image using a learned convolutional neural network, the abnormality degree of the inspection image with respect to the plurality of good product images can be calculated using the feature amounts extracted from the inspection image and the feature amounts extracted from each of the plurality of good product images, and the quality of the casting to be inspected can be determined based on the abnormality degree of the inspection image. Therefore, the quality of the casting can be accurately determined. The present disclosure can also be realized in various forms other than an image inspection system, an image inspection method, and a computer program. For example, it can be realized in the form of an image inspection device or the like.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0008] A. First Embodiment: FIG. 1 is an explanatory diagram schematically showing the schematic configuration of an image inspection system 10 according to the first embodiment. The image inspection system 10 inspects an inspection object OB using an image obtained by imaging the inspection object OB. In the following description, the image obtained by imaging the inspection object OB is referred to as an inspection image KG. In the present embodiment, the inspection object OB is a casting. More specifically, the inspection object OB is a cylinder block of an automobile engine, which is made of an aluminum alloy by die casting.

[0009] In the present embodiment, the image inspection system 10 includes a conveyor 20, a robot arm 30, a robot controller 35, a camera 40, a camera controller 45, a control device 50, and a display device 60.

[0010] The conveyor 20 is arranged, for example, inside a manufacturing factory that manufactures the object OB to be inspected. The conveyor 20 conveys the object OB to be inspected. In other embodiments, instead of the conveyor 20, for example, a stage on which the object OB to be inspected is placed may be provided.

[0011] The robot arm 30 is arranged in the vicinity of the conveyor 20. In the present embodiment, the robot arm 30 is composed of an articulated robot having six rotational axes J1 to J6. The robot arm 30 is driven under the control of a robot controller 35. In other embodiments, the robot arm 30 may be composed of, for example, a horizontal articulated robot or a Cartesian robot. In other embodiments, the robot controller 35 may be provided inside the control device 50.

[0012] The camera 40 is fixed to the tip of the robot arm 30. The position and orientation of the camera 40 are changed by the robot arm 30. The camera 40 images the object OB to be inspected. The camera 40 has, for example, a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 40 is connected to a camera controller 45 by wired communication or wireless communication, and the output signal from the image sensor of the camera 40 is transmitted to the camera controller 45.

[0013] The camera controller 45 generates an image by processing the output signal from the image sensor of the camera 40. The camera controller 45 is connected to the control device 50 by wired communication or wireless communication, and the image generated by the camera controller 45 is transmitted to the control device 50. In other embodiments, the camera controller 45 may be provided inside the control device 50.

[0014] The control device 50 is configured by a computer including a processor 51, a memory 52, and an input / output interface 53 that performs input / output of signals with the outside. In the present embodiment, the control device 50 has an image acquisition unit 110, an image segmentation unit 120, a first determination unit 130, a second determination unit 140, and an inspection result generation unit 150. The image acquisition unit 110, the image segmentation unit 120, the first determination unit 130, the second determination unit 140, and the inspection result generation unit 150 are realized software-wise by the processor 51 executing a computer program stored in the memory 52. The memory 52 stores in advance a first learned model MD1 and a second learned model MD2, which will be described later. Note that the memory 52 may be referred to as the storage unit 52. The control device 50 may be referred to as the image inspection device 50.

[0015] The image acquisition unit 110 acquires an inspection image KG. In the present embodiment, the image acquisition unit 110 controls the conveyor 20 to convey the inspection object OB to a predetermined inspection position, controls the robot arm 30 via the robot controller 35 to adjust the position and orientation of the camera 40 with respect to the inspection object OB, and controls the camera 40 via the camera controller 45 to image the inspection object OB, thereby acquiring the inspection image KG. The inspection image KG is stored in the memory 52.

[0016] The image segmentation unit 120 segments the inspection image KG to generate a plurality of partial inspection images PKG. In the following description, when the inspection image KG and the partial inspection images PKG are not particularly distinguished, they are simply referred to as the inspection image KG. Further, the image segmentation unit 120 assigns an identifier for specifying the position of each partial inspection image PKG in the inspection image KG to each partial inspection image PKG. Each partial inspection image PKG and each identifier are stored in the memory 52.

[0017] The first determination unit 130 determines whether the object to be inspected OB is good or bad, that is, whether the object to be inspected OB is a good product or a defective product. The second determination unit 140 determines the type of defect of the object to be inspected OB determined to be a defective product by the first determination unit 130. The types of defects of the object to be inspected OB include, for example, scratches, wrinkles, and water residue. In the present embodiment, the first determination unit 130 uses each partial inspection image PKG representing the object to be inspected OB and the first pre-trained model MD1 to determine whether the object to be inspected OB is good or bad. The second determination unit 140 uses the partial inspection image PKG representing the defective part of the object to be inspected OB and the second pre-trained model MD2 to determine the type of defect of the object to be inspected OB. In other embodiments, the first determination unit 130 may use the inspection image KG and the first pre-trained model MD1 instead of the partial inspection image PKG to determine whether the object to be inspected OB is good or bad. The second determination unit 140 may use the inspection image KG and the second pre-trained model MD2 instead of the partial inspection image PKG to determine the type of defect of the object to be inspected OB.

[0018] The inspection result generation unit 150 generates an inspection result RS of the object to be inspected OB including the determination results by the first determination unit 130 and the second determination unit 140. The inspection result RS is stored in the memory 52.

[0019] The first pre-trained model MD1 is configured to be able to extract feature amounts from the inspection image KG. The first pre-trained model MD1 has a pre-trained convolutional neural network. In the present embodiment, as the first pre-trained model MD1, EfficientNet pre-trained in advance using ImageNet as a learning dataset is used. ImageNet includes a plurality of images and a plurality of labels indicating the types of objects represented in each image. The first pre-trained model MD1 may be referred to as a pre-trained first convolutional neural network.

[0020] In this embodiment, the feature amounts TR extracted from each of a plurality of images obtained by imaging a plurality of cast products determined to be non-defective products in advance using the first learned model MD1 are stored in the memory 52. In the following description, an image obtained by imaging a cast product determined to be a non-defective product in advance is referred to as a non-defective product image. Each non-defective product image can be obtained, for example, by imaging a cast product determined to be a non-defective product in advance with the camera 40. The cast product represented in each non-defective product image is a product of the same type as the inspection object OB. The cast product represented in each non-defective product image has been determined to be a non-defective product, for example, by visual inspection by an operator. In this embodiment, since the partial inspection image PKG generated by dividing the inspection image KG is input to the first learned model MD1, each non-defective product image input to the first learned model MD1 is divided in the same manner as the inspection image KG. In another embodiment in which the inspection image KG instead of the partial inspection image PKG is input to the first learned model MD1, each non-defective product image input to the first learned model MD1 is not divided.

[0021] The second trained model MD2 is configured to be able to determine the type of defect of the inspection object OB represented in the inspection image KG by a classification method. The second trained model MD2 has a trained convolutional neural network. In the present embodiment, as the second trained model MD2, EfficientNet that has been pre-trained using ImageNet as a training dataset and further performs transfer learning using a training dataset including a plurality of images obtained by imaging a plurality of castings previously determined to be defective products and a plurality of labels indicating the types of defects of the castings represented in each image is used. In the following description, an image obtained by imaging a casting previously determined to be a defective product is referred to as a defective product image. Each defective product image for transfer learning can be obtained, for example, by imaging a casting previously determined to be a defective product with the camera 40. The casting represented in each defective product image is a product of the same type as the inspection object OB. The casting represented in each defective product image has been determined to be a defective product and the type of defect, for example, by visual inspection by an operator. In the present embodiment, since the partial inspection image PKG generated by dividing the inspection image KG is input to the second trained model MD2, each defective product image for transfer learning is divided in the same way as the inspection image KG. In another embodiment in which the inspection image KG instead of the partial inspection image PKG is input to the second trained model MD2, each defective product image for transfer learning is not divided. The second trained model MD2 may be referred to as a trained second convolutional neural network.

[0022] The display device 60 is connected to the control device 50 by wired communication or wireless communication. The display device 60 is composed of, for example, a liquid crystal display or an organic EL display. The inspection result RS of the inspection object OB is displayed on the display device 60.

[0023] FIG. 2 is a flowchart showing the content of the image inspection process executed by the control device 50 in the present embodiment. FIG. 3 is an explanatory diagram schematically showing the state in which the inspection image KG is divided. FIG. 4 is an explanatory diagram schematically showing the state in which the first determination unit 130 determines the quality of the inspection object OB. FIG. 5 is an explanatory diagram schematically showing the degree of abnormality of the inspection image KG. FIG. 6 is an explanatory diagram schematically showing the state in which the second determination unit 140 determines the type of defect of the inspection object OB.

[0024] The image inspection process shown in FIG. 2 is started by the processor 51 executing a computer program stored in the memory 52 of the control device 50. First, in step S110, the image acquisition unit 110 conveys the inspection object OB to a predetermined inspection position by controlling the conveyor 20, adjusts the position and orientation of the camera 40 with respect to the inspection object OB by controlling the robot arm 30 via the robot controller 35, and controls the camera 40 via the camera controller 45 to image the inspection object OB and acquire the inspection image KG. In the present embodiment, the size of the inspection image KG is 2500×2500 pixels (vertical×horizontal). Note that step S110 may be referred to as the image acquisition step of the image inspection method.

[0025] Next, in step S120, the image division unit 120 generates a plurality of partial inspection images PKG by dividing the inspection image KG. FIG. 3 schematically shows how the inspection image KG is divided. In the present embodiment, the image division unit 120 divides the inspection image KG such that the size of each partial inspection image PKG is equal to or smaller than the image size that can be input to the input layer of the first learned model MD1, and the size of each partial inspection image PKG is equal to or smaller than the image size that can be input to the input layer of the second learned model MD2. More specifically, in the present embodiment, since the image size that can be input to the input layer of the first learned model MD1 and the image size that can be input to the input layer of the second learned model MD2 are each 224×224 pixels, the image division unit 120 divides the inspection image KG such that the size of each partial inspection image PKG is equal to or smaller than 224×224 pixels. Note that step S120 may be referred to as the image division step of the image inspection method.

[0026] In step S130 of FIG. 2, the first determination unit 130 determines the quality of the inspection object OB using the partial inspection image PKG and the first learned model MD1. In the present embodiment, the first determination unit 130 inputs the partial inspection image PKG to the first learned model MD1 to extract feature amounts from the partial inspection image PKG, and calculates the abnormality degree of the partial inspection image PKG using the feature amounts TR of each non-defective product image stored in the memory 52 and the feature amounts extracted from the partial inspection image PKG. The first determination unit 130 calculates the abnormality degree of all the partial inspection images PKG generated in step S120. The first determination unit 130 determines that the inspection object OB is a non-defective product when the abnormality degree of all the partial inspection images PKG is less than a predetermined threshold value, and determines that the inspection object OB is a defective product when the abnormality degree of at least one partial inspection image PKG is equal to or greater than the threshold value.

[0027] In this embodiment, prior to the image inspection process, a test using defective product images is performed. In this test, by inputting a defective product image into the first pre-trained model MD1, the feature amounts of the defective product image in each intermediate layer of the first pre-trained model MD1 are extracted, and the abnormality degree of the defective product image with respect to each non-defective product image in each intermediate layer is calculated using the feature amount TR of each non-defective product image and the feature amount of the defective product image in each intermediate layer, and the intermediate layer in which the abnormality degree of the defective product image becomes maximum is specified. As shown in FIG. 4, in this embodiment, in the above test, the abnormality degree of the defective product image becomes maximum in the intermediate layer of level 3. In the intermediate layer of level 3, the size of the feature map is 56×56 pixels, and the feature amount is represented by a vector of 24 elements. When determining the quality of the inspection object OB in step S130, the first determination unit 130 extracts the feature amount of the partial inspection image PKG in the intermediate layer of level 3 in which the abnormality degree of the defective product image becomes maximum in the above test, and calculates the abnormality degree of the partial inspection image PKG in the intermediate layer of level 3 using the feature amount TR of each non-defective product image and the feature amount of the partial inspection image PKG in the intermediate layer of level 3, and determines the quality of the inspection object OB based on the abnormality degree of the partial inspection image PKG in the intermediate layer of level 3. In the intermediate layer in which the abnormality degree of the defective product image becomes maximum, the divergence of the abnormality degree between the non-defective product image and the defective product image is more prominent than in other intermediate layers, so the difference between the non-defective product image and the defective product image can be clearly represented. That is, it can be said that the resolution of the abnormality degree is higher in the intermediate layer in which the abnormality degree of the defective product image becomes maximum than in other intermediate layers.

[0028] As shown in FIG. 5, in this embodiment, the abnormality degree of the partial inspection image PKG is represented by the Mahalanobis distance L from the center point CP of the group of non-defective product images RG composed of a plurality of non-defective product images RG in which the feature amount TR is stored in the memory 52. The longer the Mahalanobis distance L from the center point CP of the group of non-defective product images, the higher the abnormality degree. In this embodiment, the threshold value TS is set to a value three times the standard deviation of the group of non-defective product images. Note that step S130 may be referred to as the first determination step of the image inspection method.

[0029] In step S140 of FIG. 2, the second determination unit 140 determines whether the inspection object OB has been determined to be a defective product by the first determination unit 130. If the first determination unit 130 determines that the inspection object OB is a defective product, then in step S150, the second determination unit 140 uses the partial inspection image PKG in which the defective part has been detected and the second learned model MD2 to determine the type of defect of the inspection object OB. As shown in FIG. 6, in the present embodiment, the second learned model MD2 is configured to output the type of defect of the inspection object OB represented in the input partial inspection image PKG when the partial inspection image PKG is input to the second learned model MD2. The second determination unit 140 determines the type of defect of the inspection object OB represented in the partial inspection image PKG by inputting the partial inspection image PKG to the second learned model MD2. On the other hand, if the first determination unit 130 determines that the inspection object OB is a non-defective product, then the second determination unit 140 skips the process of step S150 and proceeds to the process of step S160. Note that step S150 may be referred to as the second determination step of the image inspection method.

[0030] In step S160 of FIG. 2, the inspection result generation unit 150 generates an inspection result RS and causes the display device 60 to display the inspection result RS. The inspection result RS represents the pass / fail of the inspection object OB determined by the first determination unit 130. If the first determination unit 130 determines that the inspection object OB is a defective product, then in addition to the pass / fail of the inspection object OB, the inspection result RS represents the type of defect of the inspection object OB determined by the second determination unit 140. The inspection result RS may represent the position of the defective part of the inspection object OB and the like. Then, the inspection result generation unit 150 ends this process. Note that step S160 may be referred to as the inspection result display step of the image inspection method.

[0031] In addition, when the inspection object OB is determined to be a defective product by the first determination unit 130, after the image inspection process, the operator may perform a visual inspection on the inspection object OB determined to be a defective product. If, as a result of the visual inspection by the operator, the inspection object OB is determined to be a non-defective product, that is, if it is found that there is an error in the determination result by the first determination unit 130, the first learned model MD1 may be relearned using the inspection image KG of this inspection object OB. By relearning the first learned model MD1, the accuracy of the pass / fail determination by the first determination unit 130 can be improved. On the other hand, if, as a result of the visual inspection by the operator, the inspection object OB is determined to be a defective product, the second learned model MD2 may be relearned using the inspection image KG of this inspection object OB and the label indicating the type of defect of this inspection object OB. By relearning the second learned model MD2, the accuracy of the defect type determination by the second determination unit 140 can be improved. The visual inspection by the operator for collecting defective product images is not limited to the case where the inspection object OB is determined to be a defective product by the first determination unit 130, and may also be performed when the inspection object OB is determined to be a non-defective product by the first determination unit 130.

[0032] According to the image inspection system 10 in the present embodiment described above, the first determination unit 130 extracts feature amounts from the partial inspection image PKG using the first learned model MD1 having a convolutional neural network, calculates the abnormality degree of the partial inspection image PKG with respect to each non-defective product image using the feature amounts extracted from the partial inspection image PKG and the feature amounts TR extracted from each non-defective product image, and determines the pass / fail of the inspection object OB based on the abnormality degree of the partial inspection image PKG. Generally, a convolutional neural network is excellent at identifying image features. Therefore, the first determination unit 130 can accurately determine the pass / fail of the inspection object OB.

[0033] Also, in the present embodiment, the first determination unit 130 calculates the abnormality degree of the partial inspection image PKG by using the feature amount of the partial inspection image PKG in the intermediate layer where the abnormality degree of the defective product image becomes the maximum among the intermediate layers of the convolutional neural network of the first learned model MD1 and the feature amount TR extracted from each non-defective product image. In the intermediate layer where the abnormality degree of the defective product image becomes the maximum, the resolution of the abnormality degree is higher than that of other intermediate layers. Therefore, the determination accuracy of whether the inspection object OB is good or defective by the first determination unit 130 can be improved.

[0034] Also, in the present embodiment, the second determination unit 140 determines the type of defect of the inspection object OB by using the partial inspection image PKG of the inspection object OB that has been previously determined to be defective by the first determination unit 130 and the second learned model MD2 having a convolutional neural network. When determining the type of defect of a cast product by classifying with a convolutional neural network, if a non-defective class is provided, the possibility of misjudgment increases. In contrast, in the present embodiment, since no non-defective class is provided in the classification by the convolutional neural network of the second learned model MD2, the possibility of misjudgment can be reduced. Therefore, the determination accuracy of the type of defect of the inspection object OB can be improved.

[0035] In addition, in the present embodiment, the first determination unit 130 inputs the partial inspection image PKG generated by dividing the inspection image KG, rather than the inspection image KG, into the first pre-trained model MD1 to determine the quality of the inspection object OB. Therefore, when the partial inspection image PKG is input into the first pre-trained model MD1, the partial inspection image PKG can be reduced according to the image size that can be input to the input layer of the first pre-trained model MD1, and it is possible to suppress the loss of pixels representing defective portions from the partial inspection image PKG. Further, the second determination unit 140 inputs the partial inspection image PKG, rather than the inspection image KG, into the second pre-trained model MD2 to determine the type of defect of the inspection object OB. Therefore, when the partial inspection image PKG is input into the second pre-trained model MD2, the partial inspection image PKG can be reduced according to the image size that can be input to the input layer of the second pre-trained model MD2, and it is possible to suppress the loss of pixels representing defective portions from the partial inspection image PKG. Accordingly, it is possible to suppress a decrease in the determination accuracy of the quality of the inspection object OB and a decrease in the determination accuracy of the type of defect of the inspection object OB.

[0036] B. Other Embodiments: (B1) The image inspection system 10 of the above-described embodiment has the second determination unit 140. In contrast, the image inspection system 10 may not have the second determination unit 140. In this case, in the image inspection process by the image inspection system 10, the type of defect of the inspection object OB is not determined. However, for example, the type of defect of the inspection object OB may be determined by visual inspection by an operator. For the inspection object OB determined to be a non-defective product by the image inspection process, visual inspection by an operator may not be performed, so the frequency of visual inspection by the operator can be reduced.

[0037] In the image inspection system 10 of the above-described embodiment, the image division unit 120 divides the inspection image KG so that the size of each partial inspection image PKG is equal to or smaller than the image size that can be input to the input layer of the first pre-trained model MD1 or the input layer of the second pre-trained model MD2. On the other hand, the size of each partial inspection image PKG generated by the image division unit 120 may be larger than the image size that can be input to the input layer of the first pre-trained model MD1 or the input layer of the second pre-trained model MD2. Even in this case, compared with the form in which the inspection image KG is input to the input layer of the first pre-trained model MD1 or the input layer of the second pre-trained model MD2, pixels representing defective portions are less likely to be lost due to image reduction, so that a decrease in the determination accuracy of the quality of the inspection object OB and a decrease in the determination accuracy of the type of defect of the inspection object OB can be suppressed.

[0038] In the image inspection system 10 of each of the above-described embodiments, the image segmentation unit 120 divides the inspection image KG to generate a plurality of partial inspection images PKG. The first determination unit 130 determines the quality of the inspection object OB using the partial inspection image PKG and the first learned model MD1. The second determination unit 140 determines the type of defect of the inspection object OB using the partial inspection image PKG and the second learned model MD2. In contrast, the image inspection system 10 may not have the image segmentation unit 120. In this case, the first determination unit 130 determines the quality of the inspection object OB using the inspection image KG and the first learned model MD1, and the second determination unit 140 determines the type of defect of the inspection object OB using the inspection image KG and the second learned model MD2. When the size of the inspection image KG is equal to or smaller than the image size that can be input to the input layer of the first learned model MD1 or the second learned model MD2, the inspection image KG is not reduced to match the image size that can be input to the input layer of the first learned model MD1 or the second learned model MD2, so the pixels representing the defective part are not lost. Also, even if the size of the inspection image KG is larger than the image size that can be input to the input layer of the first learned model MD1 or the second learned model MD2, if the size of the inspection image KG is close to the image size that can be input to the input layer of the first learned model MD1 or the second learned model MD2, even if the inspection image KG is reduced to match the image size that can be input to the input layer of the first learned model MD1 or the second learned model MD2, the change in the size of the inspection image KG is relatively small, so the determination accuracy of the quality of the inspection object OB and the determination accuracy of the type of defect of the inspection object OB are not likely to decrease.

[0039] (B4) In the image inspection system 10 of each of the above-described embodiments, the first determination unit 130 calculates the abnormality degree of the defective product image with respect to each non-defective product image in each intermediate layer of the first learned model MD1 by means of a test performed prior to the image inspection process, and identifies the intermediate layer in which the abnormality degree of the defective product image becomes maximum. Thereafter, when determining the pass / fail of the inspection object OB in step S130 of the image inspection process, the first determination unit 130 determines the pass / fail of the inspection object OB based on the abnormality degree of the partial inspection image PKG with respect to each non-defective product image in the intermediate layer in which the abnormality degree of the defective product image becomes maximum in the above test. On the other hand, the first determination unit 130 may determine the pass / fail of the inspection object OB based on the abnormality degree of the partial inspection image PKG with respect to each non-defective product image in the output layer of the first learned model MD1.

[0040] (B5) In the image inspection system 10 of each of the above-described embodiments, the first determination unit 130 determines that the inspection object OB is a defective product when the abnormality degree of at least one partial inspection image PKG is equal to or greater than the threshold value, and determines that the inspection object OB is a non-defective product in other cases. On the other hand, the first determination unit 130 may determine that the inspection object OB is a defective product when the abnormality degree of two or more predetermined numbers of partial inspection images PKG is equal to or greater than the threshold value, and determine that the inspection object OB is a non-defective product in other cases.

[0041] In the image inspection system 10 of each of the above-described embodiments, a plurality of second learned models MD2 are pre-stored in the memory 52, and the second determination unit 140 may determine the type of defect of the inspection object OB using the plurality of second learned models MD2. For example, the second determination unit 140 may determine the type of defect of the inspection object OB using eight second learned models MD2 configured to determine the presence or absence of one type of defect different from each other by a classification method. In this case, the second determination unit 140 determines the presence or absence of the first type of defect using the first second learned model MD2. When it is determined that there is the first type of defect, the type of the first type of defect is represented in the inspection result RS. When it is determined that there is no first type of defect, the second determination unit 140 determines the presence or absence of the second type of defect using the second second learned model MD2. When it is determined that there is the second type of defect, the type of the second type of defect is represented in the inspection result RS. When it is determined that there is no second type of defect, the second determination unit 140 determines the presence or absence of the third type of defect using the third second learned model MD2. Then, when the second determination unit 140 determines the presence or absence of the eighth type of defect using the eighth second learned model MD2 and it is determined that there is no eighth type of defect, the inspection result RS represents the type of defect as other defects. Since each second learned model MD2 determines the presence or absence of one type of defect, the determination accuracy of the type of defect of the inspection object OB can be improved.

[0042] The present disclosure is not limited to the above-described embodiments, and can be realized in various configurations without departing from the gist thereof. For example, the technical features in the embodiments corresponding to the technical features in each of the forms described in the summary of the invention can be appropriately replaced or combined in order to solve some or all of the above-described problems or to achieve some or all of the above-described effects. Further, if the technical feature is not described as essential in this specification, it can be appropriately deleted.

Description of Reference Numerals

[0043] 10…Imaging inspection system, 20…Conveyor, 30…Robot arm, 35…Robot controller, 40…Camera, 45…Camera controller, 50…Control device, 51…Processor, 52…Memory, 53…Input / output interface, 60…Display device, 110…Image acquisition unit, 120…Image segmentation unit, 130…First determination unit, 140…Second determination unit, 150…Inspection result generation unit, KG…Inspection image, MD1…First learned model, MD2…Second learned model, OB…Inspection object, PKG…Partial inspection image, RS…Inspection result, TR…Feature quantity

Claims

1. An image inspection system, comprising: an image acquisition unit that acquires an inspection image obtained by imaging a casting to be inspected; a storage unit that stores feature amounts extracted from each of a plurality of good product images obtained by imaging a plurality of castings previously determined to be good products using a learned first convolutional neural network; a first determination unit that extracts a feature amount from the inspection image using the first convolutional neural network, calculates an abnormality degree of the inspection image with respect to the plurality of good product images using the feature amount extracted from the inspection image and the feature amounts extracted from each of the plurality of good product images, and determines whether the casting to be inspected is good or bad based on the abnormality degree of the inspection image; and the first determination unit prior to calculating the abnormality degree of the inspection image, calculates the abnormality degree of the defective product image with respect to the plurality of good product images in each layer of the first convolutional neural network using the defective product image obtained by imaging a casting previously determined to be defective and the first convolutional neural network, thereby identifying the layer in which the abnormality degree of the defective product image becomes maximum; An image inspection system that determines whether the casting to be inspected is good or bad based on the abnormality degree of the inspection image in the layer where the abnormality degree of the defective product image becomes maximum.

2. The image inspection system according to Claim 1, further comprising: an image division unit that divides the inspection image into a plurality of partial inspection images; the first determination unit extracts a feature amount from each of the plurality of partial inspection images using the first convolutional neural network, calculates the abnormality degree of each of the plurality of partial inspection images with respect to the plurality of good product images using the feature amount extracted from each of the plurality of partial inspection images and the feature amounts extracted from each of the plurality of good product images, and determines whether the casting to be inspected is good or bad based on the abnormality degree of each of the plurality of partial inspection images.

3. The image inspection system according to Claim 1 or Claim 2, A learned second convolutional neural network trained using a learning dataset including a plurality of defective product images obtained by imaging a plurality of castings previously determined as defective products and a plurality of labels representing the types of defects of the castings for each of the plurality of defective product images, and an inspection image of the casting to be inspected determined as a defective product by the first determination unit. A second determination unit that determines the type of defect of the casting to be inspected determined as a defective product by the first determination unit is provided, and an image inspection system.

4. An image inspection method, comprising: An image acquisition step of acquiring an inspection image obtained by imaging a casting to be inspected; Extracting a feature amount from the inspection image using a learned convolutional neural network, and using the feature amount extracted from each of a plurality of non-defective product images obtained by imaging a plurality of castings previously determined as non-defective products using the convolutional neural network and the feature amount extracted from the inspection image. Calculating the degree of abnormality of the inspection image with respect to the plurality of non-defective product images, and determining the quality of the casting to be inspected based on the degree of abnormality of the inspection image. A determination step; Having The determination step includes Prior to calculating the degree of abnormality of the inspection image, using a defective product image obtained by imaging a casting previously determined as a defective product and the convolutional neural network to calculate the degree of abnormality of the defective product image with respect to the plurality of non-defective product images in each layer of the convolutional neural network. A step of identifying a layer in which the degree of abnormality of the defective product image becomes maximum; Based on the degree of abnormality of the inspection image in the layer where the degree of abnormality of the defective product image becomes maximum, determining the quality of the casting to be inspected. A step; An image inspection method including

5. A computer program, comprising: An image acquisition function for acquiring an inspection image obtained by imaging a casting to be inspected; Extracting a feature amount from the inspection image using a learned convolutional neural network, and using the feature amount extracted from each of a plurality of non-defective product images obtained by imaging a plurality of castings previously determined as non-defective products using the convolutional neural network and the feature amount extracted from the inspection image. Calculating the degree of abnormality of the inspection image with respect to the plurality of non-defective product images, and determining the quality of the casting to be inspected based on the degree of abnormality of the inspection image. A determination function; To be realized by a computer, The determination function Prior to calculating the degree of abnormality of the inspection image, by using a defective product image obtained by imaging a cast product previously determined to be a defective product and the convolutional neural network, calculate the degree of abnormality of the defective product image with respect to the plurality of non-defective product images in each layer of the convolutional neural network, thereby identifying the layer where the degree of abnormality of the defective product image is maximized, Based on the degree of abnormality of the inspection image in the layer where the degree of abnormality of the defective product image is maximized, a function for determining whether the cast product to be inspected is good or defective, A computer program including the above.

Images