Appearance inspection apparatus, appearance inspection method, and computer-readable recording medium

Abstract

An appearance inspection device according to the present invention comprises a data acquisition unit that acquires an image indicating a normal state as a training image, a feature map acquisition unit that acquires a plurality of feature maps that are output from an intermediate layer of a trained neural network and indicate characteristics of the training image, a grouping unit that groups the acquired plurality of feature maps into a specific number of groups, an inspection model creation unit that creates an inspection model indicating a specific inspection reference value for each group obtained by the grouping unit, and an inspection model storage unit that stores the inspection models created by the inspection model creation unit.

Description

Technical Field

[0001] The present invention relates to an appearance inspection apparatus, an appearance inspection method, and a computer-readable recording medium.

Background Art

[0002] In a manufacturing site such as a factory, a plurality of industrial machines such as machine tools and robots are installed and operated to manufacture products. The manufactured products are shipped after undergoing inspections such as appearance inspection (for example, Patent Document 1). In order to perform an appearance inspection of a product, it is necessary to create in advance a discriminator that discriminates whether an image obtained by imaging the product is an image of a normal product or an image of an abnormal product.

[0003] When creating such a discriminator, a large number of images of normal products and a large number of images of abnormal products are collected in advance. Then, machine learning is performed using the collected images. In many cases, for an image of an abnormal product, it is desired to specify which part of the image is abnormal. In such a case, a label image indicating an abnormal location in the image of the abnormal product is created in advance and learning is performed.

[0004] As another method of performing appearance inspection using a neural network, there is a method of extracting and using all or part of the output of the intermediate layer when an inspection image is input to a learned neural network as a feature map (for example, Non-Patent Document 1, etc.). This method will be described with reference to FIGS. 7 and 8. FIG. 7 is a diagram for explaining a learning method of an inspection model according to the prior art. In this method, for example, ResNet18, WideResNet50, etc. are used as the neural network. These neural networks are pre-trained using images of normal products. Then, a predetermined number (for example, N images) of images are input to the learned neural network, and a predetermined number (for example, 100) of outputs of the intermediate layer are randomly acquired as feature maps. After that, after aligning the sizes of the obtained feature maps, the mean μ and the covariance matrix Σ are calculated for each element of the feature map. The mean μ and the covariance matrix Σ for each element of this feature map are used as the inspection model.

[0005] FIG. 8 is a diagram for explaining a method of performing appearance inspection according to the prior art. When performing appearance inspection, an inspection image is input to a learned neural network to obtain a feature map. Then, the mean is calculated for each element of the obtained feature map, and the Mahalanobis distance between this and the inspection model is obtained. The value obtained by normalizing the Mahalanobis distance for each element over the entire feature map is used as the abnormality degree of the pixel position corresponding to the position of the element. It can be displayed in the form of an abnormality degree map in which colors are associated with the abnormality degree for each pixel. Then, the positions of the pixels larger than a predetermined threshold are determined as abnormal locations.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Non-Patent Documents

[0007]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0008] However, in the method of appearance inspection using the prior art, there are cases where anomalies cannot be detected. In the prior art, there are times when an inspection model is created in which the detection rate of anomalies decreases in a specific anomaly mode. Also, in the appearance inspection of products in a factory, the orientation of parts in the captured image data is not always constant, but in the prior art, there are cases where an inspection model is created that cannot successfully detect anomalies at a predetermined position of parts facing a predetermined orientation. Therefore, a technique for improving the accuracy of anomaly detection is desired.

Means for Solving the Problems

[0009] In the appearance inspection apparatus according to the present disclosure, by inputting images of a plurality of normal products into a learned neural network, all outputs of the intermediate layer are obtained as feature maps. Then, the obtained feature maps are divided into a predetermined number of groups, and an inspection reference value is calculated for each group. Examples of the inspection reference value include the mean μ for each element of the feature map and the covariance matrix Σ. And a set of the inspection reference values for each group is used as an inspection model. When performing appearance inspection, based on the inspection values calculated from the feature maps obtained by inputting the inspection image into the learned neural network, an anomaly determination is made using the inspection reference values for each group. Then, the normality / anomaly of the inspection image is determined by an ensemble inference method using the results of each determination.

[0010] And one aspect of the present disclosure is an appearance inspection apparatus using a learned neural network, including a data acquisition unit that acquires an image indicating a normal state as a learning image, a feature map acquisition unit that acquires a plurality of feature maps indicating features of the learning image output in an intermediate layer of the learned neural network, a group division unit that divides the acquired plurality of feature maps into a predetermined number of groups, an inspection model creation unit that creates a predetermined inspection reference value as an inspection model for each group divided by the group division unit, and an inspection model storage unit that stores the inspection model created by the inspection model creation unit.

[0011] Another aspect of the present disclosure is a method executed by an appearance inspection apparatus using a learned neural network, including steps of acquiring an image indicating a normal state as a learning image, acquiring a plurality of feature maps indicating features of the learning image output in an intermediate layer of the learned neural network, dividing the acquired plurality of feature maps into a predetermined number of groups, creating a predetermined inspection reference value as an inspection model for each of the divided groups, and storing the created inspection model.

[0012] Another aspect of the present disclosure is a computer-readable recording medium recording a program for operating an appearance inspection apparatus using a learned neural network, the program causing a computer to operate as a data acquisition unit that acquires an image indicating a normal state as a learning image, a feature map acquisition unit that acquires a plurality of feature maps indicating features of the learning image output in an intermediate layer of the learned neural network, a group division unit that divides the acquired plurality of feature maps into a predetermined number of groups, an inspection model creation unit that creates a predetermined inspection reference value as an inspection model for each group divided by the group division unit, and an inspection model storage unit that stores the inspection model created by the inspection model creation unit.

Advantages of the Invention

[0013] According to one aspect of the present disclosure, by incorporating an ensemble inference method for the abnormality degree of each element calculated based on the feature map obtained from the intermediate layer, it can be expected to improve the inspection accuracy and reduce false positives.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Modes for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic hardware configuration diagram showing the main part of an appearance inspection apparatus according to an embodiment of the present invention. The appearance inspection apparatus 1 according to the present embodiment can be implemented, for example, as a control device that controls industrial machinery based on a control program. Further, the appearance inspection apparatus 1 according to the present embodiment can be implemented on a personal computer installed together with a control device that controls industrial machinery based on a control program, a personal computer connected to the control device via a wired / wireless network, a cell computer, a fog computer 6, or a cloud server 7. In the present embodiment, an example in which the appearance inspection apparatus 1 is implemented on a personal computer connected to a control device via a network is shown.

[0016] The CPU 11 included in the appearance inspection apparatus 1 according to the present embodiment is a processor that controls the appearance inspection apparatus 1 as a whole. The CPU 11 reads out the system program stored in the ROM 12 via the bus 22 and controls the entire appearance inspection apparatus 1 according to the system program. Temporary calculation data, display data, and various data input from the outside are temporarily stored in the RAM 13.

[0017] The non-volatile memory 14 is composed of, for example, a memory backed up by a battery (not shown) or an SSD (Solid State Drive), and the storage state is retained even when the power of the appearance inspection apparatus 1 is turned off. The non-volatile memory 14 stores data read from the external device 72 via the interface 15, data input via the input device 71, data acquired from the industrial machine 3 (including image data captured by the imaging device 4), and the like. The data stored in the non-volatile memory 14 may be expanded to the RAM 13 at the time of execution / use. Further, various system programs such as a known analysis program are written in advance in the ROM 12.

[0018] The interface 15 is an interface for connecting the CPU 11 of the appearance inspection device 1 and an external device 72 such as a USB device. From the external device 72 side, for example, a control program stored in advance, data related to the operation of each industrial machine 3, etc. can be read. Also, the control program, setting data, etc. edited in the appearance inspection device 1 can be stored in an external storage means via the external device 72.

[0019] The interface 20 is an interface for connecting the CPU of the appearance inspection device 1 and a wired or wireless network 5. To the network 5, industrial machines 3, fog computers 6, cloud servers 7, etc. are connected, and data is exchanged mutually with the appearance inspection device 1.

[0020] The industrial machine 3 may be, for example, a robot to which an imaging device 4 such as an imaging device is attached. The industrial machine 3 transmits the image of the product to be inspected for appearance captured by the imaging device 4 to the appearance inspection device 1 via the network 5 in response to a request from the appearance inspection device 1.

[0021] To the display device 70, each data read into the memory, data obtained as a result of executing a program, etc., data output from a machine learning device 100 described later, etc. are output and displayed via the interface 17. Also, an input device 71 composed of a keyboard, a pointing device, etc. passes commands, data, etc. based on operations by an operator to the CPU 11 via the interface 18.

[0022] The interface 21 is an interface for connecting the CPU 11 and the machine learning device 100. The machine learning device 100 includes a processor 101 that controls the entire machine learning device 100, a ROM 102 that stores system programs and the like, a RAM 103 for temporarily storing data in each process related to machine learning, and a non-volatile memory 104 used for storing models and the like. The machine learning device 100 can observe each piece of information (for example, image data of a product captured by the imaging device of the industrial machine 3) that can be acquired by the appearance inspection device 1 via the interface 21. Further, the appearance inspection device 1 acquires the processing result output from the machine learning device 100 via the interface 21, stores the acquired result, displays it, or transmits it to other devices via the network 5 or the like.

[0023] FIG. 2 schematically shows, as a block diagram, the functions of the appearance inspection device 1 according to the first embodiment of the present invention. The appearance inspection device 1 according to the present embodiment has a function for creating an inspection model for appearance inspection. Each function of the appearance inspection device 1 is realized by the CPU 11 of the appearance inspection device 1 shown in FIG. 1 and the processor 101 of the machine learning device 100 executing a system program and controlling the operations of each part of the appearance inspection device 1 and the machine learning device 100.

[0024] The appearance inspection device 1 of the present embodiment includes a data acquisition unit 110, a feature map acquisition unit 120, a group division unit 140, and an inspection model creation unit 150. Further, on the RAM 13 to the non-volatile memory 14 of the appearance inspection device 1, there are a data storage unit 200 which is an area for storing the data acquired by the data acquisition unit 110, a feature map storage unit 220 which is an area for storing the feature maps acquired by the feature map acquisition unit 120, and an inspection model storage unit 230 which is an area for storing the inspection models created by the inspection model creation unit 150, which are prepared in advance. On the other hand, the machine learning device 100 includes an estimation unit 130. Further, on the RAM 103 to the non-volatile memory 104 of the machine learning device 100, a model storage unit 210 for storing a learned neural network model obtained by learning the images of normal products in advance is prepared in advance.

[0025] The data acquisition unit 110 acquires image data obtained by imaging the appearance of a product. The data acquired by the data acquisition unit 110 may be acquired via a network 5, for example, the data imaged by an imaging device 4 during the operation of an industrial machine 3. Also, the data may be acquired from an external device 72 such as a USB memory that has been pre-stored therein. Further, the data may be acquired via a fog computer 6, a cloud server 7, or the like. These data are mainly used for the learning operation of the inspection model. In the learning stage, the data acquired by the data acquisition unit 110 is image data obtained by imaging the appearance of a normal product. The data acquisition unit 110 stores the acquired image data obtained by imaging the appearance of a normal product in the data storage unit 200.

[0026] The feature map acquisition unit 120 instructs the estimation unit 130 to perform an estimation process using the learned neural network stored in the model storage unit 210 based on the image data obtained by imaging the appearance of a normal product stored in the data storage unit 200. Then, in the estimation process by the estimation unit 130, the output of the intermediate layer of the learned neural network is acquired as a feature map. The feature map acquisition unit 120 converts all the acquired feature maps to a predetermined size respectively, and then stores them in the feature map storage unit 220. A plurality of feature maps are stored in the feature map storage unit 220.

[0027] The estimation unit 130 executes an estimation process using the learned neural network stored in the model storage unit 210. Then, in the execution process of the estimation process, the output of the intermediate layer of the learned neural network is output to the feature map acquisition unit 120 as a feature map. The estimation unit 130 according to the present embodiment does not necessarily need to output the output at the output layer of the neural network to the outside of the machine learning device 100.

[0028] The group division unit 140 divides a plurality of feature maps stored in the feature map storage unit 220 into a predetermined number of groups. Regarding the number of groups into which the group division unit 140 divides the feature maps, it may be appropriately determined according to the size of the feature maps, the number of feature maps that can be obtained, and the method of ensemble inference to be performed later. For example, when using majority logic in ensemble inference, it may be divided into at least three or more groups. Also, when the number of feature maps that can be obtained is small, dividing into too many groups is not effective. It is desirable that an experienced engineer appropriately determines the number of groups to be divided.

[0029] The inspection model creation unit 150 calculates a predetermined inspection reference value for each group divided by the group division unit 140. The predetermined inspection reference value may be a set of the mean μ and the covariance matrix Σ for each element of the plurality of feature maps belonging to each group. Then, the set of inspection reference values calculated for each group is stored in the inspection model storage unit 230 as an inspection model. For example, a set of the mean μ and the covariance matrix Σ for each element of the feature maps calculated for each group is stored as an inspection model. When the group division unit 140 divides the feature maps into five groups, the inspection model consists of the mean μ and the covariance matrix Σ for each element of the five feature maps.

[0030] FIG. 3 is a diagram showing the flow of creating an inspection model for appearance inspection by the appearance inspection apparatus 1 according to the present embodiment. As illustrated in FIG. 3, when the data acquisition unit 110 included in the appearance inspection apparatus 1 acquires image data obtained by imaging the appearance of a normal product as learning image data, the estimation unit 130 inputs the learning image to the learned neural network and performs inference. The feature map acquisition unit 120 acquires the output from the intermediate layer in the inference process as a feature map. When a plurality of learning images are input and a plurality of feature maps are acquired, the group division unit 140 divides these into a plurality of groups (M groups in FIG. 3). Then, the inspection model creation unit 150 calculates the mean μ1 to μ M and the variance-covariance matrix Σ1 to Σ MCalculate this. This (μ1, Σ1) ~ (μ M , Σ M ) is stored in the inspection model storage unit 230 as an inspection model.

[0031] The appearance inspection apparatus 1 according to the present embodiment having the above configuration divides the feature maps acquired from the intermediate layer of the learned neural network into a predetermined number of groups, and uses the average μ and the variance-covariance matrix Σ for each element of the feature maps calculated for each group as an inspection model. Instead of using randomly extracted feature maps as in the prior art, all feature maps are used to calculate a plurality of averages μ and variance-covariance matrices Σ. By using this, a comprehensive inspection model can be prepared for the images of normal products. The created inspection model can be used for ensemble inference.

[0032] FIG. 4 shows, as a schematic block diagram, the functions included in the appearance inspection apparatus 1 according to the second embodiment of the present invention. The appearance inspection apparatus 1 according to the present embodiment has a function for performing ensemble inference using an inspection model for appearance inspection. Each function included in the appearance inspection apparatus 1 is realized by the CPU 11 of the appearance inspection apparatus 1 shown in FIG. 1 and the processor 101 of the machine learning apparatus 100 executing a system program and controlling the operations of each part of the appearance inspection apparatus 1 and the machine learning apparatus 100.

[0033] The appearance inspection apparatus 1 of the present embodiment includes a data acquisition unit 110, a feature map acquisition unit 120, an abnormality determination unit 160, and an output unit 170. Further, on the RAM 13 to the non-volatile memory 14 of the appearance inspection apparatus 1, a data storage unit 200 which is an area for storing the data acquired by the data acquisition unit 110 and an inspection model storage unit 230 which is an area for storing a previously created inspection model are prepared. On the other hand, the machine learning apparatus 100 includes an estimation unit 130. Further, on the RAM 103 to the non-volatile memory 104 of the machine learning apparatus 100, a model storage unit 210 for storing a model of a learned neural network that has previously learned images of normal products is prepared in advance.

[0034] The data acquisition unit 110 according to this embodiment acquires image data obtained by imaging the appearance of a product. The data acquired by the data acquisition unit 110 may be acquired via the network 5, for example, data imaged by the imaging device 4 during the operation of the industrial machine 3. Alternatively, data previously stored in an external device 72 such as a USB memory may be acquired. Furthermore, data may be acquired via the fog computer 6, the cloud server 7, or the like. The data acquisition unit 110, which is used for the appearance inspection of the imaged product, stores the image data obtained by imaging the appearance of the product in the data storage unit 200.

[0035] The feature map acquisition unit 120 according to this embodiment commands the estimation unit 130 to perform an estimation process using the learned neural network stored in the model storage unit 210 based on the image data obtained by imaging the appearance of the product stored in the data storage unit 200. Then, in the estimation process by the estimation unit 130, the output of the intermediate layer of the learned neural network is acquired as a feature map. The feature map acquisition unit 120 outputs all the acquired feature maps to the abnormality determination unit 160 after converting each of them to a predetermined size.

[0036] The estimation unit 130 according to this embodiment has the same function as the estimation unit 130 included in the appearance inspection device 1 according to the first embodiment.

[0037] The abnormality determination unit 160 performs ensemble inference using the inspection model stored in the inspection model storage unit 230 based on the feature map acquired by the feature map acquisition unit 120. Then, based on the result of the ensemble inference, it determines whether the inspection image is abnormal. Ensemble inference is a method of combining multiple inference results for inference. As the simplest method of ensemble inference, majority vote inference can be considered. In this case, the abnormality determination unit 160 determines whether there is an abnormality for each of the plurality of inspection reference values stored in the inspection model storage unit 230 with respect to the feature map indicating the features of the inspection image. Then, if there are more determinations of normal in each determination result, it is determined as normal, and if there are more determinations of abnormal, it is determined as abnormal. As another example, there are also examples such as a method of using, as the determination result, when a predetermined number or more of determination results are obtained in advance, or a method of determining as normal only when all the determination results match.

[0038] FIG. 5 is a diagram showing the flow of determining whether an inspection image is abnormal by the appearance inspection apparatus 1 according to the present embodiment. As illustrated in FIG. 5, as an inspection model in advance, a set of the mean and covariance matrix for each element of the feature map (μ1, Σ1) to (μ M , Σ M) is assumed to be stored in the inspection model storage unit 230. At this time, when the data acquisition unit 110 included in the appearance inspection apparatus 1 acquires image data obtained by imaging the appearance of the product to be inspected as inspection image data, the estimation unit 130 inputs the inspection image to the learned neural network and performs inference. The feature map acquisition unit 120 acquires, as the feature map of the inspection image, the output from the intermediate layer in the inference process. The abnormality determination unit 160 performs abnormality determination on the feature map of the inspection image input from the feature map acquisition unit 120 using each inspection reference value included in the inspection model stored in the inspection model storage unit 230. In the example of FIG. 5, since the average and covariance matrix for each element of the M feature maps are included in the inspection model, the abnormality determination unit 160 calculates the Mahalanobis distance between the feature map of the inspection image and each average and dispersion covariance matrix included in the inspection model, and normalizes it over the entire feature map. Then, if there is a portion where the Mahalanobis distance is equal to or greater than a predetermined threshold value, it is determined that there is an abnormality in that feature map. This abnormality determination process is performed (M times) for the number of the average and dispersion covariance matrices. Thereafter, ensemble inference based on the results of the M abnormality determinations is performed. For example, when performing majority vote inference as ensemble inference, if the number of times determined to be normal is greater than the number of times determined to be abnormal, it is determined to be normal. Conversely, in the case of the opposite, it is determined to be abnormal. When performing exact match inference as ensemble inference, if all the results are determined to be normal, it is determined to be normal, and in other cases, it is determined to be abnormal.

[0039] When the abnormality determination unit 160 determines that there is an abnormality in the inspection image, it may further determine the abnormal location within the inspection image. In this case, the abnormality determination unit 160 creates a map in which the elements determined to be abnormal in each feature map are set to 1 (white) and the other elements are set to 0 (black), and creates an abnormality degree map obtained by performing image processing on the created map so as to have the same resolution as the inspection image. Then, a logical sum is taken for each pixel of the M created abnormality degree maps to obtain the final abnormality degree map. In the abnormality degree map, since the locations where abnormalities are detected are displayed in white, there is an advantage that the abnormal locations can be grasped at a glance.

[0040] The output unit 170 outputs the discrimination result by the abnormality discrimination unit 160 in a predetermined method. The output unit 170 may, for example, perform display output of the abnormality discrimination result to the display device 70. Also, it may perform storage output to a storage area such as the external device 72, the RAM 13, or the non-volatile memory 14. Furthermore, it may perform transmission output to the industrial machine 3, the fog computer 6, or the cloud server 7 via the network 5.

[0041] The appearance inspection device 1 according to the present embodiment having the above configuration performs a plurality of abnormality discriminations using an inspection model including a plurality of previously created inspection reference values based on the feature map of the inspection image acquired from the intermediate layer of the learned neural network. Then, by performing ensemble inference based on the results of the plurality of abnormality discriminations, the abnormality of the inspection target is discriminated. By using ensemble inference, it becomes possible to ensure comprehensiveness in appearance inspection. For example, by using majority vote inference for ensemble inference, an improvement in the accuracy of appearance inspection can be expected. Also, by setting that it is regarded as normal when all the discrimination results match, it can be expected to reduce false positives in the inspection.

[0042] As a modified example of the appearance inspection device 1 according to the present embodiment, the feature map acquisition unit 120 may create a plurality of inspection images by performing a predetermined image expansion process on the inspection image, and output the feature maps of the created plurality of inspection images to the abnormality determination unit 160 as the feature map of the inspection image. In this case, the abnormality determination unit 160 performs abnormality determination using the inspection model for each feature map, and performs ensemble inference processing on all the determination results. FIG. 6 is a diagram showing the flow of determining the abnormality of the inspection image by the appearance inspection device 1 according to this modified example. In the example of FIG. 6, an example using rotation processing as the image expansion process is shown. As illustrated in FIG. 6, the feature map acquisition unit 120 creates an image obtained by rotating the inspection image by 90°, an image obtained by rotating the inspection image by 180°, and an image obtained by rotating the inspection image by 270° by a known image expansion process. Then, the feature maps of the respective images are acquired. The abnormality determination unit 160 performs abnormality determination using the inspection model based on each feature map. Then, ensemble inference is performed using the (M×4) abnormality determination results obtained as a result, and the final abnormality determination result is obtained. In this example, it is also possible to create an abnormality degree map in the same manner as described above. The abnormality degree maps obtained based on the respective rotated images may be rotated in the reverse direction, and a logical sum may be taken for each pixel. Examples of the image expansion process include color tone conversion, lightness conversion, and brightness conversion.

[0043] The appearance inspection device 1 according to this modified example can increase the discrimination results that serve as materials for ensemble inference by performing an image expansion process on the inspection image. As a result, an improvement in the accuracy of ensemble inference according to the purpose can be expected.

[0044] As described above, the embodiments of the present invention have been described. However, the present invention is not limited to only the examples of the above-described embodiments, and can be implemented in various forms by making appropriate changes. For example, in the above-described embodiments, the embodiment for learning and the embodiment for inference are described as separate embodiments, but it is also possible to implement these embodiments as one embodiment. In this case, the appearance inspection apparatus 1 may be provided with a function for switching the operation mode so that it can operate in the learning stage and the appearance inspection stage, respectively.

Description of Reference Numerals

[0045] 1 Appearance inspection apparatus 3 Industrial machine 4 Imaging device 5 Network 6 Fog computer 7 Cloud server 11 CPU 12 ROM 13 RAM 14 Non-volatile memory 15 Interface 17, 18, 20, 21 Interface 22 Bus 70 Display device 71 Input device 72 External device 100 Machine learning device 101 Processor 102 ROM 103 RAM 104 Non-volatile memory 110 Data acquisition unit 120 Feature map acquisition unit 130 Estimation unit 140 Group division unit 150 Inspection model creation unit 160 Abnormality determination unit 170 Output unit 200 Data storage unit 210 Model storage unit 220 Feature map storage unit 230 Inspection model storage unit

Claims

1. An appearance inspection apparatus using a learned neural network, comprising: a data acquisition unit that acquires an image indicating a normal state as a learning image; a feature map acquisition unit that acquires a plurality of feature maps indicating features of the learning image output in an intermediate layer of the learned neural network; a group division unit that divides the acquired plurality of feature maps into a predetermined number of groups; an inspection model creation unit that creates a predetermined inspection reference value as an inspection model for each group divided by the group division unit; an inspection model storage unit that stores the inspection model created by the inspection model creation unit; An appearance inspection apparatus comprising the above.

2. The data acquisition unit acquires an image to be inspected as an inspection image, the feature map acquisition unit acquires a feature map indicating features of the inspection image output in an intermediate layer of the learned neural network, an abnormality determination unit that performs ensemble inference using the inspection reference value for each group based on the feature map indicating the features of the inspection image, and determines an abnormality in the inspection image; an output unit that outputs a determination result by the abnormality determination unit; The appearance inspection apparatus according to claim 1, comprising the above.

3. The feature map acquisition unit further performs at least one image expansion process on the inspection image, acquires a feature map of the image subjected to the image expansion process, the abnormality determination unit performs ensemble inference using the inspection reference value for each group based on the feature map of the inspection image and the feature map of the inspection image subjected to the image expansion process, and determines an abnormality in the inspection image. The appearance inspection apparatus according to claim 2.

4. The image expansion process is an image rotation process. The appearance inspection apparatus according to claim 3.

5. A method executed by an appearance inspection apparatus using a learned neural network, comprising: acquiring an image indicating a normal state as a learning image; acquiring a plurality of feature maps indicating features of the learning image output in an intermediate layer of the learned neural network; dividing the acquired plurality of feature maps into a predetermined number of groups; creating a predetermined inspection reference value as an inspection model for each divided group; storing the created inspection model; A method for executing the above.

6. A method executed by an appearance inspection apparatus using a learned neural network, comprising: A step of acquiring an image to be inspected as an inspection image; A step of acquiring a feature map indicating the features of the inspection image output in the intermediate layer of the learned neural network; Based on the feature map indicating the features of the inspection image, performing ensemble inference using the inspection reference values for each group created by the method according to claim 5, and discriminating abnormalities in the inspection image; An output unit that outputs the result of the discrimination; A method for executing.

7. A computer-readable recording medium recording a program for operating an appearance inspection apparatus using a learned neural network, A data acquisition unit that acquires an image indicating a normal state as a learning image, A feature map acquisition unit that acquires a plurality of feature maps indicating the features of the learning image output in the intermediate layer of the learned neural network, A group division unit that divides the acquired plurality of feature maps into a predetermined number of groups, An inspection model creation unit that creates a predetermined inspection reference value as an inspection model for each group divided by the group division unit, An inspection model storage unit that stores the inspection model created by the inspection model creation unit, A computer-readable recording medium recording a program for operating a computer as the above.

8. The data acquisition unit acquires an image to be inspected as an inspection image, The feature map acquisition unit acquires a feature map indicating the features of the inspection image output in the intermediate layer of the learned neural network, Furthermore, An abnormality discrimination unit that performs ensemble inference using the inspection reference values for each group based on the feature map indicating the features of the inspection image, and discriminates abnormalities in the inspection image; An output unit that outputs the discrimination result by the abnormality discrimination unit, A computer-readable recording medium recording the program according to claim 7 for operating a computer as the above.

Images