Calculator for determining abnormality and method for determining abnormality

Abstract

To make it possible to perform highly accurate abnormality determination for an unknown abnormal image in abnormality determination using an auto-encoder even if learning is performed using only non-defective images.SOLUTION: In a learning step of learning parameters of an auto-encoder using a non-defective product learning image {f_i} obtained by imaging a non-defective product inspection object, a dimensional compression rate Rd of a code layer of the auto-encoder is maximized using an abnormality determination accuracy rate Rc of the non-defective product learning image {f_i} as an evaluation value so that the abnormality determination accuracy rate Rc is the maximum or a predetermined threshold value th or more, or an input signal amount to the code layer or an output signal amount from the code layer is decreased.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an abnormality determination computer and an abnormality determination method. [Background technology]

[0002] The present invention relates to a computer and method for determining anomaly based on machine learning, and provides a computer and method for automatically determining whether the object is of good quality or abnormal by inputting an inspection image acquired by imaging an object to be inspected into an autoencoder, estimating a good product image corresponding to the inspection image, and comparing the inspection image with the estimated good product image.

[0003] In many industrial products, including machinery, metals, chemicals, food, and textiles, appearance inspections are widely carried out to evaluate various aspects of workmanship, such as shape defects, assembly defects, adhesion of foreign matter, internal defects and criticality, surface scratches, spots, dirt, etc. Generally, most of these appearance inspections have been performed by visual judgment by inspectors.

[0004] On the other hand, with the increasing demand for mass production and quality improvement, the inspection costs and the burden on inspectors are increasing. In addition, sensory inspection based on human senses requires particularly high experience and skill. Issues include the personal nature of the inspection, such as different evaluation values ​​depending on the inspector and different results each time the inspection is performed, and reproducibility. There is a strong demand for the automation of inspection to address issues such as the cost, skill, and personal nature of inspection.

[0005] In recent years, the performance of machine learning has improved dramatically with the proposal of deep network models such as Convolutional Neural Network (CNN), and many visual inspection methods based on machine learning have been proposed.

[0006] In anomaly judgment, which judges whether an object to be inspected is passable or abnormal based on machine learning, it is ideal to collect a large number of passable and abnormal images of the object to be inspected in advance and train the judgment engine.

[0007] However, in industrial production lines, abnormalities generally occur rarely and abnormal patterns are diverse. Therefore, it is very difficult to comprehensively collect many abnormal images in advance and use them for learning. For this reason, a method has been disclosed in which a neural network called an autoencoder is used to learn using only images of good products (called good product learning), and an abnormality determination is made for the inspection object (for example, Patent Document 1).

[0008] An autoencoder is a network that reduces (dimensionally compresses) an input image using an encoder, and then restores it to the original size using a decoder. By training an autoencoder to input a good image and output the same good image, it is expected that when an abnormal image with defects or other anomalies is input, a good image without anomalies (estimated good image) will be output. Abnormalities can be detected by taking the difference between the inspection image and the estimated good image. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2020-160616 A Summary of the Invention [Problem to be solved by the invention]

[0010] In the abnormality judgment using an autoencoder, as typified by the above-mentioned Patent Document 1, there are cases where an appropriate estimated good product image cannot be estimated. This is because the autoencoder is trained to estimate the same good product image from a good product image, and when an unknown abnormal image not used in the learning is input, a good estimated good product image, that is, an estimated good product image in which an abnormal part has been removed from the input abnormal image, is not necessarily output. There are many variations in abnormal patterns that occur in actual manufacturing lines, etc., and it is difficult in reality to learn all of them in advance. Therefore, a mechanism is needed that can perform highly accurate abnormality judgment even for unknown abnormal images by learning using only good product images.

[0011] The present invention has been made in consideration of the above problems, and has an object to provide an anomaly determination computer and an anomaly determination method that are capable of performing highly accurate anomaly determination for unknown abnormal images, even when learning using only images of non-defective products, in anomaly determination using an autoencoder. [Means for solving the problem]

[0012] In order to solve the above problem, an anomaly judgment computer according to one aspect of the present invention is an anomaly judgment computer that judges an anomaly of an inspection object, the anomaly judgment computer having a processor, the processor processing a non-defective product learning image {f_i} (i=1, ..., Nf, The method includes a good-quality training image acquisition step of acquiring a good-quality training image f'' of an object to be inspected, a good-quality image estimation step of inputting the inspection image f'' to the autoencoder to estimate an estimated good-quality image h'', and a judgment step of performing an anomaly judgment from the inspection image f'' and the estimated good-quality image h'' by a comparison inspection engine, and the learning step is characterized in that the dimensional compression rate Rd of the code layer of the autoencoder is maximized or the amount of input signals to the code layer or the amount of output signals from the code layer is reduced so that the anomaly judgment accuracy rate Rc of the good-quality training image {f_i} is set as an evaluation value and is maximized or equal to or exceeds a predetermined threshold value th. Effect of the Invention

[0013] According to the present invention, it is possible to realize an anomaly determination computer and an anomaly determination method that are capable of performing highly accurate anomaly determination for unknown abnormal images, even when learning using only non-defective product images, in anomaly determination using an autoencoder. [Brief description of the drawings]

[0014] [Figure 1] 1 is a diagram illustrating an automatic abnormality determination system and an overall processing sequence according to a first embodiment. [Diagram 2] FIG. 13 is a diagram illustrating non-defective product learning and abnormality determination in an autoencoder. [Diagram 3] FIG. 4 is a diagram illustrating a learning method for an autoencoder and a comparison inspection engine in the automatic abnormality determination system according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a learning method of an autoencoder in the automatic abnormality determination system according to the first embodiment. [Diagram 5] FIG. 1 is a diagram illustrating an example of a learning method of an autoencoder. [Figure 6] FIG. 4 is a diagram illustrating an example of a learning method of an autoencoder in the automatic abnormality determination system according to the first embodiment. [Figure 7] 3 is a diagram illustrating inputs and outputs of nodes in a neural network in the automatic abnormality determination system according to the first embodiment. FIG. [Figure 8] FIG. 11 is a diagram illustrating a learning method of an autoencoder using an abnormal image in the automatic abnormality determination system according to the second embodiment. [Figure 9] 13A to 13C are diagrams showing variations of artificially generated abnormality images in the automatic abnormality determination system according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing a result of an abnormality determination in a general method. [Figure 11] 6A to 6C are diagrams illustrating results of abnormality determination in the automatic abnormality determination system according to the embodiment. [Figure 12] 1 is a diagram illustrating a hardware configuration of an automatic abnormality determination system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are not necessarily essential to the solution of the invention.

[0016] In the drawings explaining the embodiments, the same reference numerals are used to designate parts having the same functions, and repeated explanations thereof will be omitted.

[0017] In the following description, an expression such as "xxx data" may be used as an example of information, but the information may have any data structure. In other words, to show that the information does not depend on the data structure, "xxx data" may be referred to as "xxx table." Furthermore, "xxx data" may simply be referred to as "xxx." In the following description, the structure of each piece of information is an example, and the information may be divided and stored, or combined and stored.

[0018] In the following explanation, the process may be described with a "program" as the subject, but since a program is executed by a processor (e.g., a CPU (Central Processing Unit)) to perform a defined process using storage resources (e.g., a memory) and / or a communication interface device (e.g., a port) as appropriate, the subject of the process may also be the program. Processes explained with a program as the subject may also be processes performed by a processor or a computer having that processor.

[0019] The abnormality determination computer and the abnormality determination method of this embodiment have the following configuration, for example.

[0020] (1) The method includes a good-quality learning image acquisition step of imaging a good-quality inspection object to acquire good-quality learning images {f_i} (i = 1, ..., Nf, Nf: number of images), a learning step of learning parameters of an autoencoder using the good-quality learning images {f_i}, an inspection image acquisition step of imaging the inspection object to acquire an inspection image f'', a good-quality image estimation step of inputting the inspection image f'' to an autoencoder to estimate an estimated good-quality image h'', and a judgment step of performing an anomaly judgment from the inspection image f'' and the estimated good-quality image h'' by a comparison inspection engine, wherein the learning step is characterized in that the dimensional compression rate Rd of the encoded code layer is maximized or the amount of input signals to the code layer or the amount of output signals from the code layer is reduced so that the anomaly judgment accuracy rate Rc of the good-quality learning images {f_i} is set as an evaluation value and the Rc becomes maximum or equal to or greater than a predetermined threshold value th.

[0021] To elaborate on this feature, the anomaly detection engine consists of a "good product image estimation engine" that inputs the inspection image into an autoencoder to obtain an estimated good product image, and a "comparison inspection engine" that compares the inspection image with the estimated good product image to determine anomalies.

[0022] The comparison inspection engine may be a rule-based judgment algorithm or a machine learning-based judgment algorithm such as a neural network.

[0023] It is unclear what kind of estimated good image will be output when an inspection image containing an unlearned abnormal pattern is input to an autoencoder that has been trained to estimate good images from good images. It is also unclear whether anomalies in the inspection image can be correctly identified based on a comparison inspection between the inspection image and the estimated good image.

[0024] An example of a failure to output an estimated good image is when an autoencoder can restore an abnormal image that it inputs. In other words, even if the input image is an abnormal image, if the output image is also an abnormal image similar to the input image, it will be erroneously determined to be a good image because the difference between the input image and the output image is small. Therefore, an autoencoder should be trained so that it can restore only good images and cannot restore other images.

[0025] One of the reasons why the autoencoder can restore images other than the good learning images is that the expressive power of the autoencoder is too high. Therefore, in this embodiment, the network structure is optimized to increase the dimensional compression rate Rd of the code layer as much as possible, thereby reducing the expressive power of the autoencoder.

[0026] A high dimensional compression rate Rd means that the input image is represented by fewer nodes (features) in the code layer, reducing the possibility that the autoencoder will be able to represent images other than the good-quality training images it has learned.

[0027] At that time, the dimensional compression rate Rd is maximized using the anomaly judgment accuracy rate Rc as an evaluation value. Usually, an autoencoder is trained so that a good product learning image is restored with higher accuracy, but it is not necessary to obtain a highly accurate estimated good product image in anomaly judgment. In other words, even if an estimated good product image restored from a good product image is deteriorated, it is sufficient that the comparison inspection engine can perform a correct anomaly judgment. In addition, the anomaly judgment accuracy rate Rc is used as an evaluation value when determining the expressive power of the autoencoder, but as described above, it may be determined so that the anomaly judgment accuracy rate Rc is maximized, or it may be determined so that the anomaly judgment accuracy rate Rc is equal to or greater than a predetermined threshold value th.

[0028] (2) In the learning step, the dimensionality compression rate of the code layer is maximized by reducing the number of nodes or channels contained in each channel of the code layer.

[0029] As described above, in this embodiment, the anomaly determination accuracy rate Rc is used as an evaluation value to reduce the expressive power of the autoencoder, thereby making it impossible to restore images other than non-defective images.

[0030] One way to reduce the expressive power of an autoencoder is to maximize the dimensionality reduction ratio Rd of the code layer. Specifically, the number of nodes or channels in each channel of the code layer is reduced. Since the total number of nodes in the code layer can be regarded as the number of features after dimensionality reduction by the encoder, reducing the total number of nodes increases the dimensionality reduction ratio Rd.

[0031] (3) In the learning step, the amount of input signals to the code layer is reduced by the j-th input signal {x_ij} (i=1,…,Nn, Nn: number of nodes in the code layer, j=1,…,Nx_i, Nx_i: number of input signals to the ith node in the code layer) Enter This is done by setting the value of the weight {w_ij | (i,j)∈Φ} (Φ: a set of numbers (i,j) of weights w_ij that set their values ​​to zero) for at least one node out of the weights w_ij of the input signal (number of input signals) to zero or a value close to zero, and the set Φ is characterized by being a set of numbers (i,j) ({Φ={(i,j)} | x_ij≦x_min}) for which the input signal {x_ij} is equal to or less than a certain value x_min when a good-quality training image {f_i} is input to an autoencoder.

[0032] A supplementary explanation of this feature is provided. As a method for reducing the expressive power of an autoencoder using the anomaly judgment accuracy rate Rc as an evaluation value, it is also possible to reduce the values ​​of some input signals to nodes included in the code layer. In other words, to reduce the expressive power of an autoencoder, a direct method of reducing the number of nodes as described above is acceptable, but instead of choosing between leaving or deleting a certain node, it is possible to reduce the influence of that node in the restoration by reducing the value of the input signal to that node, thereby continuously reducing the expressive power of the autoencoder.

[0033] (4) The comparative inspection engine comprises a differential image generation engine that obtains a differential image between an inspection image f'' and an estimated good image h'' that is output by inputting the inspection image f'' into an autoencoder, and a differential image evaluation engine that judges whether the inspection image f'' is good or abnormal based on the differential image, and is characterized in that in a learning step, processing parameters of the differential image generation engine or the differential image evaluation engine are learned based on the accuracy rate of the abnormality judgment.

[0034] We will provide some additional information on this feature. There are several possible methods for obtaining the difference image between the inspection image f'' and the estimated good image h''. It is possible to simply take the difference between the pixel values ​​of both images, but even slight discrepancies between images that are not abnormal will result in differences, and this can lead to erroneous determinations in the abnormality determinations of the difference image evaluation engine.

[0035] For this reason, it is possible to obtain a difference image using, for example, the local perturbation finite difference method (see, for example, Matsuyama Yukio, Iwata Takashi, Kubota Hitoshi, and Nakagawa Yasuo, "Precise Visual Inspection of LSI Wafer Patterns by Local Perturbation Pattern Matching Method," IEICE Transactions on Electronics, Information and Communication Engineers, D, J72-D2 (12), pp.2041-2050 (1989)). The local perturbation finite difference method is a method in which one image is perturbed in the XY plane and in the brightness direction for each local area of ​​the other image to match them, and the parts that cannot be matched are taken as the difference. This makes it possible to eliminate slight discrepancies between images. The perturbation width is one of the processing parameters of the comparison inspection, and the amount of discrepancy that is eliminated changes depending on its value.

[0036] There are also several possible methods for the difference image evaluation engine that judges whether the inspection image f'' is good or bad based on the difference image. For example, it is possible to make the judgment based on the distribution of brightness values ​​in the pixel area where there is a difference, but the size of the distribution or brightness value that is judged to be bad is also one of the processing parameters of the comparison inspection.

[0037] Here, so-called rule-based algorithms have been given as examples of the difference image generation engine and the difference image evaluation engine, but machine learning-based algorithms may also be used. Even machine learning-based algorithms have processing parameters for learning.

[0038] This embodiment is characterized in that these processing parameters are optimized using the accuracy rate of anomaly determination as an evaluation value. As described in items (1) to (3), a process is performed to reduce the expressive power of the autoencoder so that the autoencoder is unable to restore images other than those of non-defective products as much as possible, but there is a limit to how much the expressive power can be reduced while maintaining the accuracy rate of anomaly determination. If the expressive power is reduced, even if the input image is a good image, the estimated good image by the autoencoder will deteriorate, and a difference will occur between the input image and the estimated good image. In order to prevent this slight difference from being erroneously determined as an abnormality, the processing parameters of the comparison inspection engine are of By adjusting the autoencoder, it becomes easier to generate a good-product image estimation engine that can further reduce the expressive power and restore only good products. By simultaneously optimizing the hyperparameters that reduce the expressive power of the autoencoder and the processing parameters of the comparison inspection engine, the performance of both complement each other, enabling more accurate anomaly detection.

[0039] (5) A step of acquiring good learning images {f_i} (i = 1, ..., Nf, Nf: number of images) by imaging good inspection objects, and an abnormal learning image {f'_i} (i = 1, ..., Nf, the learning step of acquiring an abnormality learning image f'' by imaging the object to be inspected; the good product image estimation step of inputting the inspection image f'' to the autoencoder to estimate an estimated good product image h''; and a judgment step of performing an abnormality judgment from the inspection image f'' and the estimated good product image h'' by a comparison inspection engine, wherein the learning step is characterized in that the dimensional compression rate Rd of the encoded code layer is maximized or the amount of input signal to the code layer or the amount of output signal from the code layer is reduced so that the anomaly judgment accuracy rate Rc of the good product learning image {f_i} and the abnormal learning image {f'_i} is set as an evaluation value and the Rc becomes maximum or equal to or greater than a predetermined threshold value th.

[0040] A supplementary note on this feature: Abnormal patterns are diverse, and it is generally difficult to comprehensively collect many abnormal images in advance. However, if some abnormal images are available, even if only a portion of them, they can be learned as abnormal learning images {f'_i}, allowing the autoencoder parameters to be learned more appropriately.

[0041] This embodiment is characterized in that the autoencoder is trained using the anomaly detection accuracy rate Rc as an evaluation value, but by using both the non-defective product training image {f_i} and the abnormality training image {f'_i}, it is possible to estimate the anomaly detection accuracy rate Rc more accurately.

[0042] (6) The abnormal learning image {f'_i} is a pseudo-abnormal image generated by a pseudo-abnormal image generation engine, which generates the image by applying image processing to the non-defective learning image {f_i}.

[0043] As mentioned above, collecting abnormal images is generally difficult, and there are cases where it is not possible to prepare abnormal learning images {f'_i} in advance in item (5), or even if several images can be prepared, it is not possible to comprehensively collect many abnormal patterns.

[0044] Therefore, by applying image processing to the good product learning images {f_i}, various pseudo-abnormal images are generated and used for learning. The image processing in the pseudo-abnormal image generation engine simulates abnormalities that may occur in actual inspection objects, and examples of such variations include applying foreign matter, defects such as chips, scratches, surface spots or stains, shape deformation, etc. to the good product learning images. EXAMPLES

[0045] 1. Automatic anomaly detection system and overall processing sequence FIG. 1 shows an automatic abnormality determination system and an overall processing sequence according to this embodiment.

[0046] The processing sequence in the automatic anomaly determination system of this embodiment is roughly divided into a learning phase (100) and an inspection phase (101).

[0047] In the learning phase (100), a good inspection object (102) is imaged to obtain good learning images {f_i} (i=1,...,Nf, Nf: number of images) for learning (103). The images are obtained by capturing digital images of the surface or interior of the inspection object using an imaging device such as a CCD camera, optical microscope, charged particle microscope, ultrasonic inspection device, or X-ray inspection device.

[0048] As another example of "obtaining," it is possible to simply receive an image captured by another system and store it in a storage resource of the automatic anomaly determination system.

[0049] Next, a good image estimation engine utilizing an autoencoder is trained using the good training images {f_i} (104). That is, the network parameters of the autoencoder are determined by training so that a good image is output when an input image is input.

[0050] When learning is performed using a good-product training image that is a good product, the good-product training image is basically input, and the network parameters are determined so that the same good-product training image is output.

[0051] The network parameters include "hyperparameters" such as the network structure, and "model parameters" such as the weights (coupling coefficients) between network nodes and biases, and are determined (105, 106). The hyperparameters include the number of layers in the network, the number of nodes in each layer, the activation function, the learning rate, the termination condition for learning, etc.

[0052] In addition, the parameters of a comparison inspection engine that compares the input image and output image of the non-defective image estimation engine to determine whether the input image is non-defective or defective are learned (107).

[0053] In the inspection phase (101), an image of an actual inspection object (102) is captured (103), an inspection image f'' is obtained, and an abnormality determination is performed (111).

[0054] Specifically, the inspection image f'' is input to an autoencoder based on the hyperparameters (108) and model parameters (109) determined in the learning phase (100), and an estimated good image h'' is estimated (112). Furthermore, the inspection image f'' and the estimated good image h'' are input to a comparison inspection engine based on the comparison inspection process parameters (110) determined in the learning phase (100), and it is determined (113) whether the inspection image f'' is good or defective. If necessary, an inspector checks this determination result (114).

[0055] 2. Estimation of good image using autoencoder The details of the estimation of a good image by the autoencoder will be described with reference to FIG.

[0056] An autoencoder (see, for example, Geoffrey E. Hinton and RR Salakhutdinov, "Reducing the Dimensionality of Data with Neural Networks," Science, 313 (5786), pp.504-507 (2006)) is a so-called dimensionality reduction type network, in which an input image is input to the input layer (207), compressed once by the encoder (205), and then converted into a code layer (209) that is composed of fewer nodes than the input layer.

[0057] Each value of the node {g_i} (i=1,...,Ng, Ng: number of nodes in the code layer) in the code layer can be regarded as a feature of the input image with reduced dimensions. This code layer is restored by a decoder (206) and then returned to the output layer (211) with the same dimensions as the input image.

[0058] As shown in the processing sequence 200 of the learning phase (good product learning), if the input image f_i is learned as a good product image (202) during learning, and the same good product image is learned to be reconstructed from the good product image (202) as the output image h_i (212), the autoencoder is expected to become a good product image estimation engine that can reconstruct only good products.

[0059] By using the autoencoder trained in this way, it is possible to perform abnormality judgment on an actual inspection object in the processing sequence 201 in the inspection phase (abnormality judgment).

[0060] That is, if the input inspection image f''_i is a good product, the autoencoder outputs an image similar to the inspection image f''_i as the estimated good product image: h''_i, so there is no difference between the inspection image f''_i and the estimated good product image: h''_i, and the inspection image f''_i is determined to be a good product. On the other hand, if the input inspection image f''_i (203) contains an abnormality (204), such as the attachment of a foreign substance, the autoencoder cannot accurately restore the abnormal part, and an estimated good product image: h''_i (213) without the abnormality is output. As a result, a difference occurs between the inspection image f''_i and the estimated good product image: h''_i in the abnormal part, and the inspection image f''_i is determined to be an abnormal product.

[0061] However, a correct anomaly determination result is not always obtained in the inspection phase 201 of FIG. 2 described here. This is because abnormal images are not explicitly learned in the learning phase, and it is unclear what the restoration result will be for an unknown abnormal image. In other words, since learning is performed in the learning phase 200 to restore a non-defective image from a non-defective image, it is expected that a non-defective image will be restored from a non-defective image in the inspection phase 201 as well, but whether or not an abnormal image is restored depends on the type of abnormal pattern. In actual experiments, cases have been confirmed in which an autoencoder is able to restore an abnormal image and a correct anomaly determination cannot be performed. This embodiment solves this problem.

[0062] 3. Optimizing the Autoencoder 3.1 Basic processing In order to solve the above-mentioned problems, this embodiment includes a good-quality learning image acquisition step of imaging a good-quality inspection object to acquire good-quality learning images {f_i} (i = 1, ..., Nf, Nf: number of images), a learning step of learning parameters of an autoencoder using the good-quality learning images {f_i}, an inspection image acquisition step of imaging the inspection object to acquire an inspection image f'', a good-quality image estimation step of inputting the inspection image f'' to an autoencoder to estimate an estimated good-quality image h'', and a judgment step of performing an abnormality judgment from the inspection image f'' and the estimated good-quality image h'' by a comparison inspection engine, and is characterized in that in the learning step, the dimensional compression rate Rd of the encoded code layer is maximized, or the amount of input signal to the code layer or the amount of output signal from the code layer is reduced so that the abnormality judgment accuracy rate Rc of the good-quality learning images {f_i} is set as an evaluation value and the Rc becomes maximum or equal to or greater than a predetermined threshold value th.

[0063] This feature will be explained with reference to Fig. 3. The anomaly judgment engine consists of a "good product image estimation engine (301)" that inputs an inspection image into an autoencoder to obtain an estimated good product image, and a "comparison inspection engine (311)" that compares the inspection image (the good product learning image during learning) with the estimated good product image to judge anomalies.

[0064] The comparison inspection engine may be a rule-based judgment algorithm or a machine learning-based judgment algorithm such as a neural network.

[0065] As mentioned above, it is unclear what kind of estimated good image will be output when an inspection image containing an unlearned abnormal pattern is input to an autoencoder that has been trained to estimate a good image from a good image. It is also unclear whether an abnormality in the inspection image can be correctly determined based on a comparison inspection between the inspection image and the estimated good image.

[0066] An example of a failure to output an estimated good image is when an autoencoder can restore an abnormal image that it inputs. In other words, even if the input image is an abnormal image, if the output image is also an abnormal image similar to the input image, it will be erroneously determined to be a good image because the difference between the input image and the output image is small. Therefore, an autoencoder should be trained so that it can restore only good images and cannot restore other images.

[0067] One of the reasons why the autoencoder can restore images other than the good product learning images is that the expressive power of the autoencoder is too high. Therefore, in this embodiment, the network structure is optimized to reduce the expressive power of the autoencoder so as to increase the dimensional compression rate Rd of the code layer (306) as much as possible.

[0068] A high dimensional compression rate Rd means that the input image is represented by fewer nodes (features) in the code layer (306), reducing the possibility that the autoencoder will be able to represent images other than the good-quality training images it has learned.

[0069] At this time, the dimensionality reduction rate Rd is maximized using the anomaly detection accuracy rate Rc (calculated in process 315) obtained from the detection results of the comparison inspection engine (311) as an evaluation value. In other words, the number of nodes in the code layer, which is one of the hyperparameters 309, is minimized.

[0070] Normally, an autoencoder is trained so that a good product learning image can be restored with high accuracy, but it is not necessary to obtain a highly accurate estimated good product image for anomaly detection. In other words, even if the estimated good product image (310) restored from the good product image (300) is degraded, it is sufficient that the comparison inspection engine (311) can perform correct anomaly detection.

[0071] In addition, the anomaly detection accuracy rate Rc is used as an evaluation value when determining the expressive power of the autoencoder. A feature of this embodiment is that the expressive power is determined so as to improve the anomaly detection accuracy rate Rc. As a specific method for this, as described above, the expressive power may be determined so as to maximize the anomaly detection accuracy rate Rc, or so as to make the anomaly detection accuracy rate Rc equal to or greater than a predetermined threshold value th (315).

[0072] 3.2 Specific examples of code layer optimization We will explain two main methods for optimizing the code layer.

[0073] The first feature is that in the learning step, the dimensionality compression rate of the code layer is maximized by reducing the number of nodes or channels included in each channel of the code layer.

[0074] A supplementary explanation of this feature will be provided. As described above, in the present invention, the expressive power of the autoencoder is reduced using the anomaly judgment accuracy rate Rc as an evaluation value, so that images other than non-defective images cannot be restored. One method for reducing the expressive power of the autoencoder is to maximize the dimensional compression rate Rd of the code layer. Specifically, the number of nodes or channels included in each channel of the code layer is reduced. Since the total number of nodes in the code layer can be regarded as the number of features after dimensional compression by the encoder, reducing the total number of nodes increases the dimensional compression rate Rd.

[0075] Figure 4 shows a schematic diagram of a network structure in which each layer is expressed one-dimensionally, and explains a specific method for reducing the number of nodes in the code layer.

[0076] Figure 4(a) shows an example of a network structure with a relatively large number of nodes in the code layer and high expressiveness of the autoencoder. The number of nodes in the input layer (402) is 10. However, this is a schematic diagram, and when the inspection image is input as is, the number of nodes in the input layer will be equal to the number of pixels in the inspection image.

[0077] The number of nodes in this input layer (402) is dimensionally compressed by the encoder (400) to six nodes in the code layer (404). In other words, the dimensional compression rate Rd is (10-6) / 10×100=40%.

[0078] In contrast, the number of nodes in the code layer (411) in the network structure of FIG. 4(b) is 4. The dimensional compression rate Rd is (10-4) / 10×100=60%, which is a high compression rate. Therefore, if the anomaly detection accuracy rate Rc is improved in FIG. 4(b) compared to FIG. 4(a), or if the anomaly detection accuracy rate Rc in FIG. 4(b) is equal to or greater than a predetermined threshold value th, the network structure of FIG. 4(b) should be adopted to reduce the possibility of expressing images other than non-defective learning images, and improve the anomaly detection accuracy rate for unknown abnormal images.

[0079] In FIG. 4(c), the number of nodes in the code layer (418) is 2. The dimensional compression rate Rd is (10-2) / 10×100=80%, which is even higher. If the anomaly detection accuracy rate Rc is improved further in FIG. 4(c) than in FIG. 4(b), or if the anomaly detection accuracy rate Rc in FIG. 4(b) is equal to or higher than a predetermined threshold value th, the network structure in FIG. 4(c) should be adopted, and a network structure with an even higher compression rate should also be evaluated. However, if such performance cannot be confirmed, it is possible that the expressive power of the network has been lowered too much, making it impossible to accurately restore the image of a non-defective product, and therefore FIG. 4(c) cannot be adopted.

[0080] In FIG. 4, both the encoders (400, 407, 414) and the decoders (401, 408, 415) are two-layer networks (for example, in the encoder 400, a network continuing from the input layer 402 to the intermediate layer 403 and the code layer 404), but it is also possible to use networks with more than two layers.

[0081] Although the number of nodes in the code layers (404, 411, 418) is exemplified as discrete values ​​of 6, 4, and 2, any value can be verified and selected in actual optimization. Furthermore, along with optimizing the number of nodes in the code layers, the network structure in the encoder and decoder (number of layers, number of nodes, number of channels, combination of nodes connected by edges, etc.) can be changed.

[0082] Figure 5 shows a schematic diagram of a network structure in which each layer is represented in two dimensions, and explains a specific method for reducing the number of nodes in the code layer.

[0083] Figure 4 shows an image in which the pixel values ​​of the inspection image are connected in a row and input as a one-dimensional array, but as shown in Figure 5, an inspection image with Nx1 pixels horizontally and Ny1 pixels vertically can be input directly to an input layer (L1) consisting of an Nx1 × Ny1 two-dimensional node array.

[0084] It is common to obtain the intermediate layer (L2) by performing convolution and pooling on the input layer (L1). Pooling makes the image size of the intermediate layer, Nx2×Ny2, smaller than Nx1×Ny1.

[0085] In this case, multiple images are generated by providing multiple convolution filters. The number of images is called the number of channels, and the number of channels in the intermediate layer (L2) is Nc2.

[0086] Similarly, the code layer (L3) is generated from the intermediate layer (L2), and the code layer (L3) has an image size of Nx3 × Ny3 and a number of channels of Nc3. This is the encoder. The network that goes from the code layer (L3) through the intermediate layer (L4) to the output layer (L5), which has the original image size (Nx1 = Nx5, Ny1 = Ny5), is the decoder.

[0087] In this embodiment, the expressive power of the autoencoder is reduced using the anomaly judgment accuracy rate Rc as an evaluation value. One method for reducing this is to reduce the number of nodes (Nx3 × Ny3) or the number of channels (Nc3) contained in each channel of the code layer.

[0088] Second, the reduction in the amount of input signals to the code layer in the learning step is due to the reduction in the amount of j-th input signal {x_ij} (i=1,…,Nn, Nn: number of nodes in the code layer, j=1,…,Nx_i, Nx_i: number of input signals to the ith node in the code layer) Enter Among the weights w_ij of the input signal (number of input signals), the value of the weights {w_ij | (i,j)∈Φ} (Φ: a set of the numbers (i,j) of the weights w_ij whose values ​​are set to zero) for at least one node is set to zero or close to zero. Determine The set Φ is characterized by being a set of numbers (i, j) ({Φ={(i,j)} | x_ij≦x_min}) such that when a good training image {f_i} is input to the autoencoder, the input signal {x_ij} is less than or equal to a certain value x_min.

[0089] A supplementary explanation of this feature is provided. As a method for reducing the expressive power of an autoencoder using the anomaly judgment accuracy rate Rc as an evaluation value, it is also possible to reduce the values ​​of some input signals to nodes included in the code layer. In other words, to reduce the expressive power of an autoencoder, a direct method of reducing the number of nodes as described above is acceptable, but instead of choosing between leaving or deleting a certain node, it is possible to reduce the influence of that node in the restoration by reducing the value of the input signal to that node, thereby continuously reducing the expressive power of the autoencoder.

[0090] This will be specifically described with reference to FIG. 6 and FIG.

[0091] 6 shows an example of an autoencoder that estimates a non-defective image in this embodiment. It consists of five layers, L1 to L5, and L3 is a code layer (604).

[0092] FIG. 7 shows the first (i=1) node (610) in the code layer and three nodes (607, 608, 609) in the intermediate layer (L2) that are connected to the first node (610) in the code layer by edges.

[0093] The amount of signal u output from the node (610) is calculated from the amount of signals {x_1j} (j=1, 2, 3) output from the three nodes (607, 608, 609) by the following formula: u = A(t) (Eq. 1) t = w_11*x_11+w_12*x_12+w_13*x_13+b_1 (Formula 2) Here, A() is an activation function, and although various nonlinear functions have been proposed, ReLU (ramp function) has been widely used recently. Also, {w_1j} is the weight of the edge connecting the j-th node in the hidden layer (L2) and the 1st node in the code layer (L3), and b_1 is the bias amount, both of which are model parameters (109) optimized by learning.

[0094] In this embodiment, the expressiveness of the autoencoder is reduced by intentionally setting some of the weights to zero or values ​​close to zero. The magnitude of the signal amount can be used as one of the criteria for setting the weights to be small. When a good-quality image is input to the input layer (602), it can be considered that an edge with a small signal amount does not contribute much to expressing the good-quality image. For example, if the signal amount x_12 in FIG. 7 is small, the value of the weight w_12 of the edge 701 is corrected to be smaller than the value set by learning.

[0095] Here, we have explained how to set the weights from the intermediate layer (L2) to the code layer (L3) to a small value, but the weights between any layers can be set to small values ​​in the same way. The smaller the value is set and the closer it is to zero, the more the edge to which that weight is applied is invalidated. As a result, the expressiveness of the autoencoder decreases, and the same effect as increasing the dimensionality reduction rate Rd can be obtained without reducing the total number of nodes.

[0096] 4. Adjustment of processing parameters of comparison inspection engine The comparative inspection engine comprises a differential image generation engine that obtains a differential image between an inspection image f'' and an estimated good image h'' that is output by inputting the inspection image f'' into an autoencoder, and a differential image evaluation engine that judges whether the inspection image f'' is good or abnormal based on the differential image, and is characterized in that in a learning step, processing parameters of the differential image generation engine or the differential image evaluation engine are learned based on the accuracy rate of the abnormality judgment.

[0097] This feature will be described with reference to FIG. 3. In this embodiment, the inspection image f''_i is input to a good image estimation engine (autoencoder) and an estimated good image h''_i is output. The inspection image f''_i and the estimated good image h''_i are compared by a comparative inspection engine (311) to determine whether the inspection image f''_i is good or abnormal.

[0098] Various algorithms can be used for the comparison. As an example, consider a comparison inspection engine consisting of a difference image generation engine (312) and a difference image evaluation engine (313).

[0099] There are several possible methods for obtaining the difference image between the inspection image f'' and the estimated good image h'' in the difference image generation engine (312). Although it is possible to simply take the difference between the pixel values ​​of both images, even a slight discrepancy between images that are not abnormal will result in a difference, which may lead to an erroneous determination in the abnormality determination by the difference image evaluation engine. For this reason, it is possible to obtain the difference image using, for example, the local perturbation difference method (see, for example, Yukio Matsuyama, Takashi Iwata, Hitoshi Kubota, and Yasuo Nakagawa, "Precise Visual Inspection of LSI Wafer Patterns by Local Perturbation Pattern Matching Method," IEICE Transactions on Electronics, Information and Communication Engineers, J72-D2 (12), pp.2041-2050 (1989)).

[0100] The local perturbation difference method is a method in which one image is perturbed in the XY plane and in the brightness direction for each local area of ​​the other image to match them, and the parts that cannot be matched are treated as differences. This method can eliminate slight discrepancies between images. The perturbation width is one of the processing parameters (314) of the comparison inspection, and the amount of discrepancy that is eliminated changes depending on the value of the perturbation width.

[0101] There are also several possible methods for the differential image evaluation engine (313) that judges whether the inspection image f'' is good or bad based on the differential image. For example, it is possible to make the judgment based on the distribution of brightness values ​​in pixel areas with differences, but the size of the distribution or brightness value that is judged to be bad is also one of the processing parameters (314) of the comparison inspection.

[0102] Here, so-called rule-based algorithms are given as examples of the difference image generation engine (312) and the difference image evaluation engine (313), but machine learning-based algorithms may also be used. Even machine learning-based algorithms have processing parameters (314) for learning.

[0103] This embodiment is characterized in that these processing parameters are optimized using the anomaly judgment accuracy rate as an evaluation value. As described above, a process is performed to reduce the expressiveness of the autoencoder so that the autoencoder cannot restore images other than good products as much as possible, but there is a limit to how much the expressiveness can be reduced while maintaining the anomaly judgment accuracy rate. If the expressiveness is reduced, even if the input image is good, the good product image estimated by the autoencoder will deteriorate, and a difference will occur between the input image and the estimated good product image.

[0104] By adjusting the processing parameters of the comparison inspection engine so that this slight difference is not erroneously judged as an anomaly, the autoencoder can further reduce the expressive power, making it easier to generate a good-product image estimation engine that can restore only good products. By simultaneously optimizing the hyperparameters (309) that reduce the expressive power of the autoencoder and the processing parameters (314) of the comparison inspection engine in process 315, the performance of both is complemented, making it possible to judge anomalies with higher accuracy.

[0105] One of the features of this embodiment is that the difference image generation engine uses a method that does not detect slight discrepancies as differences when comparing the inspection image f'' and the estimated good image h'', such as the local perturbation difference method. This is because, as will be described later with reference to FIG. 11, this embodiment reduces the expressive power of the autoencoder even at the expense of some degree of restoration accuracy of the estimated good image, thereby improving the anomaly determination accuracy rate Rc by reliably detecting abnormal patterns. For this reason, it is essential to use a method that has low sensitivity to slight differences that occur in good products.

[0106] Therefore, according to this embodiment, in anomaly judgment using an autoencoder, it is possible to realize an anomaly judgment computer and an anomaly judgment method that are capable of performing highly accurate anomaly judgment for unknown abnormal images, even when learning is performed using only images of non-defective products. EXAMPLES

[0107] 5. Optimization of anomaly detection method using pseudo-abnormal images A step of acquiring good learning images {f_i} (i=1,...,Nf, Nf: number of images) by imaging a good inspection target, and an abnormal learning image {f'_i} (i=1,...,Nf, the learning step of acquiring an abnormality learning image f'' by imaging the object to be inspected; the good product image estimation step of inputting the inspection image f'' to the autoencoder to estimate an estimated good product image h''; and a judgment step of performing an abnormality judgment from the inspection image f'' and the estimated good product image h'' by a comparison inspection engine, wherein the learning step is characterized in that the dimensional compression rate Rd of the encoded code layer is maximized or the amount of input signal to the code layer or the amount of output signal from the code layer is reduced so that the anomaly judgment accuracy rate Rc of the good product learning image {f_i} and the abnormal learning image {f'_i} is set as an evaluation value and the Rc becomes maximum or equal to or greater than a predetermined threshold value th.

[0108] This feature will be explained with reference to Figure 8. Abnormal patterns are diverse, and it is generally difficult to comprehensively collect many abnormal images in advance. However, if some abnormal images are available, even if only a portion of them, they can be learned as abnormal learning images {f'_i}, making it possible to learn the autoencoder parameters more appropriately.

[0109] In other words, when the good learning image f_i (800) and the estimated good image h_i (812) are as similar as possible, and when the abnormal learning image f'_i (802) and the estimated good image h'_i (813) diverge as much as possible in the abnormal parts, the accuracy rate of abnormality determination Rc tends to improve.

[0110] This embodiment is characterized in that the autoencoder is trained using the anomaly detection accuracy rate Rc as an evaluation value, but by using both the non-defective training image {f_i} (800) and the abnormal training image {f'_i} (802), it is possible to estimate the anomaly detection accuracy rate Rc more accurately.

[0111] The abnormal learning image {f'_i} is a pseudo-abnormal image generated by a pseudo-abnormal image generation engine, which generates the image by performing image processing on the non-defective learning image {f_i}.

[0112] This feature will be explained with reference to Fig. 8. As mentioned above, collecting abnormal images is generally difficult, and there are cases where abnormal learning images {f'_i} (802) cannot be prepared in advance, or even if several images can be prepared, it is not possible to comprehensively collect many abnormal patterns. Therefore, the feature of this method is that various pseudo-abnormal images are generated by performing image processing on non-defective learning images {f_i} (800), and are used for learning.

[0113] In this case, since a good product learning image (800) exists that was used to artificially generate the abnormal product learning image f'_i, in learning, if the estimated good product image h'_i (813) and the corresponding good product learning image (800) are as similar as possible, the accuracy rate of abnormality judgment Rc tends to improve.

[0114] The image processing in the pseudo-abnormal image generation engine (801) simulates abnormalities that may occur in an actual inspection target. As shown in FIG. 9, the image processing can be performed by applying variations to the non-defective product learning image (FIG. 9(a)), such as the attachment of foreign matter (FIG. 9(b)), defects such as chips (FIG. 9(c)), scratches (FIG. 9(d)), surface spots or stains (FIG. 9(e)), and shape deformation (FIG. 9(f)).

[0115] Also, in the anomaly judgment using the pseudo-abnormal images shown in Fig. 8, similarly to Fig. 3 where no pseudo-abnormal images were used, the learning step is characterized in that hyperparameters (811) (the number of nodes in the code layer, etc.) and comparison inspection processing parameters (817) (for example, processing parameters of the differential image generation engine (815) or the differential image evaluation engine (816), etc.) are learned based on the anomaly judgment accuracy rate Rc. In addition, the learning step is characterized in that image processing parameters (method of generating pseudo-abnormal images, number of images to be generated, etc.) in the pseudo-abnormal image generation engine (801) are learned based on the anomaly judgment accuracy rate Rc.

[0116] The hyperparameters (811), the comparison inspection process parameters (817), and the process parameters of the pseudo-abnormal image generation engine (801) can be simultaneously optimized in process 818.

[0117] As a practical matter, it is difficult to comprehensively collect variations in abnormal patterns, so it is necessary to improve the performance of anomaly detection by teaching good products. On the other hand, improving performance for unknown abnormal patterns is an inherently difficult task. Therefore, the best we can do with only the information on good product images is to generate an autoencoder that restores only good products. To achieve this, it is important to make full use of adjustments to all processing parameters, such as the processing parameters of the comparison inspection engine and the processing parameters of the pseudo-abnormal image generation engine, to reduce the expressiveness of the code layer as much as possible while maintaining the anomaly detection accuracy rate Rc.

[0118] Therefore, according to this embodiment, it is possible to realize an anomaly determination computer and an anomaly determination method that are capable of performing highly accurate anomaly determination for unknown abnormal images in an anomaly determination using an autoencoder, even when learning is performed using only images of non-defective products.

[0119] 6. Example of processing results The effect of this embodiment will be shown using an example of a processing result.

[0120] Figure 10 shows the results of anomaly detection processing using a typical autoencoder.

[0121] In the learning phase shown in the upper part of Fig. 10, a good product training image f (Fig. 10(a1)) is input to an autoencoder to output an estimated good product image h (Fig. 10(a2)), and the model parameters of the autoencoder are learned so that the good product training image f and the estimated good product image h are similar. As a result, the simple difference between the good product training image f and the estimated good product image h (Fig. 10(a3)) almost disappears, and the good product image is correctly determined to be a good product.

[0122] In the inspection phase, the autoencoder trained in this way is used to judge whether an inspection image f'' is abnormal. Depending on the type of abnormality pattern, a correct abnormality judgment may be made, as shown in the middle of Figure 10. That is, when an inspection image f'' (Figure 10(b1)) containing a foreign object (1000) as an abnormal part is input to an autoencoder, the abnormal part cannot be restored in an estimated good image h'' (Figure 10(b2)) that is output. In this case, the abnormal part is extracted as the difference between the inspection image f'' and the estimated good image h'' (Figure 10(b3)), and therefore the inspection image f'' is correctly judged to be abnormal.

[0123] On the other hand, as shown in the lower part of Fig. 10, there are cases where a correct anomaly determination cannot be made. That is, when an inspection image f'' (Fig. 10(c1)) containing a defect (1001) as an abnormal part is input to an autoencoder, the abnormal part can be restored in an estimated good image h'' (Fig. 10(c2)) that is output. In this case, since the abnormal part is not extracted as the difference between the inspection image f'' and the estimated good image h'' (Fig. 10(c3)), the inspection image f'' is erroneously determined to be a good product.

[0124] This is because an autoencoder is trained to estimate an identical good image from a good image, and when an unknown abnormal image that has not been used in training is input, it does not necessarily output a good estimated good image, that is, an estimated good image with the abnormal parts removed from the input abnormal image.

[0125] In contrast, the present embodiment is characterized in that the expressive power of the autoencoder is reduced using the anomaly determination accuracy rate Rc as an evaluation value, thereby making it impossible to restore images other than non-defective images.

[0126] FIG. 11 shows the results of anomaly determination processing by the autoencoder of this embodiment.

[0127] In the learning phase shown in the upper part of Fig. 11, a good-quality training image f (Fig. 11(a1)) is input to the autoencoder to output an estimated good-quality image h (Fig. 11(a2)). Using the anomaly detection accuracy rate Rc as an evaluation value, the hyperparameters are determined so as to reduce the expressive power of the autoencoder as much as possible.

[0128] As a result of reducing the expressiveness, the restoration accuracy of the estimated good product image h (FIG. 11(a2)) also decreases (for example, the shape becomes slightly distorted compared to the good product learning image), and even if the input image is a good product, a difference (FIG. 11(a3)) may occur between the good product learning image f and the estimated good product image h. However, in this embodiment, the restoration accuracy in the good product learning is not a direct evaluation value, and the anomaly judgment accuracy rate Rc is the evaluation value, so there is no problem as long as the value of Rc is high. In other words, whether the input image is good or abnormal is not simply determined by the presence or absence of a simple difference between the good product learning image f and the estimated good product image h, but is determined by the comparison inspection engine, so it is sufficient to adjust the processing parameters of the comparison inspection engine so that correct anomaly judgment can be made even if a slight difference occurs.

[0129] By lowering the expressive power of the autoencoder even if the restoration accuracy is somewhat sacrificed while maintaining the anomaly judgment accuracy rate Rc, the possibility of being unable to restore unknown abnormal images increases. As a result, as shown in the middle of Fig. 11, it is not only possible to correctly determine that an inspection image f'' (Fig. 11(b1)) containing a foreign object (1100) that could not be restored in the conventional way in the estimated good image h'' (Fig. 11(b2)) is abnormal, but as shown in the bottom of Fig. 11, a defect (1101) that could be restored in the conventional way cannot be restored in the estimated good image h'' (Fig. 11(c2)), and the inspection image f'' (Fig. 11(c1)) can be correctly determined as abnormal.

[0130] 7.Hardware Configuration of Automatic Anomaly Detection System FIG. 12 shows an abnormality determination system that realizes the abnormality determination method described in the above embodiment.

[0131] The abnormality determination system is composed of the above-mentioned imaging device and a computer (abnormality determination computer). An example of the imaging device has already been described.

[0132] The computer is a component that processes the abnormality determination method described in this embodiment, and includes the following: *Processor: Examples of processors include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field-Programmable Gate Array), but other components may be used as long as they can process the anomaly determination method. *Storage resource: Examples of storage resources include RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), and non-volatile memory (such as flash memory). The storage resource may store a program (called an abnormality determination program) that causes a processor to execute the abnormality determination method described in the above embodiments. *GUI device: Examples of GUI (Graphical User Interface) devices include displays and projectors, but other devices may also be used as long as they are capable of displaying a GUI. *Input device: Examples of input devices include a keyboard, a mouse, and a touch panel, but other devices may be used as long as they are components that can accept operations from a user. Also, the input device and the GUI device may be integrated into one device. *Communication interface device: Examples of communication interfaces include USB (Universal Serial Bus), Ethernet (registered trademark), and Wi-Fi (registered trademark), but other interface devices may be used as long as they can directly receive images from an imaging device or allow a user to send the images to a computer. In addition, a portable non-volatile storage medium (such as a flash memory, a DVD (Digital Versatile Disk), a CD-ROM, or a Blu-ray disk) on which the images are stored may be connected to the communication interface, and the images may be stored in a computer. The above is the hardware configuration of the computer. Note that the anomaly determination system may include a plurality of computers and a plurality of imaging devices.

[0133] The abnormality determination program may be stored in the computer via the following route: *The abnormality determination program is stored on a portable non-volatile storage medium, and the program is distributed to a computer by connecting the medium to a communication interface. *The abnormality determination program is distributed to the computer by a program distribution server. The program distribution server has a storage resource that stores the abnormality determination program, a processor that performs a distribution process to distribute the abnormality determination program, and a communication interface device that can communicate with the communication interface device of the computer.

[0134] 8.Other This concludes the description of the embodiments. As mentioned above, the embodiments described thus far do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are not necessarily essential to the solution of the invention.

[0135] As an example, in this embodiment, two-dimensional image data is used as input information, but the present invention can also be applied to cases where one-dimensional signals such as received ultrasonic waves, or three-dimensional volume data acquired by a laser range finder, etc. are used as input information. Also, this embodiment can be applied to cases where there are multiple input images and multiple types of estimated evaluation values ​​(the evaluation engine has multiple inputs and multiple outputs).

[0136] In addition, in the embodiment, the inspection image is judged to be either good or bad, but it is also possible to estimate the "degree of abnormality" as a continuous quantity, which quantifies the degree of abnormality according to the size or criticality of the abnormality. If the degree of abnormality is zero or a small value, the product is considered to be good, but if it is a large value, it is considered to be abnormal. Furthermore, as the degree of abnormality increases, it can be determined that a serious abnormality has occurred.

[0137] As an estimation method, it is possible to calculate the degree of anomaly in the comparative inspection engine according to the degree of difference between the inspection image and the estimated good image. In addition, it is possible to learn the hyperparameters (309, 811) and the comparative inspection process parameters (314, 817) so that the learning image has the taught degree of anomaly, by the user instructing the degree of anomaly of the learning image in the learning phase.

[0138] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium storing it constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

[0139] Furthermore, the program code for realizing the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), and Python.

[0140] Furthermore, all or part of the program code of the software that realizes the functions of each embodiment may be stored in advance in the computer's memory resources, or, if necessary, may be stored in the computer's memory resources from a non-transitory memory device of another device connected to the network, or from a non-transitory storage medium via an external I / F (not shown) provided on the computer.

[0141] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read out and execute the program code stored in the storage means or storage medium.

[0142] In the above-mentioned embodiment, the control lines and information lines are shown as those considered necessary for the explanation, and not all the control lines and information lines are shown in the product. All the components may be connected to each other. [Explanation of symbols]

[0143] 102... Inspection object 300, 800... Good product learning image 301, 803... Good product image estimation engine 306, 404, 411, 418, 604... Code layer 310, 812, 813... Estimated good product image 311... Comparison inspection engine 312, 815... Difference image generation engine 313, 81 6 ...Differential image evaluation engine 801...Pseudo abnormal image generation engine 802...Abnormal learning image

Claims

1. An abnormality determination computer that determines an abnormality in an inspection object, comprising: The anomaly determination computer has a processor, the processor comprising: a good-product learning image acquisition step for acquiring good-product learning images {f_i} (i=1, . . . , Nf, Nf: number of images) obtained by capturing images of the inspection object that is a good product; A learning step of learning parameters of an autoencoder using the good product learning image {f_i}; an inspection image acquisition step of acquiring an inspection image f″ obtained by capturing an image of the inspection object; a good-quality image estimation step of inputting the inspection image f'' to the autoencoder to estimate an estimated good-quality image h''; a determination step of performing an anomaly determination using a comparison inspection engine based on the inspection image f'' and the estimated non-defective image h''; Run In the learning step, the dimensional compression rate Rd of the code layer of the autoencoder is maximized or the amount of input signals to the code layer or the amount of output signals from the code layer is reduced so that the anomaly judgment accuracy rate Rc of the non-defective learning image {f_i} is set as an evaluation value and the anomaly judgment accuracy rate Rc is maximized or equal to or greater than a predetermined threshold value th.

2. A computer for determining an anomaly, comprising:

2. 2. The abnormality determination computer according to claim 1, The dimensional compression ratio Rd of the code layer in the learning step is maximized by reducing the number of nodes or channels included in each channel of the code layer.

2. A computer for determining an anomaly, comprising:

3. 2. The abnormality determination computer according to claim 1, the reduction in the amount of input signals to the code layer in the learning step is performed by setting the value of weights w_ij of the j-th input signal {x_ij} (i=1, ..., Nn, Nn: number of nodes in the code layer, j=1, ..., Nx_i, Nx_i: number of input signals to the i-th node in the code layer) for at least one of the nodes to zero or a value close to zero, {w_ij | (i, j) ∈ Φ} (Φ: set of numbers (i, j) of weights w_ij whose values ​​are set to zero); The set Φ is a set of numbers (i, j) ({Φ={(i, j)} | x_ij≦x_min}) for which the input signal {x_ij} is equal to or smaller than a certain value x_min when the good product learning image {f_i} is input to the autoencoder.

2. A computer for determining an anomaly, comprising:

4. 2. The abnormality determination computer according to claim 1, The comparison inspection engine includes: a difference image generation engine for obtaining a difference image between the inspection image f'' and the estimated non-defective image h''; a differential image evaluation engine for determining whether the inspection image f'' is a non-defective product or an abnormal product based on the differential image; having In the learning step, a processing parameter of the differential image generation engine or the differential image evaluation engine is learned based on the anomaly determination accuracy rate Rc.

2. A computer for determining an anomaly, comprising:

5. 5. The abnormality determination computer according to claim 4, The difference image generation engine determines the difference image using a local perturbation difference method; The processing parameters of the difference image generation engine include at least a perturbation width.

2. A computer for determining an anomaly, comprising:

6. 2. The abnormality determination computer according to claim 1, An abnormal learning image acquisition step of acquiring abnormal learning images {f'_i} (i=1, ..., Nf, Nf: number of images), In the learning step, parameters of the autoencoder are learned using the non-defective training images {f_i} and the abnormal training images {f'_i}; Furthermore, in the learning step, the dimensional compression ratio Rd of the encoded code layer is maximized or the amount of input signals to the code layer or the amount of output signals from the code layer is reduced so that the anomaly determination accuracy rate Rc of the non-defective learning image {f_i} and the abnormal learning image {f'_i} is set as an evaluation value and the anomaly determination accuracy rate Rc is maximized or equal to or greater than a predetermined threshold value th.

2. A computer for determining an anomaly, comprising:

7. 7. The abnormality determination computer according to claim 6, The abnormal learning image acquisition step includes a pseudo abnormal image generation engine, The pseudo-abnormal image generation engine generates the abnormal learning image {f'_i}, which is a pseudo-abnormal image, by performing image processing on the non-defective learning image {f_i}.

2. A computer for determining an anomaly, comprising:

8. 8. The abnormality determination computer according to claim 7, In the learning step, a processing parameter of the pseudo-abnormal image generation engine is learned based on the abnormality determination accuracy rate Rc.

2. A computer for determining an anomaly, comprising:

9. 7. The abnormality determination computer according to claim 6, The comparison inspection engine includes: a difference image generation engine for obtaining a difference image between the inspection image f'' and the estimated non-defective image h''; a differential image evaluation engine for determining whether the inspection image f'' is a non-defective product or an abnormal product based on the differential image; having In the learning step, a processing parameter of the differential image generating engine or the differential image evaluating engine is learned based on the anomaly determination accuracy rate.

2. A computer for determining an anomaly, comprising:

10. An abnormality determination method executed by an abnormality determination computer that determines an abnormality in an inspection object, comprising: a good-product learning image acquisition step for acquiring good-product learning images {f_i} (i=1, . . . , Nf, Nf: number of images) obtained by capturing images of the inspection object that is a good product; A learning step of learning parameters of an autoencoder using the good product learning image {f_i}; an inspection image acquisition step of acquiring an inspection image f″ obtained by capturing an image of the inspection object; a good-quality image estimation step of inputting the inspection image f'' to the autoencoder to estimate an estimated good-quality image h''; a determination step of performing an anomaly determination using a comparison inspection engine based on the inspection image f'' and the estimated non-defective image h''; Run In the learning step, the dimensional compression rate Rd of the code layer of the autoencoder is maximized or the amount of input signals to the code layer or the amount of output signals from the code layer is reduced so that the anomaly judgment accuracy rate Rc of the non-defective learning image {f_i} is set as an evaluation value and the anomaly judgment accuracy rate Rc is maximized or equal to or greater than a predetermined threshold value th. The abnormality determination method according to the present invention is characterized in that

11. The method for determining an abnormality according to claim 10, The dimensional compression ratio Rd of the code layer in the learning step is maximized by reducing the number of nodes or channels included in each channel of the code layer. The abnormality determination method according to the present invention is characterized in that

12. The method for determining an abnormality according to claim 10, the reduction in the amount of input signals to the code layer in the learning step is performed by setting the value of weights w_ij of the j-th input signal {x_ij} (i=1, ..., Nn, Nn: number of nodes in the code layer, j=1, ..., Nx_i, Nx_i: number of input signals to the i-th node in the code layer) for at least one of the nodes to zero or a value close to zero, {w_ij | (i, j) ∈ Φ} (Φ: set of numbers (i, j) of weights w_ij whose values ​​are set to zero); The set Φ is a set of numbers (i, j) ({Φ={(i, j)} | x_ij≦x_min}) for which the input signal {x_ij} is equal to or smaller than a certain value x_min when the good product learning image {f_i} is input to the autoencoder. The abnormality determination method according to the present invention is characterized in that

13. The method for determining an abnormality according to claim 10, The comparison inspection engine includes: a difference image generation engine for obtaining a difference image between the inspection image f'' and the estimated non-defective image h''; a differential image evaluation engine for determining whether the inspection image f'' is a non-defective product or an abnormal product based on the differential image; having In the learning step, a processing parameter of the differential image generation engine or the differential image evaluation engine is learned based on the anomaly determination accuracy rate Rc. The abnormality determination method according to the present invention is characterized in that

14. The method for determining an abnormality according to claim 13, The difference image generation engine determines the difference image using a local perturbation difference method; The processing parameters of the difference image generation engine include at least a perturbation width. The abnormality determination method according to the present invention is characterized in that

15. The method for determining an abnormality according to claim 10, An abnormal learning image acquisition step of acquiring abnormal learning images {f'_i} (i=1, ..., Nf, Nf: number of images), In the learning step, parameters of the autoencoder are learned using the non-defective training images {f_i} and the abnormal training images {f'_i}; Furthermore, in the learning step, the dimensional compression ratio Rd of the encoded code layer is maximized or the amount of input signals to the code layer or the amount of output signals from the code layer is reduced so that the anomaly determination accuracy rate Rc of the non-defective learning image {f_i} and the abnormal learning image {f'_i} is set as an evaluation value and the anomaly determination accuracy rate Rc is maximized or equal to or greater than a predetermined threshold value th. The abnormality determination method according to the present invention is characterized in that

Images