Index selection device, information processing device, information processing system, inspection device, inspection system, index selection method, and index selection program

Abstract

[Problem] To provide an index selection device, an information processing device, an information processing system, an inspection device, an inspection system, an index selection method, and an index selection program that improve the accuracy of defect detection. [Solution] An index selection device having a score calculation unit 230, and an index selection unit 240. The score calculation unit 230 calculates, on the basis of a plurality of pieces of input data of normal products and defective products, and a plurality pieces of reference data corresponding to the input data, anomaly scores for normal products and defective products by a plurality of indices. The index selection unit 240 selects, depending on the anomaly scores of normal products and defective products, one of the plurality of indices.

Description

[Technical Field]

[0001] The present invention relates to an index selection device, an information processing device, an information processing system, an inspection device, an inspection system, an index selection method, and an index selection program. [Background technology]

[0002] There are known inspection devices equipped with image processing functions that use input images of production objects (workpieces) on a production line to determine whether the workpieces are good or bad. These inspection devices use image processing algorithms to extract feature values from the input images and determine whether the workpieces are good or bad based on threshold values that separate good and bad products.

[0003] In this regard, Patent Document 1 below discloses that in order to improve inspection accuracy, rules and thresholds (inspection logic) that define the method for determining whether a workpiece is good or bad are dynamically set in accordance with fluctuations in the production environment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-327848 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology of Patent Document 1, features are extracted by directly performing image processing on an input image, and the pass / fail status of the workpiece is determined based on the extracted features. Therefore, if the workpiece has a defect and the features of this defect are similar to the structural features that a good workpiece originally has, it is not possible to distinguish whether the extracted feature is a defect or a structural feature, resulting in a problem of low defect detection accuracy.

[0006] The present invention has been made to solve such problems, and aims to provide an index selection device, an information processing device, an information processing system, an inspection device, an inspection system, an index selection method, and an index selection program that improve the accuracy of defect detection. [Means for solving the problem]

[0007] The above-mentioned problems of the present invention are solved by the following means.

[0008] (1) a data reconstruction unit that generates reconstructed data corresponding to the input data based on the input data using a generative model trained using input data of a plurality of non-defective products; A plurality of input data of good and bad products; Multiple The aforementioned Reconstruction and an index selection unit that selects one of the plurality of indexes in accordance with the abnormality scores of the non-defective and defective products.

[0009] (2) The index selection device described in (1) above, wherein the abnormality score is the difference between the result of calculating the index using the input data and the result of calculating the index using the reference data for each of good and defective products.

[0010] (3) The indicator selection device described in (1) or (2) above, wherein the indicator selection unit selects the indicator that produces the largest difference between the distribution of abnormality scores of the good products and the distribution of abnormality scores of the defective products for the plurality of input data and the reference data.

[0012] ( 4 ) the indicator selection unit uses feature quantities of a plurality of input data of good products and defective products as explanatory variables, and uses the difference between the distribution of anomaly scores of the good products and the distribution of anomaly scores of the defective products as an objective variable, and selects one of the plurality of indicators using a trained model trained to maximize the difference, 3 ) An index selection device according to any one of the above.

[0013] ( 5) The input data is color image data, Score calculation section calculates an abnormality score based on hue and / or saturation as the index, 4 ) An index selection device according to any one of the above.

[0014] ( 6 ) an input data acquisition unit that acquires input data, the input data acquired by the input data acquisition unit, and reference data corresponding to the input data; (1) to (5) above and a score calculation unit that calculates an anomaly score based on the index selected by the index selection device according to any one of claims 1 to 4.

[0015] ( 7 )the above( 6 10. An inspection device having a determination unit that determines whether a product is good or defective based on the anomaly score output by the information processing device described in

[0016] ( 8 )the above( 6 and a display device that displays the abnormality score calculated by the score calculation unit.

[0017] ( 9 )the above( 7 and a display device that displays the determination result by the determination unit.

[0018] ( 10 ) (a) generating, based on input data of a plurality of non-defective products, reconstructed data corresponding to the input data using a generative model trained using the input data; A plurality of input data of good and bad products; Multiple The aforementioned Reconstruction Calculating anomaly scores for good and defective products based on multiple indicators ( b ) and selecting one of the plurality of indices according to the abnormality scores of the non-defective and defective products ( c ) and an index selection method having:

[0019] ( 11) The anomaly score is the difference between the result of calculating the index using the input data and the result of calculating the index using the reference data for each of the non-defective and defective products. 10 ) The index selection method described in

[0020] ( 12 ) the step ( c ), an index that maximizes the difference between the distribution of abnormality scores of the non-defective products and the distribution of abnormality scores of the defective products for the plurality of input data and the reference data is selected. 10 )or( 11 ) The index selection method described in

[0022] ( 13 ) the step ( c ) in which feature quantities of a plurality of input data of good products and defective products are used as explanatory variables, and the difference between the distribution of anomaly scores of the good products and the distribution of anomaly scores of the defective products is used as an objective variable, and a trained model trained to maximize the difference is used to select one of the plurality of indicators, 10 )~( 12 ) The index selection method according to any one of the above.

[0023] ( 14 ) The input data is color image data, and in step (b), an abnormality score is calculated using hue and / or saturation as the index. 10 )~( 13 ) The index selection method according to any one of the above.

[0024] ( 15 )the above( 10 )~( 14 ) An index selection program for causing a computer to execute the processes included in the index selection method described in any one of the above. [Effects of the Invention]

[0025] According to the index selection device of the present invention, one of multiple indexes is selected according to the anomaly scores of good and defective products, thereby obtaining an index that is effective for distinguishing between good and defective products. The information processing device then calculates the anomaly score using the index selected by the index selection device, allowing the user to confirm the location and area of the anomaly in the inspection object from the anomaly score map displayed on the display. Furthermore, the inspection device can improve the accuracy of determining whether the inspection object is good or defective based on the anomaly score calculated by the information processing device. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. 1 is a schematic block diagram illustrating a hardware configuration of an information processing apparatus according to an embodiment. [Figure 2] 10 is a functional block diagram illustrating the main functions of a control unit when the information processing device functions as an index selection device. FIG. [Figure 3] 3 is a schematic block diagram illustrating the functions of a control unit shown in FIG. 2. FIG. [Figure 4] 3 is a schematic diagram for explaining calculation of an abnormality score by a score calculation unit shown in FIG. 2. FIG. [Figure 5] 1 is a schematic diagram illustrating the calculation results of abnormality scores 1 to N using index parameters 1 to N. FIG. [Figure 6] 10 is a functional block diagram illustrating the main functions of a control unit when the information processing device functions as an information processing device. FIG. [Figure 7] 7 is a schematic block diagram illustrating the functions of a control unit shown in FIG. 6. [Figure 8] 8 is a functional block diagram illustrating the function of a score calculation unit shown in FIG. 7. [Figure 9] FIG. 2 is a functional block diagram illustrating the main functions of a control unit when the information processing device functions as an inspection device. [Figure 10] 10 is a flowchart illustrating a processing procedure of an index selection method according to an embodiment. [Figure 11] 1 is a flowchart illustrating a processing procedure of an inspection method according to an embodiment. [Figure 12] FIG. 10 is a schematic diagram for explaining prevention of overdetection. DETAILED DESCRIPTION OF THE INVENTION

[0027] Hereinafter, an index selection device, an information processing device, an information processing system, an inspection device, an inspection system, an index selection method, and an index selection program according to embodiments of the present invention will be described with reference to the drawings. Note that in the drawings, identical elements are assigned the same reference numerals, and duplicated explanations will be omitted.

[0028] Fig. 1 is a schematic block diagram illustrating the hardware configuration of an information processing device 100 according to an embodiment, and Fig. 2 is a functional block diagram illustrating the main functions of a control unit 110 when the information processing device 100 functions as an index selection device. Fig. 3 is a schematic block diagram illustrating the functions of the control unit 110 shown in Fig. 2, and Fig. 4 is a schematic diagram for explaining calculation of an abnormality score by the score calculation unit 230 shown in Fig. 2. Fig. 5 is a schematic diagram illustrating calculation results of abnormality scores 1 to N using index parameters 1 to N.

[0029] <Configuration of the index selection device> The information processing device 100 functions as an index selection device that calculates an anomaly score using multiple index parameters (indexes) based on multiple input images and multiple reference images corresponding to these input images, and selects one of the multiple index parameters depending on the anomaly score. The input images are labeled as either good products (normal inspection objects) or defective products (inspection objects containing defects or other abnormalities), and index parameters that are effective for distinguishing between good and defective products are selected by learning the index parameters (deep learning or machine learning). The index parameters can be set, for example, individually or in combination, based on color attributes of the input image and the reconstructed image, such as hue, saturation, and density, as well as object shape and size. Details of the method for calculating the anomaly score will be described later.

[0030] The inspection object is not particularly limited, but may be, for example, a part used in an industrial product. The inspection includes detecting abnormalities such as creases, bends, chips, scratches, and stains. The information processing device 100 acquires multiple input images of non-defective products (hereinafter also referred to as "normal input images"), learns these images as correct images, and generates a deep learning model (generative model) that generates a reconstructed image from the input image. The reference image may be a reconstructed image of the input image.

[0031] 1, the information processing device 100 includes a control unit 110, a communication unit 120, and an operation display unit 130. These components are connected to each other via a bus 101. The information processing device 100 may be, for example, a computer such as a personal computer or a server.

[0032] The control unit 110 includes a central processing unit (CPU) 111, a random access memory (RAM) 112, a read only memory (ROM) 113, and an auxiliary storage unit 114.

[0033] The CPU 111 executes programs such as an OS (Operating System), an index selection program, and an inspection program loaded in the RAM 112, and controls the operation of the information processing device 100. The index selection program and the inspection program are stored in advance in the ROM 113 or the auxiliary storage unit 114. The RAM 112 also stores data and the like temporarily generated by the processing of the CPU 111. The ROM 113 stores programs executed by the CPU 111, as well as data, parameters, and the like used in executing the programs. The auxiliary storage unit 114 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.

[0034] 2, the control unit 110 functions as an image acquisition unit 210, an image reconstruction unit 220, a score calculation unit 230, and an index selection unit 240 when the CPU 111 executes the index selection program. The image acquisition unit 210 functions as an input data acquisition unit, and acquires an input image (input data) by working in cooperation with the communication unit 120.

[0035] The image reconstruction unit 220 functions as a data reconstruction unit, and generates a reconstructed image (reconstructed data) as reference data based on the input image acquired by the image acquisition unit 210, using a generative model trained using input images of multiple non-defective products related to the same inspection target. More specifically, the image reconstruction unit 220 extracts feature amounts from the input image using a trained first neural network, and reconstructs (restores) the input image based on the extracted feature amounts to generate the reconstructed image.

[0036] The first neural network has a multi-layer convolutional neural network and is trained in advance by supervised learning so that the difference between the reconstructed image and the input image is eliminated or minimized during training.

[0037] The first neural network functions as a generative model having, for example, an encoder-decoder structure. In this embodiment, an autoencoder (AE) or a variational autoencoder (VAE) can be suitably used as the generative model. Since AE and VAE are well-known techniques, detailed description thereof will be omitted.

[0038] As shown in FIG. 3, the image reconstruction unit 220 has, as a generative model, a VAE including, for example, an encoder 221 and a decoder 222. The VAE extracts features of the input image to extract only essential elements of the input image, and then reconstructs the image using the extracted features to generate and output a reconstructed image from which non-essential elements of the input image have been removed. That is, since the VAE learns only with normal input images, it is configured to be able to generate feature values corresponding to normal input images. However, for input images of defective products, i.e., inspection targets containing abnormalities such as defects or scratches (hereinafter also referred to as "abnormal input images"), it is unable to generate feature values corresponding to the abnormalities, and therefore has no reproducibility.

[0039] In the example shown in FIG. 3, the input image includes an image of a component M1 to be inspected. The component M1 originally has a linear texture T1. Furthermore, if the component M1 is a defective product, it may include a scratch S1 that occurred during the manufacturing process, for example. When the reconstruction process for the input image is executed, a reconstructed image of the component M1 is output. The reconstructed image is an image in which only essential elements remain from the input image of the component M1, and unnecessary elements have been removed. The texture T1 is reconstructed because it is something that the component M1 originally had (an essential element). On the other hand, the scratch S1 is not reconstructed because it is abnormal (not an essential element).

[0040] In this way, the image reconstructing unit 220 generates and outputs a reconstructed image from which non-essential elements in the input image have been removed, so the difference between the input image and the reconstructed image is greater in the case of a defective product than in the case of a non-defective product.

[0041] Although the above description exemplifies a case where a reconstructed image of an input image is used as a reference image, a predetermined image of the same inspection object can also be used as a reference image instead of the reconstructed image. The predetermined image can be, for example, an image of a typical non-defective product of the same inspection object that is not used as the input image. However, if there is a difference in position or size (magnification) between the input image and the predetermined image, it is necessary to perform a correction process for the position or size. In contrast, when a reconstructed image is used as a reference image, the above correction process is not necessary.

[0042] The score calculation unit 230 calculates an abnormality score using multiple index parameters based on multiple input images of non-defective and defective products and multiple reconstructed images corresponding to the input images. As shown in Fig. 4, the score calculation unit 230 calculates abnormality scores 1 to N for non-defective and defective products using index parameters 1 to N based on, for example, multiple input images labeled as non-defective or defective products and multiple reconstructed images corresponding to these input images. The abnormality score may be the difference between the result of calculating the value of the index parameter using the input images and the result of calculating the value of the index parameter using the reconstructed images for each of the non-defective and defective products. As described above, in this embodiment, hue and saturation can be used as index parameters, which makes it possible to detect a defective area even when the detection target has a defect that includes a subtle color change that is difficult to detect using only brightness values.

[0043] FIG. 5 illustrates the distribution of abnormality score values for non-defective and defective products in an orthogonal coordinate system in which the horizontal axis represents the normalized abnormality score and the vertical axis represents the number of samples (input images) used to select the index parameters. The abnormality scores based on index parameters 1 to N correspond to abnormality scores 1 to N, respectively. Non-defective samples are distributed mainly in areas where the normalized abnormality scores are relatively small, while defective samples are distributed mainly in areas where the normalized abnormality scores are relatively large. The data for abnormality scores 1 to N is stored in the auxiliary storage unit 114.

[0044] The index selection unit 240 selects one of index parameters 1 to N according to the abnormality scores 1 to N of the non-defective and defective products. In the example shown in FIG. 5, for abnormality scores 1 and 2, the abnormality scores of the non-defective products (the shaded areas in the figure) and the abnormality scores of the defective products (the gray areas in the figure) overlap, making it difficult to separate the two. On the other hand, for abnormality score N, there is little overlap between the abnormality scores of the non-defective products and the abnormality scores of the defective products, making it easy to separate the two. Therefore, the index selection unit 240 selects the index parameter N that maximizes the difference between the distribution of abnormality scores of non-defective products and the distribution of abnormality scores of defective products. In this embodiment, the index parameter is selected by performing deep learning.

[0045] More specifically, the index selection unit 240 has, for example, a multi-layer convolutional neural network (second neural network) and performs deep learning (or machine learning) of index parameters for multiple input images of good and bad products related to the same inspection target. By inputting multiple input images of the same inspection target, an anomaly score is accumulated for each pair of input image and reference image, and the learning of the index parameters is deepened.

[0046] In this embodiment, the indicator selection unit 240 uses the feature quantities of multiple input images of good and defective products as explanatory variables, and the difference between the distribution of abnormality scores of good products and the distribution of abnormality scores of defective products (i.e., the distance between the statistics of abnormality scores of good products and the statistics of abnormality scores of defective products) as the objective variable, and trains the second neural network so as to maximize the difference (distance) to generate a trained model. The statistics can be, for example, a histogram.

[0047] Also, the index parameters may be configured to be learned so as to reduce the rate of false positives of non-defective products.

[0048] Furthermore, although an example of deep learning of index parameters has been described, the present invention is not limited to such a case. Regression analysis may be performed after setting explanatory variables and target variables in the same manner as in deep learning, and index parameters may be selected so that the difference (distance) is maximized.

[0049] Then, the index selection unit 240 selects one of the multiple index parameters (index parameter N in the example of FIG. 5) for one test object using the trained model.

[0050] The communication unit 120 is an interface circuit (for example, a LAN card) for communicating with an external device via a network.

[0051] The operation display unit 130 has an input unit and an output unit. The input unit includes, for example, a keyboard, a mouse, etc., and is used by the user to input characters using the keyboard, mouse, etc., and to enter various instructions (inputs) such as various settings. The output unit includes a display (display device) and displays input images, etc.

[0052] Although not shown, the inspection object is photographed by an imaging device such as a camera. The imaging device transmits image data of the photographed inspection object to the information processing device 100. The information processing device 100 acquires the image data as input images. Images of the inspection object photographed in advance by the imaging device are stored in a storage device external to the information processing device 100, and the information processing device 100 can also be configured to sequentially acquire a predetermined number of images of the inspection object stored in the storage device as input images.

[0053] For example, when the inspection object is a part of an industrial product, the imaging device is installed in the inspection process, captures an image of the imaging range that includes the inspection object, and outputs image data that includes the inspection object. The imaging device outputs, for example, black-and-white or color image data of the inspection object with a predetermined number of pixels (e.g., 128 pixels x 128 pixels).

[0054] <Configuration of information processing device and information processing system> Fig. 6 is a functional block diagram illustrating the main functions of the control unit 110 when the information processing device 100 functions as an information processing device. Fig. 7 is a schematic block diagram illustrating the functions of the control unit 110 shown in Fig. 6, and Fig. 8 is a functional block diagram illustrating the functions of the score calculation unit 330 shown in Fig. 7.

[0055] As shown in FIG. 6, the control unit 110 functions as an image acquisition unit 310, an image reconstruction unit 320, and a score calculation unit 330 as a result of the CPU 111 executing the inspection program.

[0056] The image acquisition unit 310 functions as an input data acquisition unit, and cooperates with the communication unit 120 to acquire an input image (input data).

[0057] 7, the image reconstruction unit 320 functions as a data reconstruction unit, and generates a reconstructed image (reconstructed data) as reference data based on the input image acquired by the image acquisition unit 310 using a generative model trained using input images of multiple non-defective products. More specifically, the image reconstruction unit 320 extracts feature amounts from the input image using a trained first neural network, and reconstructs (restores) the input image based on the extracted feature amounts to generate the reconstructed image.

[0058] As will be described later, the judgment unit 340 judges whether a product is good or bad. However, if the input image contains noise, the noise may cause the anomaly score to be calculated incorrectly. To address this issue, the input image and the reconstructed image may be configured to be pre-filtered. The filter may be a noise removal filter (e.g., a low-pass filter, a high-pass filter, or a band-pass filter). This configuration reduces the effect of noise on the anomaly score, preventing or suppressing a deterioration in the performance of judgment (separation performance) of good or bad products due to noise.

[0059] The score calculation unit 330 calculates an abnormality score between the input image acquired by the image acquisition unit 310 and the reconstructed image reconstructed by the image reconstruction unit 320. As shown in Fig. 8, the score calculation unit 330 has an arithmetic processing unit 331. The arithmetic processing unit 331 uses the abnormality score of at least one of the index parameters 1 to N to calculate an abnormality score as an output of the score calculation unit 330.

[0060] The score calculation unit 330 calculates the value of the index parameter to be used by the calculation processing unit 331 from among the index parameters 1 to N, and calculates the abnormality score based on the value of the index parameter. For example, when the score calculation unit 330 is set to use only the abnormality score of index parameter 1 (brightness in the figure), it calculates the value of index parameter 1 and its abnormality score for the input image and the reconstructed image. In this case, the calculation processing unit 331 calculates the abnormality score as an output of the score calculation unit 330 based only on the abnormality score of index parameter 1. The abnormality score of index parameter 1 may be, for example, the difference between the brightness value of the input image and the brightness value of the reconstructed image.

[0061] The same applies when any one of index parameters 2 to N is used alone. The abnormality score for index parameter 2 can be, for example, the difference between the result of calculating index parameter 2 using the input image and the result of calculating index parameter 2 using the reconstructed image. The abnormality scores for other index parameters can be calculated in the same manner.

[0062] The calculation processing unit 331 can set whether to use or not use the abnormality score of each of the index parameters 1 to N based on the index parameters selected by the index selection device and stored in the RAM 212.

[0063] Furthermore, for example, when the abnormality score of index parameter 2 (hue in the figure) is set to be used in addition to the abnormality score of index parameter 1, the calculation processing unit 331 calculates the abnormality score as an output of the score calculation unit 330 based on the abnormality scores of both indexes, that is, the abnormality score of index parameter 1 and the abnormality score of index parameter 2. For example, the abnormality score of the score calculation unit 330 can be calculated by weighting the abnormality score of index parameter 1 and the abnormality score of index parameter 2 by a predetermined coefficient, and then adding them together.

[0064] The operation and display unit 130 displays on the display the abnormality scores (abnormality score map) calculated by the score calculation unit 330. This allows the user to check the position and area of the abnormality in the test subject from the abnormality score map displayed on the display. The control unit 110 and the operation and display unit 130 constitute an information processing system.

[0065] <Configuration of inspection equipment and inspection system> 9 is a functional block diagram illustrating the main functions of the control unit when the information processing device functions as an inspection device. The control unit 110 has an image acquisition unit 310, an image reconstruction unit 320, a score calculation unit 330, and a determination unit 340.

[0066] The determination unit 340 determines whether the inspection object is a good or bad product based on the anomaly score calculated by the score calculation unit 330. For example, when the score calculation unit 330 is set to use only the anomaly score of index parameter 1, the determination unit 340 may determine that the inspection object is a good product if the maximum value of the luminance anomaly score (anomaly score map) is equal to or less than a predetermined first threshold, and may determine that the inspection object is a bad product if the maximum value exceeds the first threshold. Therefore, in the anomaly score map, a region exceeding the first threshold is estimated to be a defective region, such as a scratch, on the inspection object. The first threshold may be experimentally determined by a user based on an anomaly score map calculated for images of multiple inspection objects, including good and bad products.

[0067] Furthermore, the determination unit 340 can determine whether a product is good or bad using only the luminance abnormality score, but if, for example, it is difficult to detect a defective area using only the luminance abnormality score, the determination unit 340 can determine whether a product is good or bad using an abnormality score calculated based on the luminance abnormality score of index parameter 1 and the hue abnormality score of index parameter 2. Alternatively, the determination unit 340 can determine whether a product is good or bad using only the index parameter 2 abnormality score.

[0068] The determination unit 340 preliminarily sets a second threshold for distinguishing between non-defective and defective products for each index parameter based on the distribution of the anomaly scores for non-defective and defective products calculated by the score calculation unit 230 of the index selection device. The determination unit 340 may determine that the inspection object is a non-defective product if the maximum value of the anomaly scores (anomaly score map) calculated by the index parameters by the score calculation unit 330 is equal to or less than the second threshold, and may determine that the inspection object is a defective product if the maximum value exceeds the second threshold. Therefore, in the anomaly score map, an area exceeding the second threshold is estimated to be a defective area, such as a scratch, on the inspection object. Furthermore, when the anomaly scores output by the score calculation unit 330 are calculated based on the anomaly scores of multiple index parameters, the second threshold may be experimentally determined by the user based on, for example, the anomaly scores calculated for multiple images of inspection objects, including non-defective and defective products. The determination result is stored in the RAM 112. Furthermore, the determination result may be displayed on the display of the operation / display unit 130. The information processing device 100 constitutes an inspection system.

[0069] <Indicator selection method> 10 is a flowchart illustrating the processing procedure of the index selection method of this embodiment. The processing of the flowchart in FIG. 10 is realized by the CPU 111 executing an index selection program.

[0070] First, an input image is acquired (step S101). The image acquisition unit 210 acquires a plurality of input images of the inspection target from, for example, an imaging device or a storage device external to the information processing device 100. These input images are previously labeled as either good or bad. The image acquisition unit 210 transmits the plurality of input images to the image reconstruction unit 220 and the score calculation unit 230.

[0071] Next, a reconstructed image is generated (step S102). Based on the input images acquired by the image acquisition unit 210, the image reconstruction unit 220 generates a plurality of reconstructed images corresponding to the input images using a generative model.

[0072] Next, the abnormality scores of the non-defective and defective products are calculated (step S103). The score calculation unit 230 calculates the abnormality scores 1 to N using index parameters 1 to N based on a plurality of input images of the non-defective and defective products and a plurality of reconstructed images corresponding to the input images.

[0073] Next, an index parameter is selected (step S104). The index selection unit 240 selects one of the index parameters 1 to N according to the abnormality scores 1 to N of the non-defective and defective products. More specifically, the index selection unit 240 learns index parameters for a plurality of input images of non-defective and defective products, for example, and selects, from the index parameters 1 to N, the index parameter that maximizes the difference between the distribution of abnormality scores of non-defective products and the distribution of abnormality scores of defective products.

[0074] <Testing method> Fig. 11 is a flowchart illustrating the processing procedure of the inspection method of this embodiment. The processing of the flowchart in Fig. 11 is realized by the CPU 111 executing an inspection program. Fig. 12 is a schematic diagram for explaining prevention of overdetection.

[0075] First, as shown in Fig. 9, an input image is acquired (step S201). The image acquisition unit 310 acquires an input image of the inspection target from, for example, an imaging device or a storage device external to the information processing device 100. The input image is an image of the inspection target that has not been determined to be pass / fail (unknown). The image acquisition unit 310 transmits the input image to the image reconstruction unit 320 and the score calculation unit 330.

[0076] Next, a reconstructed image is generated (step S202). Based on the input image acquired by the image acquisition unit 310, the image reconstruction unit 320 generates a reconstructed image corresponding to the input image using a generative model.

[0077] Next, an abnormality score is calculated (step S203). The score calculation unit 330 calculates an abnormality score of brightness based on the input image and a reconstructed image corresponding to the input image. The score calculation unit 330 also calculates an abnormality score using index parameters based on the input image, a reconstructed image corresponding to the input image, and the index parameters selected in step S104 of the index selection method. The index parameters are selected for each test object.

[0078] Next, the inspection object is judged as being good or bad (step S204). The judgment unit 340 judges whether the inspection object is good or bad based on the luminance abnormality score calculated by the score calculation unit 330 and the abnormality score based on the index parameter. In the example shown in Fig. 7, the area corresponding to the scratch S1 in the abnormality score map based on the index parameter exceeds the second threshold, so the inspection object is judged to be bad.

[0079] Furthermore, in this embodiment, defect detection can be performed on the difference between the luminance values of the input image and the reconstructed image. As shown in Fig. 10, for example, assume that an input image IM1 includes a linear defect F that was added during the manufacturing process in addition to a linear feature C that a non-defective product originally has. In a reconstructed image IM2 corresponding to the input image IM1, the feature C of the input image IM1 is reconstructed, but the defect F is not reconstructed. Therefore, a difference image IM3 is generated by subtracting the luminance values of the reconstructed image IM2 from the luminance values of the input image IM1, and the defect F remains in the difference image IM3.

[0080] Since the difference image IM3 does not contain the feature C of a non-defective product, the linear defect F can be detected, for example, by a linear defect detection algorithm. In contrast, in the conventional technology in which image processing is directly performed on the input image IM1, even if a linear defect detection algorithm is used, the input image IM1 contains the feature C of a non-defective product and the linear defect F, so there is a risk that not only the defect F but also the feature C will be detected as a defect.

[0081] In this manner, in the present embodiment, when performing defect detection, a detection algorithm can be used for the difference between the luminance values of the input image and the luminance values of the reconstructed image. Therefore, even when the characteristics of a non-defective product and the characteristics of a defect match or are similar, overdetection of non-defective products can be more effectively prevented or suppressed than in the prior art.

[0082] The index selection device, information processing device, and inspection device of the present embodiment described above can achieve the following effects.

[0083] The index selection device selects one of multiple indexes according to the anomaly scores of good and bad products, thereby obtaining an index that is effective for distinguishing good from bad products. The information processing device then calculates an anomaly score using the index selected by the index selection device, allowing the user to confirm the location and area of an anomaly in the inspection object from the anomaly score map displayed on the display. Furthermore, the inspection device can improve the accuracy of determining whether the inspection object is good or bad based on the anomaly score calculated by the information processing device. Furthermore, the robustness and defect detection accuracy can be improved for various inspection objects and defect types.

[0084] The above-described index selection device, information processing device, information processing system, inspection device, inspection system, index selection method, and index selection program have been described as main configurations in explaining the features of the above-described embodiments, but are not limited to the above configurations and can be modified in various ways within the scope of the claims. Furthermore, configurations provided in general inspection devices, etc. are not excluded.

[0085] For example, some steps in the above-described flowcharts may be omitted, other steps may be added, some of the steps may be executed simultaneously, or one step may be divided into multiple steps and executed.

[0086] In addition, in the above-described embodiment, the information processing device 100 serves as an index selection device, an information processing device, and an inspection device, but this is not limited to such a case, and the index selection device, information processing device, and inspection device may be realized on separate hardware.

[0087] The means and methods for performing various processes in the above-described device can be realized by either dedicated hardware circuits or a programmed computer. The above-described program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is typically transferred to and stored in a storage unit such as a hard disk. The above-described program may be provided as standalone application software, or may be incorporated as a function into the software of a device such as an index selection device or information processing device.

[0088] This application is based on Japanese Patent Application No. 2020-208476, filed on December 16, 2020, the disclosure of which is incorporated by reference in its entirety. [Explanation of symbols]

[0089] 100 information processing device, 110 control section, 111 CPUs, 112 RAM, 113 ROMs, 114 Auxiliary storage, 120 Communications Department, 130 Operation display section, 210 image acquisition unit, 220 Image reconstruction unit, 221 encoder, 222 decoder, 230 score calculation unit, 240 Index selection unit, 310 image acquisition unit, 320 Image reconstruction unit, 321 encoder, 322 decoder, 330 score calculation unit, 331 processing unit, 340 Judgment Department.

Claims

1. A data reconstruction unit that generates reconstructed data corresponding to input data based on the input data using a generative model trained using input data of a plurality of non-defective products; a score calculation unit that calculates anomaly scores for the non-defective and defective products using a plurality of indices based on a plurality of input data for the non-defective and defective products and a plurality of the reconstructed data; an index selection unit that selects one of the plurality of indexes according to the abnormality scores of the non-defective and defective products.

2. The index selection device according to claim 1 , wherein the anomaly score is a difference between a result of calculating the index using the input data and a result of calculating the index using the reconstructed data for each of a non-defective product and a defective product.

3. 3. The index selection device according to claim 1, wherein the index selection unit selects an index that maximizes a difference between a distribution of abnormality scores of the non-defective products and a distribution of abnormality scores of the defective products for the plurality of input data and the plurality of reconstructed data.

4. The index selection unit The indicator selection device according to any one of claims 1 to 3, wherein feature quantities of a plurality of input data of good products and defective products are used as explanatory variables, and a difference between the distribution of anomaly scores of the good products and the distribution of anomaly scores of the defective products is used as a target variable, and a trained model trained to maximize the difference is used to select one of the plurality of indicators.

5. the input data is color image data, 5. The index selection device according to claim 1, wherein the score calculation unit calculates the abnormality score using hue and / or saturation as the index.

6. an input data acquisition unit that acquires input data; 6. An information processing device comprising: a score calculation unit that calculates an anomaly score based on input data acquired by the input data acquisition unit, reconstruction data corresponding to the input data, and an index selected by the index selection device according to any one of claims 1 to 5.

7. An inspection device having a determination unit that determines whether a product is good or bad based on the anomaly score output by the information processing device according to claim 6.

8. The information processing device according to claim 6 ; and a display device that displays the abnormality score calculated by the score calculation unit.

9. The inspection device according to claim 7; a display device that displays the determination result by the determination unit.

10. A step (a) of generating reconstruction data corresponding to the input data based on the input data using a generative model trained using input data of a plurality of non-defective products; (b) calculating anomaly scores for the non-defective and defective products using a plurality of indices based on a plurality of input data for the non-defective and defective products and a plurality of the reconstructed data; and (c) selecting one of the plurality of indices according to the anomaly scores of the non-defective and defective products.

11. The index selection method according to claim 10 , wherein the anomaly score is a difference between a result of calculating the index using the input data and a result of calculating the index using the reconstructed data for each of a good product and a defective product.

12. 12. The index selection method according to claim 10, wherein in step (c), an index that maximizes the difference between the distribution of anomaly scores of the non-defective products and the distribution of anomaly scores of the defective products for the plurality of input data and the plurality of reconstructed data is selected.

13. The index selection method according to any one of claims 10 to 12, wherein in the step (c), feature quantities of a plurality of input data of good products and defective products are used as explanatory variables, and a difference between the distribution of anomaly scores of the good products and the distribution of anomaly scores of the defective products is used as a target variable, and one of the plurality of indexes is selected using a trained model trained to maximize the difference.

14. the input data is color image data, The index selection method according to any one of claims 10 to 13, wherein in the step (b), an abnormality score is calculated using hue and / or saturation as the index.

15. An index selection program for causing a computer to execute the processes included in the index selection method according to any one of claims 10 to 14.

Images