TESTING DEVICE, TESTING PROCEDURE, TESTING PROGRAM, LEARNING DEVICE, LEARNING PROCEDURE AND LEARNING PROGRAM

Abstract

Test device, comprising: a target image procurement section that achieves a target image obtained by capturing a test target, a target image masking section that masks part of the target image, a masking area prediction section that predicts an image of a masked area that is masked in the target image, a replica image generation section that generates a replica image using a multitude of predicted images, each predicted for a multitude of masked areas containing the masked area, and a difference detection section that detects a difference between the target image and the replicated image, wherein the target image masking section successively masks one cell at a time from a plurality of cells obtained by subdividing the target image, and produces a plurality of masked images obtained by masking each one from the plurality of cells, The masking area prediction section predicts the image of the masked area for each of the multitude of cells and generates a multitude of predicted images, and The replica image generation section generates the replicated image by arranging each of the multitude of predicted images, each predicted for the masked area of ​​the multitude of cells, at the position of each of the multitude of cells.

Description

[0001] The present invention relates to a testing apparatus, a testing method, a testing program, a learning apparatus, a learning method and a learning program. State of the art

[0002] Traditionally, an appearance inspection of a test target is performed to assess the suitability of that test target.

[0003] For example, in an appearance inspection of a probe pin for testing a semiconductor, an image of the probe pin is captured and digitized, and the digitized data is evaluated using predefined rules to assess the suitability of the probe pin.

[0004] US 2018 / 374207 A1 describes systems and methods for detecting and correcting defective products, comprising capturing at least one image of a product with at least one image sensor to generate an original image of the product. An encoder encodes portions of an image extracted from the original image to generate feature-space vectors. A decoder decodes the feature-space vectors to convert the portions of the image into reconstructed portions by predicting defect-free structural features in each of the portions using hidden layers trained to predict defect-free products. Each of the reconstructed portions is merged into a reconstructed image of a defect-free representation of the product. The reconstructed image is transmitted to a contraster to detect anomalies indicative of product defects.WO 2019 / 075656 A1 describes an image processing method comprising the steps of: recognizing a source object from a source image and recognizing the pose and size of the source object depicted in the source image according to feature points of the source object, adjusting the pose and size of the source object depicted in the source image according to a correspondence relationship between the pose and size of the source object depicted in the source image and the pose and size of a target object depicted in a target image, adjusting the shape of the source object depicted in the source image and the shape of the target object depicted in the target image according to an average shape of an image of the source object and an image of the target object, and merging the source object and the target object according to an alignment manner of the shape depicted by the source object and the shape depicted by the target object. Problem to be solved

[0005] However, when testing the test objective, there is a desire to make the state of the test objective easily understandable. General Revelation

[0006] The present invention provides a testing device according to claims 1 and 7, a testing method according to claims 12 and 13, a testing program according to claims 14 and 15, a learning device according to claim 16, a learning method according to claim 17, and a learning program according to claim 18. Preferred embodiments are the subject of the subclaims.

[0007] To solve the above problem, according to a first aspect of the present invention, an inspection apparatus is provided. The inspection apparatus may include a target image acquiring section that acquires a target image obtained by capturing an inspection target. The inspection apparatus may include a target image masking section that masks a part of the target image. The inspection apparatus may include a masked area predicting section that predicts an image of a masked area masked in the target image. The inspection apparatus may include a replica image generating section that generates a replica image using a plurality of predicted images, each predicted for a plurality of masked areas including the masked area. The inspection apparatus may include a difference detecting section that detects a difference between the target image and the replica image.

[0008] The difference detection section may compare the target image with the replicated image in each predetermined area to calculate a difference measure in each predetermined area.

[0009] The testing apparatus may further comprise a judgment section that judges that the test target is unacceptable if the difference measure does not meet a predetermined quality standard.

[0010] The assessment section may judge that the test objective is not acceptable if a largest difference measure from the difference measures of each given range exceeds a given threshold.

[0011] If the judging section judges that the inspection target is unacceptable, the judging section can predict an electrical property of the inspection target from the target image obtained by capturing the inspection target judged as unacceptable and confirm that the inspection target is unacceptable if the electrical property does not meet a predetermined quality standard.

[0012] If the judging section judges that the inspection target is unacceptable, the judging section can predict an electrical property of the inspection target from the target image obtained by capturing the inspection target judged as unacceptable and confirm that the inspection target is acceptable if the electrical property meets a predetermined quality standard.

[0013] The difference detection section may output a detection image in which a display attribute differs in each predetermined area according to the difference measure.

[0014] The difference detection section may calculate the difference measure based on a Euclidean distance between the target image and the replicated image.

[0015] The target image masking section may sequentially mask one cell at a time from a plurality of cells obtained by dividing the target image, and the replica image generating section may predict the replica image using a plurality of predicted images each obtained for different cells.

[0016] The target image acquiring section may acquire, as a target image, an image obtained by performing grayscale conversion on an image obtained by capturing the inspection target.

[0017] The target image acquiring section may acquire an image as a target image obtained by performing object detection to narrow down a target area.

[0018] According to a second aspect of the present invention, an inspection method is provided. The inspection method may include: acquiring a target image obtained by capturing an inspection target, masking a portion of the target image, predicting an image of a masked area masked in the target image, generating a replicated image using a plurality of predicted images each predicted for a plurality of masked areas including the masked area, and detecting a difference between the target image and the replicated image.

[0019] According to a third aspect of the present invention, an inspection program is provided. The inspection program, when executed by a computer, causes the computer to function as: a target image acquiring section that acquires a target image obtained by capturing an inspection target; a target image masking section that masks a part of the target image; a masked area predicting section that predicts an image of a masked area masked in the target image; a replica image generating section that generates a replica image using a plurality of predicted images each predicted for a plurality of masked areas including the masked area; and a difference detecting section that detects a difference between the target image and the replica image.

[0020] According to a fourth aspect of the present invention, a learning device is provided. The learning device may include: a training image acquisition section that acquires a training image, a training image masking section that masks a portion of the training image, a prediction model that receives the masked training image and outputs a model image obtained by predicting the training image, and a model updating section that updates the prediction model based on an error between the training image and the model image.

[0021] According to a fifth aspect of the present invention, a learning method is provided. The learning method may include: obtaining a training image, masking a portion of the training image, inputting the masked training image to a prediction model, outputting a model image obtained by predicting the training image, and updating the prediction model based on an error between the training image and the model image.

[0022] According to a sixth aspect of the present invention, a learning program is provided. When executed by a computer, the learning program causes the computer to function as: a training image acquisition section that acquires a training image, a training image masking section that masks a portion of the training image, a prediction model that receives the masked training image and outputs a model image obtained by predicting the training image, and a model updating section that updates the prediction model based on an error between the training image and the model image.

[0023] The summary does not necessarily describe all necessary features of the embodiments of the present invention. The present invention may also be a subcombination of the features described above. Short description of the drawing Fig. 1 shows a block diagram of the testing apparatus 100 according to the present embodiment. Fig. 2 shows an example of a flow for testing a test target with the test apparatus 100 according to the present embodiment. Fig. 3 shows an example of a target image 310, masked images 320, predicted images 330, and a replicated image 340 in the inspection using the inspection apparatus 100 according to the present embodiment. Fig. 4 shows an example of the test result in a case where the test target is acceptable in the present embodiment. Fig. Figure 5 shows an example of test results in a case where the test objective is not acceptable. Fig. 6 shows an example of a block diagram of a learning device 600 according to the present embodiment. Fig. 7 shows an example of a learning process of the prediction model 630 by the learning device 600 according to the present embodiment. Fig. Figure 8 shows an example of a computer 2200 in which aspects of the present invention may be embodied in whole or in part. Description of exemplary embodiments

[0024] Some embodiments of the present invention are described below. However, the following embodiments do not limit the invention defined in the claims. The embodiments do not limit the invention according to the claims, and all combinations of the features described in the embodiments are not necessarily essential to the means provided by the aspects of the invention.

[0025] Fig. 1 shows a block diagram of a test apparatus 100 according to the present embodiment. The test apparatus 100 can easily understand the state of an inspection target by detecting the difference between an image created by capturing the actual state of the inspection target and an image created by predicting the state the inspection target should be in. In the present embodiment, an example will be described in which the test apparatus 100 uses a probe pin for testing a semiconductor as the inspection target. However, the inspection target is not limited to this. The test apparatus 100 can be used to analyze an image of bumps of a semiconductor device, inspect a pattern of a wiring substrate, or inspect other electronic components, or can be used to inspect any type of object that is not an electrical component.

[0026] The test apparatus 100 may be a computer such as a PC (personal computer), a tablet computer, a smartphone, a workstation, a server computer, or a general user computer, or may be a computer system in which a plurality of computers are connected. Such a computer system is also a computer in a broad sense. The test apparatus 100 may be implemented in a virtual computing environment that can be executed in one or more computers. Instead, the test apparatus 100 may be a specialized computer designed to test the test target or may be specialized hardware realized by specialized circuitry. If the test apparatus 100 can connect to the Internet, the test apparatus 100 may be realized through cloud computing.

[0027] The inspection apparatus 100 includes a target image acquiring section 110, a target image masking section 120, a masking area predicting section 130, a replica image generating section 140, a difference detecting section 150, and a judging section 160.

[0028] The target image acquisition section 110 acquires a target image obtained by capturing an image of the inspection target. The target image acquisition section 110 may acquire an image obtained by preprocessing a captured image of the inspection target as the target image. In this case, the target image acquisition section 110 may acquire the target image via a network, a user input, or a storage device or the like that can store data, for example. The target image acquisition section 110 supplies the acquired target image to the target image masking section 120 and the difference detection section 150.

[0029] The target image masking section 120 masks a portion of the target image. The target image masking section 120 then provides the masking area prediction section 130 with a plurality of masked images obtained by masking different portions of the target image, respectively.

[0030] The masked area prediction section 130 predicts an image of the masked area that has been masked in the target image. The masked area prediction section 130 then provides the replica image generation section 140 with a plurality of predicted images, each predicted for the masked area in the plurality of masked images.

[0031] The replica image generation section 140 generates a replica image using the plurality of predicted images each predicted for the plurality of masked areas. The replica image generation section 140 supplies the generated replica image to the difference detection section 150.

[0032] The difference detection section 150 detects the difference between the target image and the replica image. Here, the difference detection section 150 can compare the target image provided by the target image acquisition section 110 with the replica image provided by the replica image generation section 140 in each predetermined area, for example, to calculate a difference measure in each predetermined area. The difference detection section 150 then provides the calculated difference measure to the judgment section 160.

[0033] The judgment section 160 judges that the inspection target is unacceptable if the difference measure does not meet a predetermined quality standard. The judgment section 160 outputs the judgment result to another functional section, another device, and the like. The following procedure is used to describe the details of testing an inspection target using such an inspection device 100.

[0034] Fig. 2 shows an example of a flow for testing a test target with the test apparatus 100 according to the present embodiment. In step 210, the target image acquisition section 110 acquires the target image obtained by capturing an image of the test target. For example, the target image acquisition section 110 acquires, via a network, an image of a probe pin for testing a semiconductor, which was obtained using an optical microscope or the like.

[0035] The target image acquisition section 110 converts an image obtained by capturing the inspection target into grayscale. The probe pin image captured using the optical microscope or the like may include three channels, each corresponding to the three colors R, G, and B. However, from the inspection point of view according to the present embodiment, these three channels have approximately the same characteristics, and no channel has a feature unique from the other channels. Accordingly, the target image acquisition section 110 converts the acquired image into a single channel by performing grayscale conversion on the acquired image. In this way, the target image acquisition section 110 can acquire an image obtained by performing grayscale conversion on an image obtained by capturing the inspection target as the target image.By performing grayscale conversion on the acquired image using only a single channel, the inspection apparatus 100 can reduce the burden on the inspection process when acquiring the target image. In a case where it is preferable to use multiple channels, for example, in a case where the accuracy of the inspection is to be improved, the target image acquisition section 110 can acquire an image obtained by capturing the inspection target as it is as the target image without performing grayscale conversion.

[0036] The target image acquisition section 110 narrows down an image obtained by capturing the inspection target. For example, the target image acquisition section 110 detects the position and size of the probe pin in the image using an object detection algorithm such as YOLO (You Only Look Once). The target image acquisition section 110 narrows down the target area by cropping the image based on the position and size of the detected probe pin. In this way, the target image acquisition section 110 can perform object detection of the inspection target in an image obtained by capturing the inspection target and acquire the image in which the target area has been narrowed down as the target image. In this way, the inspection apparatus 100 can improve inspection accuracy and speed up the inspection process when acquiring the target image by performing object detection of the inspection target and narrowing down the target area.The target image acquiring section 110 acquires the pre-processed image (for example, the image to which grayscale conversion and narrowing have been performed) as a target image and supplies this target image to the target image masking section 120 and the difference detecting section 150.

[0037] In step 220, the target image masking section 120 masks a portion of the target image. For example, the target image masking section 120 divides the target image obtained in step 210 into a plurality of cells. The target image masking section 120 then sequentially masks one cell at a time from the plurality of cells obtained by dividing the target image. The target image masking section 120 supplies the plurality of masked images obtained by masking each of the plurality of cells to the masking area prediction section 130.

[0038] In step 230, the masking region prediction section 130 predicts an image of a masked region that is masked in the target image. Here, the masking region prediction section 130 may use a trained model, such as a CNN (Convolution Neural Network), that has been taught to predict an image of the masked region from another image that is unmasked when the image in which a partial region is masked is input, for example, using only training images of inspection targets known to be acceptable. In other words, the masking region prediction section 130 may use a trained model that has been learned to predict the state of the masked region when the inspection target is acceptable using only training images of inspection targets known to be acceptable.As an example, the masking region prediction section 130 feeds each of the plurality of masked images obtained in step 220 to the trained model and predicts the masked region image for each of the plurality of cells. The learning of such a model will be described below. The above description is an example in which the masking region prediction section 130 uses a trained CNN model, but the present embodiment is not limited thereto. The masking region prediction section 130 may predict the masked region image using a trained model of another algorithm, or may predict the masked region image using an algorithm different from the trained one.The masking area prediction section 130 supplies the replica image generation section 140 with the plurality of predicted images each predicted for the plurality of masked areas.

[0039] In step 240, the replica image generating section 140 generates a replica image using the plurality of predicted images each predicted for the plurality of masked areas. For example, the replica image generating section 140 generates the replica image using the plurality of predicted images each predicted for different cells. Here, the replica image generating section 140 may generate the replica image by arranging the plurality of predicted images each predicted for the plurality of cells in step 230, for example, at the original positions of these cells. The replica image generating section 140 supplies the generated replica image to the difference detecting section 150.

[0040] In step 250, the difference detection section 150 detects the difference between the target image and the replica image. For example, the difference detection section 150 performs a comparison between the target image provided by the target image acquisition section 110 and the replica image provided by the replica image generation section 140 in each predetermined area (e.g., each pixel, each pixel group, each cell used in masking the target image, and the like) and calculates the difference measure in each predetermined area. Here, the difference detection section 150 may calculate the difference measure based on the L2 norm, that is, the Euclidean distance between the target image and the replica image.Furthermore, the difference detection section 150 can output a detection image in which the display attributes (e.g., color, concentration, etc.) differ in each predetermined area according to the difference measure. The difference detection section 150 then provides the judgment section 160 with the difference measure in each predetermined area.

[0041] In step 260, the judging section 160 judges whether the difference measure calculated in step 250 is less than or equal to a predetermined threshold. If the difference measure is less than or equal to the predetermined threshold, that is, if the difference measure meets a predetermined quality standard, the judging section 160 proceeds to step 270, judges that the inspection target is acceptable, and terminates the process. On the other hand, if the difference measure exceeds the predetermined threshold, that is, if the difference measure does not meet the predetermined quality standard, the judging section 160 proceeds to step 280, judges that the inspection target is unacceptable, and terminates the process. Here, the judging section 160 may judge the inspection target as unacceptable if, for example, the largest difference measure among the difference measures for each predetermined range exceeds the predetermined threshold.The judgment section 160 outputs the judgment result to another functional section, another device, and the like. The threshold value used for this judgment may be the minimum value, or a value slightly lower than this minimum value, obtained when the difference measure is calculated by the inspection apparatus 100 according to the present embodiment using an image obtained by capturing an inspection target known to be unacceptable. Moreover, in the above description, an example is shown in which the judgment section 160 judges the suitability of the inspection target based on the largest difference measure among the difference measures of each predetermined area, but the present embodiment is not limited thereto.The judging section 160 may judge the suitability of the test target based on another statistical value of the difference measure, such as the median value, the average value, the distribution, or the like.

[0042] Fig. 3 shows an example of a target image 310, masked images 320, predicted images 330, and a replica image 340 in the inspection using the inspection apparatus 100 according to the present embodiment. The target image acquiring section 110 acquires the target image 310, for example, as shown in this drawing. The target image masking section 120 divides the target image 310 into a plurality of cells 322 (a total of 25 cells, where [vertical, horizontal] = [1, 1] to [5, 5] in the present drawing). The target image masking section 120 sequentially masks each of these cells 322 using a mask 324 to generate each of the plurality of masked images 320. For example, the upper, middle, and lower masked images 320 in the present drawing are cases where cell [2, 3], cell [3, 3], and cell [4, 3] are masked, respectively.The masking area prediction section 130 predicts the image of the masked area 332 for each of the plurality of cells 322 to generate each of the plurality of predicted images 330. The replica image generation section 140 generates the replica image 340 by arranging the plurality of predicted images 330, each predicted for the plurality of cells 322, at the original positions of those cells 322.

[0043] Fig. 4 shows an example of the inspection result in a case where the inspection target is acceptable in the present embodiment. When the inspection target is acceptable, the replica image 340 generated by the inspection apparatus 100 is approximately the same as the target image 310. This is because, since the inspection apparatus 100 predicts the state that the masked area should have when the inspection target is acceptable using the trained model learned using only training images of inspection targets known to be acceptable, the replica image 340 generated by the inspection apparatus 100 is an image that replicates the state that the inspection target should be in when that inspection target is acceptable. Accordingly, even in the detection image 400 in which the display attributes of each predetermined area differ according to the difference measure, all of the areas have approximately the same display attributes.Furthermore, in a distribution (lower part of the present drawing) obtained by counting the number of unit areas (cell unit areas in the present drawing) of each difference measure, the majority of cells with a difference measure close to 0 are counted, and none of the cells are counted at positions where the difference measure is greater than 1.5 (the threshold). Here, the unit areas can be areas such as pixel units, pixel group units, and cell units, and the present drawing shows an example in which the unit areas are cell unit areas. Moreover, a larger width on the vertical axis indicates a larger number of counted cells. In this way, the inspection apparatus 100 judges the inspection target in the target image 310 to be acceptable when the difference between the target image 310 and the replica image 340 is small.

[0044] Fig. 5 shows an example of inspection results in a case where the inspection target is unacceptable. When the inspection target is unacceptable (for example, when the inspection target is broken), the replica image 340 generated by the inspection apparatus 100 differs from the target image 310. In this case, areas where the difference degree differs are indicated by different attributes in the detection image 400. In addition, in the distribution of the difference degrees, many cells are counted at positions where the difference degree is large, and several cells are counted at positions where the difference degree is greater than 1.5. When the difference between the target image 310 and the replica image 340 is large in this way, the inspection apparatus 100 judges the inspection target in the target image 310 to be unacceptable.

[0045] In this way, according to the inspection apparatus 100 of the present embodiment, it is possible to easily understand the state of the inspection target by detecting the difference between an image (target image 310) obtained by capturing the actual state of the inspection target and an image (recreated image 340) obtained by predicting and replicating the state the inspection target should be in when the inspection target is acceptable. Furthermore, the test apparatus 100 outputs the detection image 400 in which the display attributes in each area differ according to the difference degree, and therefore it is possible to easily understand the location of defects in the inspection target.Furthermore, the inspection apparatus 100 outputs the distribution obtained by counting the number of unit regions of each difference measure, and therefore it is possible to understand the frequency with which regions with different difference measures appear in the image.

[0046] Such an inspection using the test apparatus 100 according to the present embodiment can be performed, for example, during the manufacture of the probe and before the probe is shipped, or can be performed immediately before or at intervals during the actual testing of a semiconductor device using the probe. Furthermore, in a case where a visual inspection using images from multiple directions would be useful, the test apparatus 100 can better understand the condition of the test target by performing this inspection using images of the test target taken from multiple directions.

[0047] The above description uses an example in which the test apparatus 100 judges the suitability of the test target based only on an appearance inspection using an image, but the present embodiment is not limited to this. The test apparatus 100 may judge the suitability of the test target based on both an appearance inspection using an image and electrical characteristics.

[0048] For example, if the judging section 160 judges that the inspection target is unacceptable, the electrical properties of the inspection target are predicted from the target image obtained by capturing an inspection target judged as unacceptable, and if these electrical properties do not meet a predetermined quality standard, the inspection target can be confirmed as unacceptable. For example, the judging section 160 may use a trained model that has learned to predict the electrical properties of the inspection target when the target image input thereto is input. In other words, the judging section 160 may use a trained model that has learned to predict the electrical properties, such as the resistance value and the like, of a probe pin when the target image obtained by capturing the inspection target is input thereto. Then, in step S280 of Fig. 2, the judgment section 160 may input the target image obtained by capturing a probe pin judged to be unacceptable into this trained model to predict the resistance value of the probe pin. If the resistance value does not meet a predetermined quality standard, the judgment section 160 may confirm that the probe pin, which is the inspection target, is unacceptable. The above description shows an example in which the judgment section 160 predicts the electrical characteristics of the inspection target using a trained model, but the present embodiment is not limited to this. The judgment section 160 may predict the electrical characteristics of the inspection target from the target image using an algorithm other than learning.

[0049] Similarly, if the inspection target is judged to be unacceptable, the judging section 160 may predict the electrical characteristics of the inspection target from the target image obtained by capturing the inspection target judged to be unacceptable, and if the electrical characteristics meet the predetermined quality standard, may confirm that the inspection target is acceptable. For example, in step S280 of Fig. 2, the judgment section 160 may input the target image obtained by capturing the test pin judged to be unacceptable into the trained model to predict the resistance value of the test pin, and, if the resistance value meets a predetermined quality standard, may confirm that the test pin, which is the inspection target, is acceptable.

[0050] In this way, the inspection apparatus 100 can accurately assess the suitability of the inspection target by considering the electrical properties in addition to the appearance inspection using an image. For example, even if an inspection target was judged unacceptable in the appearance inspection using an image due to the effects of reflected shadows, particles, or the like during image acquisition of the inspection target, there may be cases where there are no problems with the electrical properties. Therefore, the inspection apparatus 100 can more accurately assess the suitability of the inspection target by combining this appearance inspection with the electrical properties.

[0051] Fig. 6 shows an example of a block diagram of a learning device 600 according to the present embodiment. The learning device 600 causes a model to learn to be able to predict a masked region from an image of another region that is not masked when an image in which a partial region is masked is input thereto, using only training images of targets known to be acceptable. In other words, the learning device 600 causes a model to be able to predict the state a masked partial region should be in if the test target is acceptable, using only training images of targets known to be acceptable.The inspection apparatus 100 according to the present embodiment can predict the masked area using a learned model caused to learn by the learning apparatus 600, for example, as shown in the present drawing.

[0052] The learning device 600 may be a computer such as a PC (personal computer), a tablet computer, a smartphone, a workstation, a server computer, or a general-purpose computer, or may be a computer system in which a plurality of computers are connected. Such a computer system is also a computer in a broad sense. The learning device 600 may be implemented in a virtual computing environment that can be executed on one or more computers. Instead, the learning device 600 may be a specialized computer designed for the purpose of model learning or may be specialized hardware realized by specialized circuitry. If the learning device 600 can connect to the Internet, the learning device 600 may be realized through cloud computing.

[0053] The learning device 600 includes a training image acquiring section 610, a training image masking section 620, a prediction model 630, an error calculating section 640, and a model updating section 650.

[0054] The training image acquisition section 610 acquires a training image. For example, the training image acquisition section 610 may acquire a plurality of images in which inspection targets known to be acceptable are captured as training images. Here, the training image acquisition section 610 may, for example, acquire the training images via a network, acquire the training images via user input, or acquire the training images via a storage device or the like that can store data. The training image acquisition section 610 supplies the acquired training images to the training image masking section 620 and the error calculation section 640.

[0055] The training image masking section 620 masks a portion of a training image. For example, the training image masking section 620 provides the prediction model 630 with a plurality of masked images obtained by randomly masking the plurality of images obtained as training images.

[0056] The prediction model 630 receives a training image in which a partial area is masked and outputs a model image obtained by predicting the training image. For example, when the training image in which the area is masked is input, the prediction model 630 predicts the state in which the masked partial area should be when the inspection target is acceptable and outputs the model image obtained by predicting this training image. Here, the prediction model 630 may use an algorithm such as CNN to predict the masked area. The above description shows an example in which the prediction model 630 uses CNN, but the present embodiment is not limited to this. The prediction model 630 may predict the image of the masked area using an algorithm other than CNN. The prediction model 630 provides the output model image to the error calculation section.

[0057] The error calculation section 640 calculates the error between the training image provided by the training image acquisition section 610 and the model image provided by the prediction model 630. The error calculation section 640 provides the calculated error to the model update section 650.

[0058] The model updating section 650 updates the prediction model 630 based on the error between the training image provided by the training image acquiring section 610 and the model image provided by the prediction model 630. The following describes the details of learning a model using such a learning device 600.

[0059] Fig. 7 shows an example of a flow of learning the prediction model 630 by the learning device 600 according to the present embodiment. In step 710, the training image acquisition section 610 acquires the training images. For example, the training image acquisition section 610 acquires, via a network, a plurality of images in which inspection targets known to be acceptable have been captured. The training image acquisition section 610 acquires the images obtained by preprocessing the acquired images as the training images in the same way as the target image acquisition section 110. Here, the training image acquisition section 610 may discard images in which the focus has shifted from the inspection target using a contour detection filter or the like, without acquiring these images as training images.The training image acquiring section 610 supplies the training image masking section 620 and the error calculating section 640 with the acquired training images.

[0060] In step 720, the training image masking section 620 masks portions of the training images. For example, the training image masking section 620 randomly selects a plurality of images obtained as training images. Then, the training image masking section 620 randomly masks one cell from a plurality of cells obtained by dividing the image area in each of the randomly selected images. The training image masking section 620 then provides the prediction model 630 with the plurality of masked images obtained by randomly masking the randomly selected images.

[0061] In step 730, the prediction model 630 receives the masked training images and outputs a model image obtained by predicting the training images. For example, if the masked images in which partial regions are randomly masked are input, the prediction model 630 predicts the images of the masked regions from the images of other regions that are not masked. The prediction model 630 then outputs the model image by embedding the predicted images in the masked regions of the training images. The prediction model 630 feeds the model image to the error calculation section 640.

[0062] In step 740, the error calculation section 640 calculates the error between the training image provided by the training image acquisition section 610 and the model image provided by the prediction model 630. The error calculation section 640 provides the calculated error to the model update section 650.

[0063] In step 750, the model updating section 650 updates the prediction model 630 based on the error between the training image provided by the training image acquiring section 610 and the model image provided by the prediction model 630. For example, the model updating section 650 updates parameters, such as weights, in the prediction model 630 in such a way as to minimize an objective function, which is the error calculated in step 740.

[0064] In step 760, the learning device 600 judges whether the training is complete. If it is judged in step 760 that the training is not complete, the learning device 600 returns the processing to step 710 and repeats these processes. On the other hand, if it is judged in step 760 that the training is complete, the learning device 600 ends the processing. Here, the learning device 600 can judge whether the training is complete based on conditions such as the training time, the number of training sessions, and the training accuracy.

[0065] In this way, according to the learning device 600 of the present embodiment, the prediction model 630 is updated in a manner that minimizes the error between the training image and the model image, using only a plurality of images in which the captured inspection targets are known to be acceptable as training images, and therefore the learning device 600 can update the prediction model 630 to be able to predict the image of a state in which the masked area should be when the inspection target is acceptable.

[0066] Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, whose blocks may represent (1) steps of processes in which operations are performed, or (2) portions of devices responsible for performing operations. Certain steps and portions may be implemented by dedicated circuitry, programmable circuitry supplied with computer-readable instructions stored on computer-readable media, and / or processors supplied with computer-readable instructions stored on computer-readable media. Dedicated circuitry may include digital and / or analog hardware circuitry and may include integrated circuits (ICs) and / or discrete circuitry.Programmable circuits may include reconfigurable hardware circuits that include logical AND, OR, XOR, NAND, NOR and other logical operations, flip-flops, registers, memory elements, etc., such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0067] The computer-readable medium may be a tangible device capable of storing instructions to be executed by a suitable device. Thus, a computer-readable medium having instructions stored thereon is a product containing executable instructions to provide the means for performing the operations indicated by flowcharts and block diagrams. Examples of the computer-readable medium may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic recording medium, a magnetic recording medium, an optical recording medium, an electromagnetic recording medium, a semiconductor recording medium, and the like.Specific examples of the computer-readable medium may be a floppy disk (registered trademark), a floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an electrically erasable programmable read-only memory (EEPROM), a static random access memory (SRAM), a portable CD read-only memory (CD-ROM), a digital versatile disk (DVD), a Blu-ray disk (registered trademark), a memory stick, an integrated circuit card, or the like.

[0068] The computer-readable instructions may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, JAVA (registered trademark), JavaScript (registered trademark), C++, or the like, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0069] The computer-readable instructions may be provided to a processor or programmable circuit of a general-purpose computer, special-purpose computer, or other programmable data processing device to produce a machine either locally over a local area network (LAN) or over a wide area network (WAN), such as the Internet, and may be executed to generate the means for performing the operations indicated by the flowcharts and block diagrams. Examples of the processor include a computer processor, a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, and the like.

[0070] Fig.8 shows an example of a computer 2200 in which aspects of the present invention may be embodied in whole or in part. A program installed in the computer 2200 may cause the computer 2200 to function as devices of the embodiments of the present invention or one or more portions thereof, or to perform operations associated with devices of the embodiments of the present invention, and / or cause the computer 2200 to perform processes of the embodiments of the present invention or steps thereof. Such a program may be executed by the CPU 2212 to cause the computer 2200 to perform certain operations associated with some or all of the flowchart and block diagram blocks described herein.

[0071] The computer 2200 according to the present embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are mutually connected through a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0072] The CPU 2212 operates according to programs stored in the ROM 2230 and the RAM 2214, thereby controlling each unit. The graphics controller 2216 receives image data generated by the CPU 2212 in a frame buffer or the like provided in the RAM 2214 or within itself, and causes the image data to be displayed on the display device 2218.

[0073] The communication interface 2222 communicates with other electronic devices over a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 within the computer 2200. The DVD-ROM drive 2226 reads the programs or data from the DVD-ROM 2201 and supplies the hard disk drive 2224 with the programs or data via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to the IC card.

[0074] The ROM 2230 stores therein a boot program or the like that is executed by the computer 2200 when activated, and / or a program depending on the hardware of the computer 2200. The input / output chip 2240 can also connect various input / output devices to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0075] A program is provided by computer-readable media such as the DVD-ROM 2201 or the IC card. The program is read from the computer-readable media, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. The information processing described in these programs is read into the computer 2200, resulting in cooperation between a program and the various types of hardware resources mentioned above. An apparatus or method can be constituted by implementing the operation or processing of information according to the use of the computer 2200.

[0076] For example, when communication is performed between the computer 2200 and an external device, the CPU 2212 may execute a communication program loaded into the RAM 2214 to instruct the communication processing to the communication interface 2222 based on the processing described in the communication program. The communication interface 2222, under the control of the CPU 2212, reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or the IC card, and transmits the read transmission data to a network or writes reception data received from a network into a reception buffer area or the like provided on the recording medium.

[0077] In addition, the CPU 2212 can cause all or a necessary part of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), the IC card, etc., to be read into the RAM 2214, and perform various types of processing on the data in the RAM 2214. The CPU 2212 can then write the processed data back to the external recording medium.

[0078] Various types of information, such as various types of programs, data, tables, and databases, can be stored in the recording medium to undergo information processing. The CPU 2212 can perform various types of processing on the data read from the RAM 2214, including various types of operations, information processing, condition judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and denoted by an instruction sequence of programs, and write the result back to the RAM 2214. In addition, the CPU 2212 can search for information in a file, a database, etc. on the recording medium.For example, if a plurality of entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 2212 may search for an entry that satisfies the condition of which the attribute value of the first attribute is designated, from the plurality of entries, and reads the attribute value of the second attribute stored in the entry, thereby obtaining the attribute value of the second attribute associated with the first attribute that satisfies the predetermined condition.

[0079] The program or software modules discussed above may be stored in computer-readable media on or near the computer 2200. Furthermore, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communications network or the Internet may be used as a computer-readable medium, thereby providing the program to the computer 2200 over the network.

[0080] While the embodiments of the present invention have been described, the technical scope of the invention is not limited to the above-described embodiments. Those skilled in the art will appreciate that various modifications and improvements can be made to the above-described embodiments. It should also be apparent from the scope of the claims that the embodiments added with such modifications or improvements are within the technical scope of the invention.

[0081] The operations, procedures, steps, and stages of each process performed by an apparatus, system, program, and method shown in the claims, embodiments, or diagrams may be performed in any order, as long as the order is not indicated by "before," "previous," or the like, and as long as the output of a previous process is not used in a subsequent process. Even if the process flow is described in the claims, embodiments, or diagrams using terms such as "first" or "next," this does not necessarily mean that the process must be performed in that order. Explanation of reference symbols 100 test fixtures 110 Target Image Procurement Section 120 Target image masking section 130 Masking area prediction section 140 Replica image generation section 150 Difference Detection Section 160 Assessment Section 600 learning device 610 Training image acquisition section 620 Training image masking section 630 prediction model 640 Error calculation section 650 Model Update Section 2200 computers 2201 DVD-ROM 2210 host controller 2212 CPU 2214 RAM 2216 graphics controller 2218 Display device 2220 Input / Output Controller 2222 Communication interface 2224 hard disk drive 2226 DVD-ROM drive 2230 ROM 2240 input / output chip 2242 keyboard

Claims

[1] Test device comprising: a target image acquisition section that acquires a target image obtained by capturing a test target, a target image masking section that masks a part of the target image, a masking area prediction section that predicts an image of a masked area masked in the target image, a replica image generating section that generates a replica image using a plurality of predicted images each predicted for a plurality of masked areas including the masked area, and a difference detecting section that detects a difference between the target image and the replicated image, wherein the target image masking section sequentially masks one cell at a time from a plurality of cells obtained by dividing the target image, and generates a plurality of masked images obtained by masking each of the plurality of cells, the masking area prediction section predicts the image of the masked area for each of the plurality of cells and generates a plurality of predicted images, and the replica image generating section generates the replica image by arranging each of the plurality of predicted images, each predicted for the masked area of the plurality of cells, at the position of each of the plurality of cells. [2] The inspection apparatus according to claim 1, wherein the difference detecting section compares the target image with the replicated image in each predetermined area to calculate a difference amount in each predetermined area. [3] The inspection apparatus according to claim 2, further comprising a judging section that judges that the inspection target is unacceptable when the difference amount does not meet a predetermined quality standard. [4] The inspection apparatus according to claim 3, wherein the judging section judges that the inspection target is unacceptable when a largest difference measure among the difference measures of each predetermined range exceeds a predetermined threshold value. [5] The inspection apparatus according to claim 3 or 4, wherein the judging section, when the judging section judges that the inspection target is unacceptable, predicts an electrical characteristic of the inspection target from the target image obtained by capturing the inspection target judged to be unacceptable, and confirms that the inspection target is unacceptable if the electrical characteristic does not meet a predetermined quality standard. [6] The inspection apparatus according to any one of claims 3 to 5, wherein the judging section, when the judging section judges that the inspection target is unacceptable, predicts an electrical characteristic of the inspection target from the target image obtained by capturing the inspection target judged to be unacceptable, and confirms that the inspection target is acceptable if the electrical characteristic meets a predetermined quality standard. [7] Test device comprising: a target image acquisition section that acquires a target image obtained by capturing a test target, a target image masking section that masks a part of the target image, a masking area prediction section that predicts an image of a masked area masked in the target image, a replica image generating section that generates a replica image using a plurality of predicted images each predicted for a plurality of masked areas including the masked area, and a difference detecting section that detects a difference between the target image and the replicated image, wherein the difference detecting section compares the target image with the replicated image in each predetermined area to calculate a difference measure in each predetermined area, wherein the testing device further comprises a judging section which judges that the test target is unacceptable if the difference measure does not meet a predetermined quality standard, wherein, when the judging section judges that the test target is unacceptable, the judging section predicts an electrical property of the test target from the target image obtained by capturing the test target judged to be unacceptable, and confirms that the test target is unacceptable if the electrical property does not meet a predetermined quality standard, and / or wherein, when the judging section judges that the inspection target is unacceptable, the judging section predicts an electrical property of the inspection target from the target image obtained by capturing the inspection target judged to be unacceptable, and confirms that the inspection target is acceptable if the electrical property meets a predetermined quality standard. [8] The inspection apparatus according to any one of claims 2 to 7, wherein the difference detecting section outputs a detection image in which a display attribute differs in each predetermined area according to the difference amount. [9] The inspection apparatus according to any one of claims 2 to 8, wherein the difference detecting section calculates the difference measure based on a Euclidean distance between the target image and the replicated image. [10] The inspection apparatus according to any one of claims 1 to 9, wherein the target image acquiring section acquires an image obtained by performing grayscale conversion on an image obtained by capturing the inspection target as the target image. [11] The inspection apparatus according to any one of claims 1 to 10, wherein the target image acquiring section acquires an image as a target image obtained by performing object detection to define a target area. [12] Testing procedures, including: Obtaining a target image obtained by capturing a test target, Masking part of the target image, Predicting an image of a masked area that is masked in the target image, Generating a replicated image using a plurality of predicted images each predicted for a plurality of masked areas including the masked area, and Detecting a difference between the target image and the simulated image, wherein masking a part of the target image comprises successively masking each cell from a plurality of cells obtained by dividing the target image and generating a plurality of masked images obtained by masking each of the plurality of cells, predicting the image of the masked area comprises predicting the image of the masked area for each of the plurality of cells and generating a plurality of predicted images, and generating the replicated image comprises generating the replicated image by arranging each of the plurality of predicted images, each predicted for the masked area of the plurality of cells, at the position of each of the plurality of cells. [13] Testing procedures, including: Obtaining a target image obtained by capturing a test target, Masking part of the target image, Predicting an image of a masked area that is masked in the target image, Generating a replicated image using a plurality of predicted images each predicted for a plurality of masked areas including the masked area, and Detecting a difference between the target image and the simulated image, Comparing the target image with the replicated image in each given area to calculate a difference measure in each given area, Judging the test target as unacceptable if the difference measure does not meet a predetermined quality standard, and if the inspection target is judged to be unacceptable, predicting an electrical property of the test target from the target image obtained by capturing the test target that was judged to be unacceptable, and confirming that the test target is unacceptable if the electrical property does not meet a predetermined quality standard, and / or if the inspection target is judged to be unacceptable, predicting an electrical property of the inspection target from the target image obtained by capturing the inspection target judged to be unacceptable, and confirming that the inspection target is acceptable if the electrical property meets a predetermined quality standard. [14] A test program which, when executed by a computer, causes the computer to act as: a target image acquisition section that acquires a target image obtained by capturing a test target, a target image masking section that masks a part of the target image, a masking area prediction section that predicts an image of a masked area masked in the target image, a replica image generating section that generates a replica image using a plurality of predicted images each predicted for a plurality of masked areas including the masked area, and a difference detection section that detects a difference between the target image and the replicated image, wherein the target image masking section sequentially masks one cell at a time from a plurality of cells obtained by dividing the target image, and generates a plurality of masked images obtained by masking each of the plurality of cells, the masking area prediction section predicts the image of the masked area for each of the plurality of cells and generates a plurality of predicted images, and the replica image generating section generates the replica image by arranging each of the plurality of predicted images, each predicted for the masked area of the plurality of cells, at the position of each of the plurality of cells. [15] A test program which, when executed by a computer, causes the computer to act as: a target image acquisition section that acquires a target image obtained by capturing a test target, a target image masking section that masks a part of the target image, a masking area prediction section that predicts an image of a masked area masked in the target image, a replica image generating section that generates a replica image using a plurality of predicted images each predicted for a plurality of masked areas including the masked area, and a difference detection section that detects a difference between the target image and the replicated image, wherein the difference detecting section compares the target image with the replicated image in each predetermined area to calculate a difference measure in each predetermined area, wherein the test program, when executed by a computer, further causes the computer to function as an assessment section which judges that the test objective is unacceptable if the difference measure does not meet a predetermined quality standard, wherein, when the judging section judges that the test target is unacceptable, the judging section predicts an electrical property of the test target from the target image obtained by capturing the test target judged to be unacceptable, and confirms that the test target is unacceptable if the electrical property does not meet a predetermined quality standard, and / or wherein, when the judging section judges that the inspection target is unacceptable, the judging section predicts an electrical property of the inspection target from the target image obtained by capturing the inspection target judged to be unacceptable, and confirms that the inspection target is acceptable if the electrical property meets a predetermined quality standard. [16] Learning device comprising: a training image acquisition section that acquires a training image, a training image masking section that masks a part of the training image, a prediction model that receives the masked training image and outputs a model image obtained by predicting the training image, and a model update section that updates the prediction model based on an error between the training image and the model image, wherein the training image masking section randomly masks one cell from a plurality of cells obtained by dividing the training image and generates a masked image obtained by randomly masking one cell from the plurality of cells, and the prediction model predicts an image of a masked region of the masked image and outputs the model image by embedding the predicted image into a masked cell of the training images. [17] Learning methods, including: Obtaining a training image, Masking part of the training image, Inputting the masked training image into a prediction model and outputting a model image obtained by predicting the training image, and Updating the prediction model based on an error between the training image and the model image, wherein masking a portion of the training image comprises randomly masking one cell from a plurality of cells obtained by dividing the training image and generating a masked image obtained by randomly masking one cell from the plurality of cells, and the prediction model predicts an image of a masked portion of the masked image and outputs the model image by embedding the predicted image into a masked cell of the training images. [18] A learning program which, when executed by a computer, causes the computer to function as: a training image acquisition section that acquires a training image, a training image masking section that masks a part of the training image, a prediction model that receives the masked training image and outputs a model image obtained by predicting the training image, and a model update section that updates the prediction model based on an error between the training image and the model image, wherein the training image masking section randomly masks one cell from a plurality of cells obtained by dividing the training image and generates a masked image obtained by randomly masking one cell from the plurality of cells, and the prediction model predicts an image of a masked region of the masked image and outputs the model image by embedding the predicted image into a masked cell of the training images.

Images