Inspection device, inspection method, and inspection program, as well as learning device, learning method, and learning program

Abstract

There is provided an inspection device including: an object image acquisition unit that acquires an object image obtained by photographing an inspection object; an object image masking unit that masks a part of the object image; a masked area prediction unit that predicts an image of a masked area masked in the object image; a reproduced image generation unit that generates a reproduced image using a plurality of predicted images predicted for respective ones of a plurality of masked areas; and a difference detection unit that detects a difference between the object image and the reproduced image.

Description

Technical Field

[0001] The present invention relates to an inspection device, an inspection method, an inspection program, a learning device, a learning method, and a learning program. Background Art

[0002] Previously, visual inspections were performed to determine the quality of inspection objects. For example, in the visual inspection of probes applicable to semiconductor testing, an image of the probe is digitized, and the digitized data is evaluated according to a pre-determined rule to determine the quality of the probe.

[0003] Problems to be Solved by the Invention

[0004] However, when inspecting an inspection object, it is desirable to easily grasp the state of the inspection object. Summary of the Invention

[0005] To solve the above problems, a first aspect of the present invention provides an inspection device. The inspection device may include an object image acquisition unit that acquires an object image obtained by photographing an inspection object. The inspection device may include an object image masking unit that masks a part of the object image. The inspection device may include a masked area prediction unit that predicts an image of a masked area masked in the object image. The inspection device may include a reproduced image generation unit that generates a reproduced image using a plurality of predicted images predicted for each of a plurality of masked areas. The inspection device may include a difference detection unit that detects a difference between the object image and the reproduced image.

[0006] The difference detection unit may compare the object image and the reproduced image for each pre-determined area and calculate the degree of difference for each pre-determined area.

[0007] The inspection device may further include a determination unit that determines the inspection object as a defective product when the degree of difference does not meet a pre-determined quality.

[0008] The determination unit may determine the inspection object as a defective product when the maximum degree of difference among the degrees of difference for each pre-determined area exceeds a pre-determined threshold.

[0009] When the inspection object has been determined to be a defective product, the determination unit predicts the electrical characteristics of the inspection object using the object image obtained by photographing the inspection object determined to be a defective product, and may determine the inspection object as a defective product when the electrical characteristics do not meet a pre-determined quality.

[0010] When the inspection object has been determined to be a defective product, the determination unit predicts the electrical characteristics of the inspection object using the object image obtained by photographing the inspection object determined to be a defective product, and may determine the inspection object as a non-defective product when the electrical characteristics meet a pre-determined quality.

[0011] The difference detection unit outputs a detection map, and the display state of each predetermined area is different according to the degree of the difference corresponding to the detection map.

[0012] The difference detection unit calculates the degree of difference based on the Euclidean distance between the object image and the reproduced image.

[0013] The object image masking unit sequentially masks one cell among a plurality of cells obtained by dividing the object image; the reproduced image generation unit generates a reproduced image using a plurality of predicted images predicted for each of the different cells.

[0014] The object image acquisition unit acquires an image obtained by gray-scaling an image obtained by photographing an inspection object as the object image.

[0015] The object image acquisition unit acquires an image in which the object area is narrowed by performing object detection on the inspection object in the image obtained by photographing the inspection object as the object image.

[0016] A second aspect of the present invention provides an inspection method. The inspection method may include the following steps: acquiring an object image obtained by photographing an inspection object; masking a part of the object image; predicting an image of a masked area in the object image; generating a reproduced image using a plurality of predicted images predicted for each of the plurality of masked areas; and detecting a difference between the object image and the reproduced image.

[0017] A third aspect of the present invention provides an inspection program. The inspection program can be executed by a computer and cause the computer to function as the following means: an object image acquisition unit that acquires an object image obtained by photographing an inspection object; an object image masking unit that masks a part of the object image; a masked area prediction unit that predicts an image of a masked area in the object image; a reproduced image generation unit that generates a reproduced image using a plurality of predicted images predicted for each of the plurality of masked areas; and a difference detection unit that detects a difference between the object image and the reproduced image.

[0018] A fourth aspect of the present invention provides a learning device. The learning device may include: a training image acquisition unit that acquires a training image; a training image masking unit that masks a part of the training image; a prediction model that is input with the masked training image and outputs a model image predicted for the training image; and a model update unit that updates the prediction model based on the error between the training image and the model image.

[0019] A fifth aspect of the present invention provides a learning method. The learning method may include the following steps: obtaining training images; masking a part of the training images; inputting the masked training images and outputting model images predicted for the training images; and updating the prediction model based on the error between the training images and the model images.

[0020] A sixth aspect of the present invention provides a learning program. The learning program can be executed by a computer and cause the computer to function as the following means: a training image acquisition unit that acquires training images; a training image masking unit that masks a part of the training images; a prediction model that is input with the masked training images and outputs model images predicted for the training images; and a model update unit that updates the prediction model based on the error between the training images and the model images.

[0021] In addition, the above description of the invention does not enumerate all the essential features of the present invention. In addition, sub-combinations of these feature groups can also be inventions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 An example of a block diagram showing the inspection device 100 of the present embodiment.

[0023] Figure 2 An example of a flowchart showing the process of the inspection device 100 of the present embodiment inspecting an inspection object.

[0024] Figure 3 An example showing object image 310, masked image 320, predicted image 330, and reproduced image 340 in the inspection using the inspection device 100 of the present embodiment.

[0025] Figure 4 An example of the inspection result when the inspection object is a good product in the present embodiment.

[0026] Figure 5 An example of the inspection result when the inspection object is a defective product in the present embodiment.

[0027] Figure 6 An example of a block diagram showing the learning device 600 of the present embodiment.

[0028] Figure 7 An example of a flowchart showing the process of the learning device 600 of the present embodiment learning the prediction model 630.

[0029] Figure 8 An example of a computer 2200 that can embody the present invention in whole or in part. DETAILED DESCRIPTION OF THE INVENTION

[0030] Hereinafter, the present invention will be described by way of embodiments of the invention, but the following embodiments do not limit the invention claimed. In addition, all combinations of the features described in the embodiments are not necessarily required in the solution of the invention.

[0031] Figure 1 FIG. 4 shows an example of a block diagram of the inspection apparatus 100 according to the present embodiment. The inspection apparatus 100 can easily grasp the state of the inspection object by detecting the difference between the image obtained by photographing the actual state of the inspection object and the image generated by predicting the state that the detection object should be in. The example described in the present embodiment is a case where the inspection apparatus 100 uses a probe suitable for semiconductor testing as the inspection object. However, it is not limited to such an embodiment. The inspection apparatus 100 can also be used for image analysis of bumps in semiconductor devices, pattern inspection in wiring boards, etc., or inspection of other electrical components, and can also be used for inspection of any article different from electrical components.

[0032] The inspection apparatus 100 can be a computer such as a PC (personal computer), a tablet computer, a smart phone, a workstation, a server computer, or a general-purpose computer, or can also be a computer system in which a plurality of computers are connected. Such a computer system is also a computer in a broad sense. In addition, the inspection apparatus 100 can also be configured in a computer through one or a plurality of executable virtual computer environments. As an alternative to the above, the inspection apparatus 100 can also be a dedicated computer designed for inspection of an inspection object, or can also be dedicated hardware implemented by a dedicated circuit. In addition, when the inspection apparatus 100 can be connected to the Internet, the inspection apparatus 100 can also be implemented through cloud computing.

[0033] The inspection apparatus 100 includes: an object image acquisition unit 110, an object image masking unit 120, a masking area prediction unit 130, a reproduced image generation unit 140, a difference detection unit 150, and a determination unit 160.

[0034] The object image acquisition unit 110 acquires an object image obtained by photographing the inspection object. The object image acquisition unit 110 can also acquire, as the object image, an image obtained by preprocessing the image obtained by photographing the inspection object. At this time, the object image acquisition unit 110 can acquire the object image via a network, for example, can acquire the object image via user input, or can acquire the object image via a storage device capable of storing data. And the object image acquisition unit 110 supplies the acquired object image to the object image masking unit 120 and the difference detection unit 150.

[0035] The object image shielding unit 120 shields a part of the object image. Moreover, the object image shielding unit 120 supplies a plurality of shielded images, each of which shields a different part of the object image, to the shielding area prediction unit 130.

[0036] The shielding area prediction unit 130 predicts the image of the shielded area in the object image. Moreover, the shielding area prediction unit 130 supplies a plurality of predicted images, which predict each of the shielding areas in the plurality of shielded images, to the reproduced image generation unit 140.

[0037] The reproduced image generation unit 140 generates a reproduced image using the plurality of predicted images predicted for each of the plurality of shielding areas. The reproduced image generation unit 140 supplies the generated reproduced image to the difference detection unit 150.

[0038] The difference detection unit 150 detects the difference between the object image and the reproduced image. At this time, the difference detection unit 150 can, for example, compare the object image supplied from the object image acquisition unit 110 and the reproduced image supplied from the reproduced image generation unit 140 for each predetermined area, and calculate the degree of difference for each predetermined area. Moreover, the difference detection unit 150 supplies the calculated degree of difference to the determination unit 160.

[0039] When the degree of difference does not meet the predetermined quality, the determination unit 160 determines that the inspection object is a defective product. Moreover, the determination unit 160 outputs the determination result to other functional units, other devices, etc. The case of inspecting an inspection object using such an inspection device 100 will be described in detail using a flow.

[0040] Figure 2 An example of the flow of the inspection device 100 of the present embodiment inspecting an inspection object is shown. In step 210, the object image acquisition unit 110 acquires an object image obtained by photographing the inspection object. As an example, the object image acquisition unit 110 acquires, via a network, an image obtained by photographing a probe applicable to a semiconductor test using an optical microscope or the like.

[0041] Further, the object image acquisition unit 110 grayscales the image obtained by photographing the object to be inspected. The image obtained by photographing the probe using an optical microscope or the like may include three color channels corresponding to each of the colors R (red), G (green), and B (blue). However, from the perspective of the inspection of the present embodiment, these three color channels have almost the same characteristics, and none of the color channels has unique characteristics compared to the other color channels. Therefore, the object image acquisition unit 110 single-colorizes the acquired image by grayscaling the acquired image. In this way, the object image acquisition unit 110 can obtain the image obtained by grayscaling the image obtained by photographing the object to be inspected as the object image. When the inspection device 100 acquires the object image, it can create a single color channel by grayscaling the acquired image to reduce the burden of the inspection process. In addition, in the case where it is preferable to use a plurality of color channels, for example, when improving the accuracy of the inspection, etc., the object image acquisition unit 110 may also obtain the image obtained by photographing the object to be inspected as the object image without grayscaling it.

[0042] In addition, the object image acquisition unit 110 narrows down the image obtained by photographing the object to be inspected. For example, the object image acquisition unit 110 uses an object detection algorithm such as Yolo (You Only Look Once) to identify the position and size of the probe in the image. And the object image acquisition unit 110 clips the image based on the identified position and size of the probe to narrow the object area. In this way, the object image acquisition unit 110 can obtain, in the image obtained by photographing the object to be inspected, the image with the object area narrowed by object detection of the object to be inspected as the object image. In this way, when the inspection device 100 acquires the object image, it can narrow the object area by object detection of the object to be inspected to achieve high-speed inspection processing and improve the accuracy of the inspection. The object image acquisition unit 110 acquires the image that has been pre-processed (such as grayscaling and narrowing) as the object image and supplies the object image to the object image masking unit 120 and the difference detection unit 150.

[0043] In step 220, the object image masking unit 120 masks a part of the object image. As an example, the object image masking unit 120 divides the object image acquired in step 210 into a plurality of cells. And the object image masking unit 120 sequentially masks one cell out of the plurality of divided cells for the object image. The object image masking unit 120 supplies the plurality of masked images in which each of the plurality of cells has been masked to the masked area prediction unit 130.

[0044] In step 230, the occlusion area prediction unit 130 predicts the image of the occluded area that has been occluded in the object image. At this time, the occlusion area prediction unit 130 can use a learned model such as a CNN (Convolutional Neural Network), for example. This CNN only uses training images known to be good products, and is learned so that once an image with a partially occluded area is input, the image of the occluded area can be predicted from the images of other non-occluded areas. That is, the occlusion area prediction unit 130 can use a learned model that is learned to have the following characteristics: only using training images known to be good products, and being able to predict what state the occluded area should be in when the object to be inspected is a good product. For example, the occlusion area prediction unit 130 inputs the plurality of occluded images occluded in step 220 into the learned model respectively, and predicts the image of the occluded area for each of the plurality of cells. The learning of such a model will be described later. In addition, the example shown in the above description is the case where the occlusion area prediction unit 130 uses a learned CNN model, but it is not limited to such an embodiment. The occlusion area prediction unit 130 can use a learned model of other algorithms to predict the image of the occluded area, and can also use an algorithm different from learning to predict the image of the occluded area. The occlusion area prediction unit 130 supplies the plurality of predicted images predicted for each of the plurality of occluded areas to the reproduced image generation unit 140.

[0045] In step 240, the reproduced image generation unit 140 generates a reproduced image using the plurality of predicted images predicted for each of the plurality of occluded areas. As an example, the reproduced image generation unit 140 generates a reproduced image using the plurality of predicted images predicted for each of different cells. At this time, for example, the reproduced image generation unit 140 can reconfigure the plurality of predicted images predicted for each of the plurality of cells in step 230 to the positions of the original cells, thereby generating a reproduced image. The reproduced image generation unit 140 supplies the generated reproduced image to the difference detection unit 150.

[0046] In step 250, the difference detection unit 150 detects the difference between the object image and the reproduced image. As an example, for each predetermined area (such as each pixel, each pixel group, and each cell used when masking the object image, etc.), the difference detection unit 150 compares the object image supplied from the object image acquisition unit 110 and the reproduced image supplied from the reproduced image generation unit 140, and calculates the degree of difference in each predetermined area. At this time, the difference detection unit 150 can calculate the degree of difference based on the L2 norm, that is, the Euclidean distance, between the object image and the reproduced image. In addition, the difference detection unit 150 can output a detection map, which makes the display state (such as color or density, etc.) of each predetermined area different corresponding to the degree of difference. And the difference detection unit 150 supplies the degree of difference in each predetermined area to the determination unit 160.

[0047] In step 260, the determination unit 160 determines whether the degree of difference calculated in step 250 is below a predetermined threshold. And when the degree of difference is below the predetermined threshold, that is, when the degree of difference meets the predetermined quality, the determination unit 160 proceeds to step 270, determines that the object to be inspected is a good product, and ends the process. On the other hand, when the degree of difference exceeds the predetermined threshold, that is, when the degree of difference does not meet the predetermined quality, the determination unit 160 proceeds to step 280, determines that the object to be inspected is a defective product, and ends the process. At this time, for example, the determination unit 160 can determine that the object to be inspected is a defective product when the maximum degree of difference among the degrees of difference in each predetermined area exceeds the predetermined threshold. And the determination unit 160 outputs the determination result to other functional units and other devices, etc. In addition, the threshold for this determination can be the minimum value that can be obtained when calculating the degree of difference by the inspection device of this embodiment and using the image obtained by photographing the object to be inspected known to be a defective product, or a value slightly smaller than this minimum value. In addition, in the example shown in the above description, the determination unit 160 determines the quality of the object to be inspected based on the maximum degree of difference among the degrees of difference in each predetermined area, but is not limited to such an embodiment. The determination unit 160 can also determine the quality of the object to be inspected based on other statistical values of the degree of difference, such as the median and the average value, or the distribution.

[0048] Figure 3This shows an example of the object image 310, the occlusion image 320, the prediction image 330, and the reproduction image 340 in the inspection using the inspection apparatus 100 of the present embodiment. The object image acquisition unit 110 acquires, for example, the object image 310 as shown in this figure. The object image occlusion unit 120 divides the object image 310 into a plurality of cells 322 (a total of 25 cells of [vertical, horizontal]=[1,1] to [5,5] in this figure). Then, the object image occlusion unit 120 sequentially occludes each cell 322 using the occlusion mask 324 to generate a plurality of occlusion images 320 respectively. As an example, the upper, middle, and lower parts of this figure respectively show the cases where the cell [2,3], the cell [3,3], and the cell [4,3] are occluded. Further, the occlusion area prediction unit 130 predicts the image of the occlusion area 332 for each of the plurality of cells 322 and generates a plurality of prediction images 330. Then, the reproduction image generation unit 140 reconfigures the plurality of prediction images 330 predicted for each of the plurality of cells 322 to the positions of the original cells 322 to thereby generate the reproduction image 340.

[0049] Figure 4 This shows an example of the inspection result when the inspection object is a good product in the present embodiment. When the inspection object is a good product, the reproduction image 340 generated by the inspection apparatus 100 is almost the same as the object image 310. This is because the inspection apparatus 100 uses a learned model that has been learned only through training images known to be good products to predict the state that the occlusion area should be in when the inspection object is a good product. Therefore, the reproduction image 340 generated by the inspection apparatus 100 is an image that reproduces the state that the inspection object should have when it is a good product. Thus, in the detection map 400 in which the display pattern of each predetermined area varies according to the degree of difference, the display patterns in all areas become almost the same. Further, in the distribution (the lower part of this figure) that counts the number of unit areas (the cell unit area in this figure) for each degree of difference, most of the cells are counted near the degree of difference = 0, and no cells are counted at positions where the degree of difference > 1.5 (threshold). Here, the unit area can be, for example, an area such as a pixel unit, a pixel group unit, and a cell unit. In this figure, as an example, it shows the case where the unit area is the cell unit area. Further, the wider the vertical axis, the more cells are counted. The inspection apparatus 100 determines that the inspection object in the object image 310 is a good product when the difference between the object image 310 and the reproduction image 340 is small like this.

[0050] Figure 5This shows an example of the inspection result when the object to be inspected is a defective product in this embodiment. When the object to be inspected is a defective product (for example, when the object to be inspected includes a cracked condition), the reproduced image 340 generated by the inspection device 100 may be different from the object image 310. At this time, in the detection map 400, regions with different degrees of difference may also be displayed in different forms. In addition, in the distribution of the degree of difference, there are also multiple cells counted at positions with a larger degree of difference, and several cells are counted at positions where the degree of difference > 1.5. When the difference between the object image 310 and the reproduced image 340 is large like this, the inspection device 100 determines that the object to be inspected within the object image 310 is a defective product.

[0051] Thus, according to the inspection device 100 of this embodiment, by detecting the difference between the image obtained by photographing the actual state of the object to be inspected (object image 310) and the image predicted and reproduced for the state that the object to be inspected should have when it is a non-defective product (reproduced image 340), the state of the object to be inspected can be easily grasped. In addition, since the inspection device 100 outputs the detection map 400 with different display forms for each region corresponding to the degree of difference, the defective position of the object to be inspected can be easily grasped. In addition, since the inspection device 100 outputs the distribution that counts the number of unit regions for each degree of difference, it can be grasped at what frequency regions with different degrees of difference appear in the image.

[0052] Such an inspection using the inspection device 100 of this embodiment can be implemented, for example, before the detector (probe) is shipped during the manufacturing process of the detector, or before or during the test of the actual semiconductor component using the detector. In addition, in a case where it is useful to perform an appearance inspection using images from multiple directions, the inspection device 100 can also use images obtained by photographing the object to be inspected from a plurality of directions to perform this inspection, thereby grasping the state of the object to be inspected in more detail.

[0053] In addition, in the example shown in the above description, the inspection device 100 determines the quality of the object to be inspected only based on the appearance inspection using images, but is not limited to such an embodiment. The inspection device 100 can also determine the quality of the object to be inspected based on both the appearance inspection using images and the electrical characteristics.

[0054] As an example, when the determination unit 160 determines that the inspection object is a defective product, the object image obtained by photographing the inspection object determined to be a defective product can be used to predict the electrical characteristics of the inspection object, and when the electrical characteristics do not meet the predetermined quality, the inspection object is determined to be a defective product. For example, if the determination unit 160 is input with the object image obtained by photographing the inspection object, it can use the learned model that has been learned in a way that can predict the electrical characteristics of the inspection object. That is, if the determination unit 160 is input with the object image obtained by photographing the probe, it can use the learned model that has been learned in a way that can predict the electrical characteristics such as the resistance value of the probe. And, in Figure 2 In step 280, when the determination unit 160 determines that the probe is a defective product, the object image obtained by photographing the probe that has been determined to be a defective product is input to the learned model, and the resistance value of the probe is predicted. When the resistance value does not meet the predetermined quality, the probe of the inspection object is determined to be a defective product. In addition, in the example shown in the above description, it is the case where the determination unit 160 uses the learned model to predict the electrical characteristics of the inspection object, but it is not limited to such an embodiment. The determination unit 160 can also predict the electrical characteristics of the inspection object from the object image using an algorithm different from learning.

[0055] Similarly, when the determination unit 160 determines that the inspection object is a defective product, the object image obtained by photographing the inspection object determined to be a defective product is used to predict the electrical characteristics of the inspection object, and when the electrical characteristics meet the predetermined quality, the inspection object is determined to be a non-defective product. For example, in Figure 2 In step 280, when the determination unit 160 determines that the probe is a defective product, the object image obtained by photographing the probe that has been determined to be a defective product can be input to the learned model, and the resistance value of the probe is predicted. When the resistance value meets the predetermined quality, the probe of the inspection object is determined to be a non-defective product.

[0056] Thus, the inspection device 100 can correctly determine the quality of the inspection object not only by considering the visual inspection using the image, but also by considering the electrical characteristics. For example, even if the inspection object is determined to be a defective product in the visual inspection using the image due to the influence of shadows or particles that are incident when photographing the inspection object, its electrical characteristics may not be a problem. Therefore, the inspection device 100 can more correctly determine the quality of the inspection object by combining the visual inspection and the electrical characteristics.

[0057] Figure 6An example of a block diagram showing the learning device 600 of the present embodiment. The learning device 600 uses only training images known to be good products, and learns a model in such a way that if an image with a part of the area obscured is input, it can predict the obscured area from the images of other non-obscured areas. That is, the learning device 600 learns a model using only training images known to be good products in such a way that it can predict what the state of a part of the obscured area should be when the inspection object is a good product. The inspection device 100 of the present embodiment, as an example, can use the learned model learned by the learning device 600 as shown in this figure to predict the obscured area.

[0058] The learning device 600 can be a computer such as a PC (personal computer), a tablet computer, a smartphone, a workstation, a server computer, or a general-purpose computer, or can also be a computer system in which a plurality of computers are connected. Such a computer system is also a computer in a broad sense. In addition, the learning device 600 can also be configured in a computer through one or a plurality of executable virtual computer environments. As an alternative to the above, the learning device 600 can also be a dedicated computer designed for learning a model, or can also be a dedicated hardware implemented through a dedicated circuit. In addition, when the learning device 600 can be connected to the Internet, the learning device 600 can also be implemented through cloud computing.

[0059] The learning device 600 includes: a training image acquisition unit 610, a training image masking unit 620, a prediction model 630, an error calculation unit 640, and a model update unit 650.

[0060] The training image acquisition unit 610 acquires training images. As an example, the training image acquisition unit 610 can acquire a plurality of images known to be of good inspection objects as training images. At this time, the training image acquisition unit 610 can acquire training images via a network, for example, can also acquire training images via user input, or can acquire training images via a storage device capable of storing data, etc. And the training image acquisition unit 610 supplies the acquired training images to the training image masking unit 620 and the error calculation unit 640.

[0061] The training image masking unit 620 masks a part of the training images. As an example, the training image masking unit 620 randomly masks a plurality of images acquired as training images, and supplies the plurality of masked images to the prediction model 630.

[0062] The prediction model 630 is input with a training image in which a part of the area is occluded, and outputs a model image obtained by predicting the training image. As an example, when the prediction model 630 is input with a training image in which a part of the area is occluded, it predicts what state the occluded area should be in when the inspection object is a good product, and outputs a model image obtained by predicting the training image. At this time, in predicting the occluded area, the prediction model 630 can use an algorithm such as CNN, for example. In addition, in the above description, an example in which the prediction model 630 uses CNN is described, but it is not limited to such an implementation. The prediction model 630 can also use an algorithm different from CNN to predict the image of the occluded area. The prediction model 630 supplies the output model image to the error calculation unit.

[0063] The error calculation unit 640 calculates the error between the training image supplied from the training image acquisition unit 610 and the model image supplied from the prediction model 630. The error calculation unit 640 supplies the calculated error to the model update unit 650.

[0064] The model update unit 650 updates the prediction model 630 based on the error between the training image supplied from the training image acquisition unit 610 and the model image supplied from the prediction model 630. The case of learning the model using such a learning device 600 will be described in detail using a flowchart.

[0065] Figure 7 An example of the process of the learning device 600 of the present embodiment learning the prediction model 630 is shown. In step 710, the training image acquisition unit 610 acquires a training image. As an example, the training image acquisition unit 610 acquires a plurality of known images of a good inspection object via a network. And, the training image acquisition unit 610, similar to the object image acquisition unit 110, acquires an image obtained by preprocessing the acquired image as a training image. At this time, the training image acquisition unit 610 can use a contour detection filter or the like to discard an image with a focus deviation from the inspection object without acquiring it as a training image. The training image acquisition unit 610 supplies the acquired training image to the training image occlusion unit 620 and the error calculation unit 640.

[0066] In step 720, the training image occlusion unit 620 occludes a part of the training image. For example, the training image occlusion unit 620 randomly selects from among the plurality of images acquired as training images. Next, for each of the randomly selected images, the training image occlusion unit 620 randomly occludes one of the plurality of cells obtained by dividing the image area. And, the training image occlusion unit 620 supplies the plurality of occluded images obtained by randomly occluding the randomly selected images to the prediction model 630.

[0067] In step 730, the prediction model 630 is input with the masked training image and outputs a model image that predicts the training image. For example, if the prediction model 630 is input with a masked image in which a part of the area is randomly masked, it predicts the image of the masked area from the image of the other unmasked areas. Further, the prediction model 630 outputs a model image by embedding the predicted image into the masked area of the training image. The prediction model 630 supplies the model image to the error calculation unit 640.

[0068] In step 740, the error calculation unit 640 calculates the error between the training image supplied from the training image acquisition unit 610 and the model image supplied from the prediction model 630. The error calculation unit 640 supplies the calculated error to the model update unit 650.

[0069] In step 750, the model update unit 650 updates the prediction model 630 based on the error between the training image supplied from the training image acquisition unit 610 and the model image supplied from the prediction model 630. For example, the model update unit 650 uses the error calculated in step 740 as an objective function and updates parameters such as weights in the prediction model 630 in such a way as to minimize the objective function.

[0070] In step 760, the learning device 600 determines whether the training has ended. When it is determined in step 760 that the training has not ended, the learning device 600 returns the process to step 710 and repeats the process. On the other hand, when it is determined in step 760 that the training has ended, the learning device 600 ends the process. At this time, the learning device 600 can determine whether the training has ended based on various conditions such as the training time, the number of training times, and the training accuracy.

[0071] Thus, according to the learning device 600 of the present embodiment, since only a plurality of images of known good inspection objects are used as training images and the prediction model 630 is updated in such a way as to minimize the error between the training image and the model image, the learning device 600 can update the prediction model 630 to be able to predict the image of the state that the masked area should be in when the inspection object is good.

[0072] Various embodiments of the present invention can be described with reference to flowcharts and block diagrams. Here, a block can represent (1) a stage of a process that performs an operation or (2) a section of a device responsible for performing the operation. Specific stages and sections can be constructed by the following: a dedicated circuit, a programmable circuit provided together with computer-readable instructions stored in a computer-readable medium, and / or a processor provided together with computer-readable instructions stored in a computer-readable medium. The dedicated circuit can include digital and / or analog hardware circuits and can also include integrated circuits (ICs) and / or discrete circuits. The programmable circuit can include a reconfigurable hardware circuit, which includes memory elements such as AND logic gates, OR logic gates, XOR logic gates, NAND logic gates, NOR logic gates, and other logical operations, flip-flops, buffers, field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc.

[0073] The computer-readable medium can include any physical device capable of storing instructions that can be executed by an appropriate device. As a result, a computer-readable medium having instructions stored therein is a product that includes instructions capable of being executed to generate means for performing the operations specified in the flowchart or block diagram. Examples of computer-readable media can include: electronic memory media, magnetic memory media, optical memory media, electromagnetic memory media, semiconductor memory media, etc. More specific examples of computer-readable media can include: floppy disk (registered trademark), magnetic disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), optical disc (CD-ROM), digital versatile disc (DVD), Blu-ray (RTM) disc, MS memory card, integrated circuit card, etc.

[0074] Computer-readable instructions include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microprogram code, firmware instructions, status setting data, or object-oriented programming languages such as Smalltalk, JAVA (registered trademark), C++, etc., and previous procedural programming languages such as the "C" programming language or equivalent programming languages. It can also include either source code or object code written in any combination of one or more programming languages.

[0075] Computer-readable instructions are provided to a processor or programmable circuitry of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, via a local or wide area network (LAN), the Internet, etc., and execute the computer-readable instructions to generate means for performing the operations specified in the flowchart or block diagram. Examples of processors include: computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0076] Figure 8 An example of a computer 2200 that can embody all or part of the present invention. The program installed in the computer 2200 enables the computer 2200 to perform operations associated with the apparatus of the embodiments of the present invention or function as one or more sections of the apparatus, or to execute the operation or one or more sections of the operation, and / or enable the computer 2200 to execute the process of this embodiment or a stage of the process. Such a program can be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with several or all of the blocks in the flowcharts and block diagrams described in this specification.

[0077] The computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, an image controller 2216, and a display device 2218, and these components are interconnected via a host controller 2210. The computer 2200 also includes an input / output unit such as a communication interface 2222, a hardware driver 2224, a DVD-ROM drive 2226, and an IC card drive, and these components are connected to the host controller 2210 by an input / output controller 2220. The computer also includes a ROM 2230 and a conventional input / output unit such as a keyboard 2242, and these components are connected to the input / output controller 2220 via an input / output chip 2240.

[0078] The CPU 2212 operates according to the programs stored in the ROM 2230 and the RAM 2214 and thereby controls each unit. The image controller 2216 acquires frame buffers, etc. provided in the RAM 2214, or image data generated by the CPU 2212 in the frame buffer, and displays the image data on the display device 2218.

[0079] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data to be used by the CPU 2212 within the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to the IC card.

[0080] The ROM 2230 stores therein a startup program to be executed by the computer 2200 at startup and / or programs dependent on the hardware of the computer 2200. The input / output chip 2240 can further connect various input / output units to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0081] Programs are provided via a computer-readable medium such as the DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium and installed into the hard disk drive 2224, the RAM 2214, or the ROM 2230, which can also be examples of computer-readable media, and are executed by the CPU 2212. The information processing written in these programs is read by the computer 2200 to achieve cooperation between the programs and the above various types of hardware data. The device or method can be constituted by using the computer 2200 to implement the operation or processing of information.

[0082] For example, in the case of performing communication between the computer 2200 and an external device, the CPU 2212 can execute a communication program read into the RAM 2214 and issue an instruction for communication processing to the communication interface 2222 based on the processing written in the communication program. Under the control of the CPU 2212, the communication interface 2222 reads the transmission data stored in the transmission buffer processing area and transmits the read transmission data to the network, or writes the received data received from the network to the reception buffer processing area, etc. The transmission buffer processing area and the reception buffer processing area are provided in a recording medium such as the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or the IC card.

[0083] In addition, the CPU 2212 can read all or a necessary part of a file or database stored in an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), the IC card, etc. into the RAM 2214 and perform various types of processing on the data in the RAM 2214. The CPU 2212 then writes the processed data back to the external recording medium.

[0084] Various types of information such as various types of programs, data, tables, and databases can be stored in a recording medium and undergo information processing. The CPU 2212 can perform various types of processing described throughout the present invention on the data read from the RAM 2214, and write the results back to the RAM 2214. The above processing includes various types of operations, information processing, conditional judgments, conditional branches, unconditional branches, information search / replacement, etc. specified by the instruction sequence of the program. In addition, the CPU 2212 can search for information in files, databases, etc. within the recording medium. For example, in a case where a plurality of entries are stored in the recording medium, and these entries have attribute values of a first attribute respectively associated with the attribute values of a second attribute, the CPU 2212 can search for an entry that matches the specified condition of the attribute value of the first attribute from these plurality of entries, and read the attribute value of the second attribute stored in the entry, thereby obtaining the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0085] The programs or software modules described above can be stored in the computer 2200 or in a computer-readable medium near the computer 2200. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, and the program can be provided to the computer 2200 via the network.

[0086] The present invention has been described using the embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. Those of ordinary skill in the art in the technical field to which the present case belongs can clearly understand that various changes or improvements can be made to the above embodiments. It can be clearly understood from the claims that the ways in which such changes or improvements have been made are also included in the technical scope of the present invention.

[0087] It should be noted that the execution order of each process such as actions, methods, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings can be implemented in any order as long as it is not specifically stated as "before...", "firstly...", etc., and the output of the previous process is not used in the subsequent process. Regarding the action flow in the claims, the specification, and the drawings, even if it is described for convenience using "firstly", "next", etc., it does not necessarily mean that it must be implemented in that order.

[0088] Description of Reference Numerals

[0089] 100: Inspection Device

[0090] 110: Object Image Acquisition Unit

[0091] 120: Object Image Masking Unit

[0092] 130: Occlusion Area Prediction Unit

[0093] 140: Reproduced Image Generation Unit

[0094] 150: Difference Detection Unit

[0095] 160: Determination Unit

[0096] 600: Learning Device

[0097] 610: Training Image Acquisition Unit

[0098] 620: Training Image Occlusion Unit

[0099] 630: Prediction Model

[0100] 640: Error Calculation Unit

[0101] 650: Model Update Unit

[0102] 2200: Computer

[0103] 2201: DVD-ROM

[0104] 2210: Host Controller

[0105] 2212: CPU

[0106] 2214: RAM

[0107] 2216: Image Controller

[0108] 2218: Display Device

[0109] 2220: Input / Output Controller

[0110] 2222: Communication Interface

[0111] 2224: Hard Disk Drive

[0112] 2226: DVD-ROM Drive

[0113] 2230: ROM

[0114] 2240: Input / Output Chip

[0115] 2242: Keyboard

Claims

1. An inspection device, comprising: An object image acquisition unit that acquires an object image obtained by photographing an inspection object; An object image masking unit that masks a part of the object image; A masked area prediction unit that predicts an image of a masked area masked in the object image; A reproduced image generation unit that generates a reproduced image using a plurality of predicted images predicted for each of a plurality of masked areas; and A difference detection unit that detects a difference between the object image and the reproduced image.

2. The inspection device according to claim 1, wherein, The difference detection unit compares the object image and the reproduced image for each predetermined area and calculates the degree of the difference for each predetermined area.

3. The inspection device according to claim 2, wherein, Further comprising: A determination unit that determines that the inspection object is a defective product when the degree of the difference does not meet a predetermined quality.

4. The inspection device according to claim 3, wherein, The determination unit determines that the inspection object is a defective product when the maximum degree of the differences among the degrees of the differences for each predetermined area exceeds a predetermined threshold.

5. The inspection device according to claim 3, wherein When the inspection object has been determined to be a defective product, the determination unit predicts the electrical characteristics of the inspection object using the object image obtained by photographing the inspection object determined to be a defective product, and determines that the inspection object is a defective product when the electrical characteristics do not meet a predetermined quality.

6. The inspection device according to claim 3, wherein, When the inspection object has been determined to be a defective product, the determination unit predicts the electrical characteristics of the inspection object using the object image obtained by photographing the inspection object determined to be a defective product, and determines that the inspection object is a non-defective product when the electrical characteristics meet a predetermined quality.

7. The inspection device according to claim 2, wherein The difference detection unit outputs a detection map, and the display state of each predetermined area is different according to the degree of the difference in the detection map.

8. The inspection device according to claim 2, wherein, The difference detection unit calculates the degree of the difference based on the Euclidean distance between the object image and the reproduced image.

9. The inspection device according to any one of claims 1 to 8, wherein, The object image masking unit sequentially masks one cell among a plurality of cells obtained by dividing the object image; The reproduced image generation unit generates the reproduced image using a plurality of predicted images predicted for different cells.

10. The inspection device according to any one of claims 1 to 8, wherein, The object image acquisition unit acquires an image obtained by gray-scaling an image obtained by photographing the inspection object as the object image.

11. The inspection device according to any one of claims 1 to 8, wherein, The object image acquisition unit acquires an image in which the object area is narrowed by object detection in the image obtained by photographing the inspection object as the object image.

12. An inspection method, comprising the following steps: Acquiring an object image obtained by photographing an inspection object; Masking a part of the object image; Predicting an image of a masked area masked in the object image; Generating a reproduced image using a plurality of predicted images predicted for each of a plurality of masked areas; and Detecting a difference between the object image and the reproduced image.

13. A computer program product including an inspection program, the inspection program being executed by a computer and causing the computer to function as the following means: An object image acquisition unit that acquires an object image obtained by photographing an examination object; An object image masking unit that masks a part of the object image; A masking area prediction unit that predicts an image of a masked area in the object image; A reproduced image generation unit that generates a reproduced image using a plurality of predicted images predicted for each of a plurality of masked areas; and, A difference detection unit that detects a difference between the object image and the reproduced image.

14. A learning device comprising: A training image acquisition unit that acquires a training image; A training image masking unit that masks a part of the training image; A prediction model that is input with the masked training image and outputs a model image predicted for the training image; and, A model update unit that updates the prediction model based on an error between the training image and the model image.

15. A learning method comprising the following steps: Acquire a training image; Mask a part of the training image; Input the masked training image into a prediction model and output a model image predicted for the training image; and, Update the prediction model based on an error between the training image and the model image.

16. A computer program product including a learning program, the learning program being executed by a computer and causing the computer to function as the following means: A training image acquisition unit that acquires a training image; A training image masking unit that masks a part of the training image; A prediction model that is input with the masked training image and outputs a model image predicted for the training image; and, A model update unit that updates the prediction model based on an error between the training image and the model image.

Images