Inspection device, inspection method, manufacturing method, and inspection program

By generating low-resolution images and using reconstruction models to identify high-defect likelihood areas, the method enhances inspection efficiency and accuracy by focusing inspections on relevant regions.

JP2024159256AActive Publication Date: 2024-11-08NIPPON ELECTRIC GLASS CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2023075124
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08
Estimated Expiration
2043-04-28

AI Technical Summary

Technical Problem

Conventional quality inspection methods apply a trained neural network uniformly to both high and low defect likelihood areas without distinction, leading to inefficiencies and potential over-detection of defects.

Method used

The method involves generating a low-resolution image of the inspection target, using a reconstruction model to generate a reconstructed image, and determining a target area based on pixel reconstruction errors to perform inspections efficiently.

Benefits of technology

This approach improves inspection efficiency and accuracy by focusing inspections on high-defect likelihood areas, reducing unnecessary defect detection and maintaining high accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024159256000001_ABST
    Figure 2024159256000001_ABST
Patent Text Reader

Abstract

To improve inspection efficiency in a quality inspection using an image obtained by imaging an inspection target.SOLUTION: An inspection device (1) comprises: a reconstruction unit (103) which generates a reconstructed image by using a reconstruction model (111) to reconstruct a low-resolution image obtained by reducing the resolution of an inspection target image; and an inspection unit (106) which inspects the inspection target by an inspection model (112) targeting an inspection target area determined on the basis of a reconstruction error of each pixel in the reconstructed image.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an inspection device or the like that performs quality inspection using an image obtained by photographing an object to be inspected. [Background technology]

[0002] In the inspection of the quality of an object to be inspected, a technique for judging the quality of the object by using an image of the object has been conventionally known. For example, the following Patent Document 1 discloses a printed circuit board inspection device that judges the quality of the printed circuit board by applying a trained neural network to specific area data extracted from data obtained by scanning the printed circuit board with an image sensor. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-100753 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional technology described above, a trained neural network is uniformly applied to make a pass / fail judgment without making any particular distinction between specific area data in which the possibility of a defect is high and specific area data in which the possibility of a defect is low. In this respect, there is room for improvement in the conventional technology.

[0005] An object of one aspect of the present invention is to provide an inspection device or the like that can improve the efficiency of quality inspection using an image of an object to be inspected. [Means for solving the problem]

[0006] In order to solve the above problems, an inspection device according to one embodiment of the present invention includes a reconstruction unit that generates a reconstructed image by reconstructing a low-resolution image of an inspection object obtained by photographing the inspection object using a reconstruction model generated by learning the characteristics of an image of a good inspection object, a target area determination unit that determines an inspection target area in the inspection object image based on the reconstruction error of each pixel in the reconstructed image, and an inspection unit that inspects the inspection object using the trained inspection model for the inspection target area.

[0007] In order to solve the above problems, an inspection method according to one embodiment of the present invention is an inspection method executed by one or more information processing devices, and includes a reconstruction step of generating a reconstructed image by reconstructing a low-resolution image of an inspection object obtained by photographing the inspection object using a reconstruction model generated by learning the characteristics of an image of a good inspection object, a target area determination step of determining an inspection target area in the inspection object image based on the reconstruction error of each pixel in the reconstructed image, and an inspection step of inspecting the inspection object using the trained inspection model with the inspection target area as the target.

[0008] In order to solve the above problems, a manufacturing method according to one aspect of the present invention is a method for manufacturing a substrate, and includes a reconstruction step of generating a reconstructed image by reconstructing a low-resolution image of an inspection target image obtained by photographing the manufactured substrate using a reconstruction model generated by learning the characteristics of an image of a good substrate, a target area determination step of determining an inspection target area in the inspection target image based on the reconstruction error of each pixel in the reconstructed image, and an inspection step of inspecting the substrate using the trained inspection model with the inspection target area as the target.

[0009] In order to solve the above problems, an inspection program according to one embodiment of the present invention causes a computer to perform the following processes: a process of generating a reconstructed image by reducing the resolution of an inspection object image obtained by photographing the inspection object, using a reconstruction model generated by learning the characteristics of an image of a good inspection object; a process of determining an inspection object area in the inspection object image based on the reconstruction error of each pixel in the reconstructed image; and a process of inspecting the inspection object using the trained inspection model for the inspection object area. Effect of the Invention

[0010] According to one aspect of the present invention, it is possible to improve the efficiency of inspection in pass / fail inspection using an image of an inspection object. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing an example of a configuration of a main part of an inspection device according to an embodiment of the present invention; [Diagram 2] 1A and 1B are diagrams showing an example of a substrate inspection performed by the inspection device. [Diagram 3] 1 is a flowchart showing a flow of an inspection method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] [Configuration of the inspection device] The configuration of an inspection device 1 according to one embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the main parts of the inspection device 1. The inspection device 1 is a device equipped with a function for inspecting the quality of an inspection object by using an image obtained by photographing the inspection object.

[0013] As shown in the figure, the inspection device 1 includes a control unit 10 that controls each unit of the inspection device 1, and a storage unit 11 that stores various data used by the inspection device 1. The inspection device 1 also includes a communication unit 12 that enables the inspection device 1 to communicate with other devices, an input unit 13 that accepts input of various data to the inspection device 1, and an output unit 14 that enables the inspection device 1 to output various data.

[0014] The control unit 10 also includes an image acquisition unit 101, a low-resolution image generation unit 102, a reconstruction unit 103, a heat map generation unit 104, a target region determination unit 105, an inspection unit 106, and a model generation unit 107. The memory unit 11 stores a reconstruction model 111 and an inspection model 112.

[0015] The image acquisition unit 101 acquires an inspection target image obtained by photographing a part or the whole of an inspection target. The method of acquiring the inspection target image is not particularly limited. For example, the image acquisition unit 101 may acquire an inspection target image inputted via the input unit 13, or may acquire an inspection target image stored in advance in a storage device such as the storage unit 11. The image acquisition unit 101 may also acquire an inspection target image from another device (for example, an imaging device that captures the inspection target image) via the communication unit 12. A imaging device such as a camera may also be incorporated in the inspection device 1, and in this case, the image acquisition unit 101 may directly acquire an image captured by the imaging device incorporated in the inspection device 1.

[0016] The low-resolution image generating unit 102 generates a low-resolution image by lowering the resolution of the inspection target image acquired by the image acquiring unit 101. The lower the resolution, the faster the generation of a reconstructed image by the reconstruction unit 103, which will be described below. For this reason, it is preferable that the low-resolution image generating unit 102 generates a low-resolution image by lowering the resolution of the inspection target image as much as possible within a range that does not interfere with the identification of the inspection target area in the inspection target image (i.e., an area that is likely to include a defective part).

[0017] The reconstruction unit 103 reconstructs the low-resolution image generated by the low-resolution image generation unit 102 using the reconstruction model 111 to generate a reconstructed image. Specifically, the reconstruction unit 103 inputs the low-resolution image to the reconstruction model 111, which then outputs a reconstructed image.

[0018] The reconstruction model 111 is a generative AI (Artificial Intelligence) that is generated by learning the features of an image of a good inspection object, and that takes an image as an input and outputs an image. The reconstruction model 111 can reconstruct an image of a good inspection object with high accuracy, but the reconstruction accuracy of an image including unlearned features (e.g. features due to a defective part) is low. For this reason, it can be said that an area with a large reconstruction error in the reconstructed image is an area that is likely to be a defective part.

[0019] The heat map generating unit 104 generates a heat map indicating the reconstruction error of each pixel in the reconstructed image. The reconstruction error is the difference between the pixel value of a pixel in the low-resolution image before reconstruction and the pixel value of a pixel corresponding to the pixel in the reconstructed image. Therefore, the heat map generating unit 104 can generate the heat map by calculating the difference between the low-resolution image before reconstruction and the reconstructed image on a pixel-by-pixel basis.

[0020] The target region determining unit 105 determines a region in the inspection target image that is likely to be a defective part as the inspection target region. Specifically, the target region determining unit 105 determines the inspection target region in the inspection target image based on the reconstruction error of each pixel in the reconstructed image. More specifically, the target region determining unit 105 first binarizes the heat map with a predetermined threshold value, and detects a region consisting of pixels whose pixel values ​​are equal to or greater than the threshold value. As described above, since the pixel values ​​of the heat map indicate the reconstruction error, it is possible to identify a region consisting of pixels whose reconstruction error in the reconstructed image is equal to or greater than the predetermined threshold value by detecting a region consisting of pixels whose pixel values ​​are equal to or greater than the threshold value in the heat map.

[0021] Note that the "region consisting of pixels whose reconstruction error is equal to or greater than the threshold" here means a region that mainly includes pixels whose reconstruction error is equal to or greater than the threshold, and does not necessarily mean that the reconstruction error of all pixels included in the region is equal to or greater than the threshold. For example, the target region determination unit 105 may detect a continuous region consisting of pixels whose pixel values ​​are equal to or greater than the threshold in the heat map, and determine a rectangular region surrounding the continuous region as the "region consisting of pixels whose reconstruction error is equal to or greater than the threshold."

[0022] Then, the target region determining unit 105 determines the region in the inspection target image at a position corresponding to the detected "region consisting of pixels whose reconstruction error is equal to or greater than a threshold" as the inspection target region. The inspection target region is set not on the low-resolution image but on the inspection target image before it is reduced in resolution, thereby enabling a high-precision pass / fail judgment. Here, the size of the inspection target region may correspond to the size of the "region consisting of pixels whose reconstruction error is equal to or greater than a threshold" or may be fixed regardless of the size of the "region consisting of pixels whose reconstruction error is equal to or greater than a threshold".

[0023] The inspection unit 106 inspects the inspection object using the inspection model 112, with the inspection object area determined by the object area determination unit 105 as the target. Here, the inspection of the inspection object may be to determine whether the inspection object is a non-defective product or not, or may be to determine the type of defect in the inspection object.

[0024] Inspection model 112 is a trained model for inspecting an object to be inspected. Inspection model 112 may be a model for classifying objects to be inspected into good products and defective products, or a model for classifying defective products by type. Inspection model 112 may be a combination of a model for classifying objects to be inspected into good products and defective products and a model for classifying defective products by type.

[0025] The inspection model 112 for pass / fail judgment can be generated by machine learning using data (e.g., an image of the inspection object) that represents the characteristics of the inspection object, the presence or absence of a defect of which is known, as training data. For example, an area of ​​a predetermined size that shows a defective part may be cut out from an image of the inspection object that is a defective product and used as training data (hereinafter, referred to as a training image). In addition, an area of ​​the predetermined size cut out from an image of an inspection object that is a non-defective product may also be included in the training image.

[0026] Furthermore, the inspection model 112 for classifying the type of defect can be generated by machine learning using data (e.g., an image of the inspection object) representing the characteristics of an inspection object whose type of defect is known as training data. The algorithm of the machine learning is not particularly limited. For example, the inspection model 112 may be a convolutional neural network model with high image classification accuracy.

[0027] In the following, an example will be described in which an inspection model 112 is used that classifies an inspection object into either a good product or a plurality of types of defective products by inputting an image of the inspection object. Such an inspection model 112 can be generated by using images of good inspection objects and images of each type of defective inspection objects as teacher images and learning the relationship between the teacher images and the pass / fail status and types of defects of the inspection objects.

[0028] The model generation unit 107 generates a reconstructed model 111 by learning using an image of the inspection object which is a non-defective product. The image used for learning (hereinafter, referred to as a teacher image) is an image obtained by photographing the inspection object which is a non-defective product.

[0029] Specifically, the model generation unit 107 reconstructs a teacher image using the reconstruction model under learning to generate a reconstructed image, calculates a reconstruction error of the generated reconstructed image, and updates the reconstruction model so as to reduce the reconstruction error, repeating this process while changing the teacher image. In this way, a reconstruction model 111 capable of generating a reconstructed image that reproduces the features of the teacher image, i.e., the features of an image of a non-defective inspection object, is generated.

[0030] In the inspection device 1, the model generation unit 107 is not an essential component, but providing the model generation unit 107 has the advantage that the generation or update of the reconstruction model 111 can also be performed by a single inspection device 1. When the model generation unit 107 is not provided in the inspection device 1, the reconstruction unit 103 may generate a reconstructed image using the reconstruction model 111 generated by another device.

[0031] As described above, the inspection device 1 includes a reconstruction unit 103 that generates a reconstructed image by reconstructing a low-resolution image, which is an inspection object image obtained by photographing the inspection object, using a reconstruction model 111 generated by learning the image features of a good inspection object, a target area determination unit 105 that determines an inspection object area in the inspection object image based on the reconstruction error of each pixel in the reconstructed image, and an inspection unit 106 that inspects the inspection object using the trained inspection model 112 for the determined inspection object area.

[0032] The reconstruction model 111 generated by learning the features of an image of a good inspection object can reconstruct an image of a good inspection object (i.e., one without defects) with high accuracy, but the reconstruction accuracy of an image of an inspection object that should be treated as a defective product is low. Therefore, in the reconstructed image generated using the reconstruction model 111, the reconstruction error of pixels that are unlikely to have a defect is small, and the reconstruction error of pixels that are likely to have a defect is large.

[0033] Therefore, according to the above-mentioned configuration for determining the inspection target area in the inspection target image based on the reconstruction error of each pixel in the generated reconstructed image, an area where there is a high possibility of a defect being present is set as the inspection target area, and an efficient inspection can be performed by narrowing down the inspection target to the inspection target area. Furthermore, since the inspection using the inspection model 112 is performed not on the low-resolution image but on the inspection target area in the inspection target image that is the basis of the low-resolution image, the inspection accuracy is not reduced.

[0034] Furthermore, the lower the resolution of the image, the less calculation is required to generate the reconstructed image. Therefore, according to the above configuration in which a reconstructed image is generated not from the inspection target image itself but from a low-resolution image obtained by lowering the resolution of the inspection target image, the decrease in inspection efficiency caused by the process of generating the reconstructed image is suppressed. Therefore, according to the above configuration, it is possible to improve the inspection efficiency in pass / fail inspection using an image of an inspection target.

[0035] Furthermore, with the above configuration, minor defects that should be determined as non-defective in the inspection target image are less noticeable in the low-resolution image, which leads to suppression of overdetection of defects.

[0036] Furthermore, even if the shape of the inspection object changes due to a change in specifications, etc., the appearance of the defective portion does not basically change. Therefore, according to the above configuration, when the shape of the inspection object changes due to a change in specifications, it is only necessary to re-learn the reconstruction model 111 with the new specifications, and there is also the advantage that the inspection model 112 can be used as is.

[0037] The object to be inspected may be any object that allows detection of defective parts from a captured image. For example, the object to be inspected may be a component or semi-finished product such as an electrode or substrate used in an industrial product, or a finished product. Depending on the size or shape of the object to be inspected, it may be difficult to capture the entire object in a single image, or to capture the object with a resolution sufficient for pass / fail judgment. For this reason, each image of each part of the object to be inspected may be used as an image to be inspected, and the pass / fail judgment results for each image to be inspected may be combined to judge the pass / fail of the object to be inspected.

[0038] [Examples of tests] A specific example of inspection by the inspection device 1 will be described with reference to FIG. 2. FIG. 2 is a diagram showing an example of inspecting a board by the inspection device 1. More specifically, in the example of FIG. 2, the image acquisition unit 101 acquires an image A1 of six workpieces on the main board. This board may be, for example, a functional board in which a functional film, grooves, or depressions are formed on a glass, metal, resin, or ceramic base material. This board may also be a circuit board such as a ceramic circuit board in which a circuit is formed on a ceramic base material with a conductor such as metal.

[0039] Next, the image acquisition unit 101 cuts out a portion of the area A11 in which one workpiece is captured in the image A1 to acquire an inspection target image A2. In this manner, by acquiring the inspection target image A2 corresponding to one workpiece, it is possible to determine the quality of the board for each workpiece. Note that the extent to which the inspection target is included in one inspection target image may be appropriately determined depending on the type of item the inspection target is, the resolution of the imaging device that captures the item, and the like.

[0040] Next, the low-resolution image generating unit 102 reduces the resolution of the inspection target image A2 to generate a low-resolution image A3. For example, if the inspection target image A2 is an image of a size of 1200×800 pixels, the low-resolution image generating unit 102 may reduce (or reduce) the resolution of the image to a size of 120×80 pixels to generate the low-resolution image A3.

[0041] Moreover, it is preferable that the size of the low-resolution image A3 is a size that can be input to the reconstruction model 111, that is, the same size as the teacher image used for learning the reconstruction model 111. This eliminates the need to adjust the size of the low-resolution image A3 during reconstruction. Note that there is no problem even if the ratio of the length and width of the low-resolution image A3 and the inspection target image A2 differ slightly.

[0042] Next, the reconstruction unit 103 reconstructs the low-resolution image A3 using the reconstruction model 111 to generate a reconstructed image A4. In the reconstructed image A4, it is difficult to reproduce features of the low-resolution image A3 that are not included in the teacher image. Therefore, if a defective portion is captured in the low-resolution image A3, the area in which the defective portion is captured is difficult to be reflected in the reconstructed image A4.

[0043] Next, the heat map generating unit 104 generates a heat map A5 indicating the reconstruction error of each pixel in the reconstructed image A4. Specifically, the heat map generating unit 104 generates the heat map A5 by calculating the difference in pixel values ​​between pixels in each pair of pixels at corresponding positions in the low-resolution image A3 and the reconstructed image A4.

[0044] In the heat map A5, pixel values ​​in areas with large reconstruction errors in the reconstructed image A4 become large. In the example of FIG. 2, there is an area with large pixel values ​​to the right of the center of the heat map A5, and the target area determination unit 105 detects a rectangular area A51 surrounding this area. Area A51 corresponds to the above-mentioned "area consisting of pixels whose reconstruction error is equal to or greater than a threshold value." Area A51 is the "area consisting of pixels whose reconstruction error is equal to or greater than a threshold value" in the heat map A5, and can also be said to indicate the "area consisting of pixels whose reconstruction error is equal to or greater than a threshold value" in the reconstructed image A4.

[0045] Next, the target region determining unit 105 determines the inspection target region in the inspection target image A2 based on the above-mentioned reconstruction error. For example, the target region determining unit 105 may determine the inspection target region A21 in the inspection target image A2, which has a position and size corresponding to the region A51. In this case, for example, the target region determining unit 105 may convert the coordinate values ​​of the four corners of the region A51 into coordinate values ​​on the inspection target image A2 according to the magnification when generating the low-resolution image A3, and may determine the rectangular region defined by the converted four coordinates as the region A21.

[0046] Then, the inspection unit 106 inspects the inspection object using the inspection model 112 for the area A21 determined as described above. For example, the inspection unit 106 may inspect the inspection object by inputting an image A6 obtained by cutting out the area A21 from the inspection object image A2 to the inspection model 112. Note that before inputting the image A6 to the inspection model 112, the size of the image A6 may be adjusted so that the image to be input to the inspection model 112 is the same size as the teacher image used for learning the inspection model 112.

[0047] The larger the size (or resolution) of the image input to inspection model 112, the higher the inspection accuracy can be, while reducing the size of the image input to inspection model 112 can speed up inspection using inspection model 112. For this reason, it is preferable to determine the size of the teacher image used to generate inspection model 112, taking into consideration both the accuracy and speed required for inspection.

[0048] It should be noted that if the size of the "region consisting of pixels whose reconstruction error is equal to or greater than the threshold" in the reconstructed image A4 (equal to the size of region A51) is small, the object to be inspected is likely to be a non-defective product. For this reason, the inspection unit 106 may determine the inspection result of the object to be inspected as a non-defective product if the size of the "region consisting of pixels whose reconstruction error is equal to or greater than the threshold" is less than a predetermined lower limit. This makes it possible to obtain valid inspection results while omitting inspection using the inspection model 112 and increasing inspection efficiency.

[0049] On the other hand, if the size of the "region consisting of pixels whose reconstruction error is equal to or greater than the threshold" is large, the object to be inspected is likely to be defective. For this reason, the inspection unit 106 may determine the inspection result of the object to be inspected as defective if the size of the "region consisting of pixels whose reconstruction error is equal to or greater than the threshold" (equal to the size of the region A51) exceeds a predetermined upper limit value. This makes it possible to obtain a valid inspection result while improving inspection efficiency by omitting the inspection using the inspection model 112. Note that if the inspection model 112 is a classification model that determines the type of defect, the inspection using the inspection model 112 may not be omitted, and the type of defect may be determined using the inspection model 112.

[0050] Furthermore, when the size of the "region consisting of pixels with a reconstruction error equal to or greater than a threshold" (equal to the size of region A51) is equal to or greater than a predetermined lower limit and equal to or less than a predetermined upper limit, the target region determining unit 105 may determine, as the region to be inspected, a region of a predetermined size at a position corresponding to the region in the inspection target image A2. This makes it possible to prevent a decrease in inspection accuracy caused by adjusting the size of the image to be input to the inspection model 112 during inspection using the inspection model 112. Note that this configuration does not prevent the size of the image to be input to the inspection model 112 from being adjusted within a range that does not impair the inspection accuracy.

[0051] [Testing method flow] The inspection method according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flow chart showing the flow of the inspection method according to this embodiment.

[0052] In S1, the image acquiring unit 101 acquires an inspection target image. The image acquiring unit 101 may acquire an image that captures a wider area than the inspection target range in one inspection, such as image A1 in Fig. 2, and acquire the inspection target image by cutting out the inspection target image from the image, or may acquire an inspection target image that captures only the inspection target range in one inspection.

[0053] In S2, the low-resolution image generating unit 102 reduces the resolution of the inspection image acquired in S1 to generate a low-resolution image. The extent to which the resolution is reduced may be determined in advance. In addition, it is preferable to generate a low-resolution image of a size that can be input directly to the reconstruction model 111.

[0054] In S3 (reconstruction step), the reconstruction unit 103 reconstructs the low-resolution image generated in S2 using the reconstruction model 111 to generate a reconstructed image.

[0055] In S4, the heat map generating unit 104 uses the low-resolution image generated in S2 and the reconstructed image generated in S3 to generate a heat map indicating reconstruction errors in the reconstructed image.

[0056] In S5, the target region determining unit 105 judges whether or not to skip the inspection by the inspection model 112. Specifically, the target region determining unit 105 binarizes the heat map generated in S4 with a predetermined threshold value, and detects the above-mentioned "region consisting of pixels whose reconstruction error is equal to or greater than the threshold value." Then, the target region determining unit 105 judges whether the size of the detected region (1) exceeds a predetermined upper limit value, (2) is less than a predetermined lower limit value, or (3) is equal to or greater than a predetermined lower limit value and equal to or less than a predetermined upper limit value. If the judgment result is (1) or (2), the judgment result of S5 is YES, and if the judgment result is (3), the judgment result of S5 is YES. If the judgment result of S5 is YES, the process proceeds to S8, and if the judgment result of S5 is NO, the process proceeds to S6. Note that even if the judgment result of S5 is (1), the judgment result of S5 may be set to NO (i.e., the inspection by the inspection model 112 may not be skipped).

[0057] In S6 (target region determining step), the target region determining unit 105 determines an inspection target region in the inspection target image based on the reconstruction error of each pixel in the reconstructed image shown in the heat map generated in S4. Specifically, the target region determining unit 105 sets an inspection target region in the inspection target image acquired in S1 at a position corresponding to the "region consisting of pixels whose reconstruction error is equal to or greater than a threshold" detected in S5. The size of the inspection target region to be set may be a predetermined size regardless of the size of the "region consisting of pixels whose reconstruction error is equal to or greater than a threshold", or may be a size corresponding to the size of the "region consisting of pixels whose reconstruction error is equal to or greater than a threshold".

[0058] In S7 (inspection step), inspection unit 106 inspects the inspection target object using inspection model 112 for the inspection target area determined in S6. Specifically, inspection unit 106 cuts out a portion of the inspection target area in the inspection target image, and inputs the cut out portion to inspection model 112. As a result, inspection model 112 outputs a pass / fail judgment result for the inspection target object (for example, whether the object is good or defective, and if it is defective, the type of defect).

[0059] In S8, the inspection unit 106 outputs the inspection result, and the inspection method of FIG. 3 ends. When the process moves from S5 to S8, if the size of the detected area (1) exceeds a predetermined upper limit, the inspection result is "defective," and if the size of the detected area (2) is less than a predetermined lower limit, the inspection result is "good." On the other hand, when the process moves from S7 to S8, the inspection result according to the output value of the inspection model 112 is output. Note that the process of S8 may be performed only when the inspection result is defective. When inspecting multiple workpieces as in the example of FIG. 2, the processes of S1 to S8 are performed for each workpiece.

[0060] As described above, the inspection method according to this embodiment includes a reconstruction step (S3) of generating a reconstructed image by reconstructing a low-resolution image obtained by photographing the inspection object using a reconstruction model 111 generated by learning the features of an image of a good inspection object, an object region determination step (S6) of determining an inspection object region in the inspection object image based on the reconstruction error of each pixel in the reconstructed image, and an inspection step (S7) of inspecting the inspection object using the trained inspection model 112 for the inspection object region. Thus, it becomes possible to improve the inspection efficiency in quality inspection using an image of the inspection object.

[0061] [Substrate manufacturing method] The above-mentioned inspection method may be implemented as a part of the process of manufacturing a substrate. That is, the manufacturing method of a substrate according to this embodiment includes a reconstruction step of generating a reconstructed image by reconstructing a low-resolution image obtained by photographing a manufactured substrate using a reconstruction model 111 generated by learning the characteristics of an image of a non-defective substrate, a target area determination step of determining an inspection target area in the inspection target image based on the reconstruction error of each pixel in the reconstructed image, and an inspection step of inspecting the substrate using the inspection model 112 that has been trained on the inspection target area. According to this manufacturing method, it is possible to improve the inspection efficiency of the quality inspection using the image of the substrate photographed in the substrate manufacturing process, and to efficiently manufacture the substrate.

[0062] [Modifications] The executing entity of each process described in the above embodiment is arbitrary and is not limited to the above example. That is, an inspection system having the same function as the inspection device 1 can be constructed by a plurality of information processing devices capable of communicating with each other. For example, an inspection system having the same function as the inspection device 1 can be constructed by distributing each block shown in FIG. 1 among a plurality of inspection devices. Also, each step shown in FIG. 3 may be shared and executed by a plurality of information processing devices included in the above inspection system. That is, the executing entity of each process of each step included in the inspection method and manufacturing method described in the above embodiment may be one information processing device (for example, the inspection device 1) or may be a plurality of information processing devices.

[0063] [Software implementation example] The functions of the inspection device 1 can be realized by a program (inspection program) that causes a computer to function as the device and causes a computer to function as each control block (particularly each unit included in the control unit 10) of the inspection device 1. This inspection program causes the computer to execute the following processes: a process of generating a reconstructed image by reconstructing a low-resolution image obtained by photographing an inspection object by reducing the resolution of the inspection object image using a reconstruction model generated by learning the characteristics of an image of a good inspection object; a process of determining an inspection object region in the inspection object image based on the reconstruction error of each pixel in the generated reconstructed image; and a process of inspecting the inspection object using the learned inspection model for the determined inspection object region.

[0064] In this case, the inspection device 1 includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the inspection program. The control device and storage device execute the program to realize each function described in the above embodiment.

[0065] The inspection program may be recorded in one or more non-transient computer-readable recording media. The recording media may or may not be included in the inspection device 1. In the latter case, the inspection program may be supplied to the inspection device 1 via any wired or wireless transmission medium.

[0066] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of each of the control blocks can be realized by, for example, a quantum computer.

[0067] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0068] 1. Inspection equipment 103 Reconstruction part 105 Target Area Determination Unit 106 Inspection Department 107 Model Generation Unit 111 Reconstruction Model 112 Inspection Model

Claims

1. a reconstruction unit that generates a reconstructed image by reconstructing a low-resolution image obtained by photographing an inspection object using a reconstruction model generated by learning features of an image of a non-defective inspection object; an object region determining unit that determines an inspection object region in the inspection object image based on a reconstruction error of each pixel in the reconstructed image; An inspection device comprising: an inspection unit that inspects the object to be inspected using a trained inspection model targeting the inspection target area.

2. The inspection apparatus according to claim 1 , further comprising a model generation unit configured to generate the reconstruction model by learning using an image of the inspection object that is a non-defective product.

3. 3. The inspection device according to claim 1, wherein the inspection unit determines an inspection result of the inspection object to be a pass if a size of an area in the reconstructed image consisting of pixels having a reconstruction error equal to or greater than a predetermined threshold is less than a predetermined lower limit.

4. 3. The inspection device according to claim 1, wherein the inspection unit determines an inspection result of the inspection object to be defective if a size of an area in the reconstructed image consisting of pixels having a reconstruction error equal to or greater than a predetermined threshold exceeds a predetermined upper limit value.

5. 3. The inspection device according to claim 1, wherein the target area determination unit determines, when a size of a region in the reconstructed image consisting of pixels having a reconstruction error equal to or greater than a predetermined threshold is equal to or greater than a predetermined lower limit and equal to or less than a predetermined upper limit, as the inspection area, a region of a predetermined size at a position corresponding to the region in the inspection object image.

6. 1. A testing method executed by one or more information processing devices, comprising: a reconstruction step of generating a reconstructed image by reconstructing a low-resolution image obtained by photographing an object to be inspected using a reconstruction model generated by learning the characteristics of an image of a non-defective object to be inspected; an object region determining step of determining an inspection object region in the inspection object image based on a reconstruction error of each pixel in the reconstructed image; and an inspection step of inspecting the object to be inspected using a trained inspection model targeting the area to be inspected.

7. A manufacturing method for manufacturing a substrate, comprising the steps of: a reconstruction step of generating a reconstructed image by reconstructing a low-resolution image obtained by photographing the manufactured substrate and reducing the resolution of the inspection target image obtained by photographing the manufactured substrate using a reconstruction model generated by learning the characteristics of images of a non-defective substrate; an object region determining step of determining an inspection object region in the inspection object image based on a reconstruction error of each pixel in the reconstructed image; and an inspection step of inspecting the substrate using a trained inspection model targeting the inspection target area.

8. A process of generating a reconstructed image by reconstructing a low-resolution image of an inspection object obtained by photographing the inspection object using a reconstruction model generated by learning the characteristics of an image of a non-defective inspection object; determining an inspection target area in the inspection target image based on a reconstruction error of each pixel in the reconstructed image; and a process of inspecting the object to be inspected using a trained inspection model for the area to be inspected.

Citation Information

Patent Citations

  • Super-resolution reconstruction and feature extraction method for defect detection

    CN115170483A

  • Image inspection device

    JP2021086208A

  • Substrate foreign matter inspection device and substrate foreign matter inspection method

    JP2022088818A

  • Deep learning based defect detection

    US20220044391A1

  • Inspection device and inspection method

    WO2018105028A1