Image inspection device and image processing method

Abstract

Provided are an image inspection device and an image processing method which are robust with respect to variations in image quality and with which it is possible to detect a defect or another abnormality with high accuracy. Provided is an image inspection device for comparing a first image with a second image and acquiring a difference between the images to inspect an abnormality that could be present in either the first image or the second image. The image inspection device comprises: an image quality feature amount extraction unit that extracts an image quality feature amount, which is a feature amount vector representing image quality for each partial region of the first image; an image quality correction unit that brings the image qualities of the first image and / or the second image relatively closer for each partial region on the basis of the feature amount vector obtained by the image quality feature amount extraction unit and the second image; and a comparison processing unit that compares any one of the images corrected by the image quality correction unit and the other image, or compares the corrected first and second images, to acquire a difference between the images. The image quality feature amount extraction unit includes a region division processing unit that divides the first image into partial regions in units larger than the size of a predetermined abnormality, and a feature amount extraction processing unit that extracts a feature amount vector representing the image quality of each partial region from the other partial regions of the first image.

Description

Image inspection device and image processing method

[0001] The present invention relates to an image processing technique for processing image data, and more particularly to an image inspection apparatus and an image processing method suitable for inspection using image data.

[0002] For evaluation of semiconductor circuit defects, a method is used in which an inspection image is compared with a reference image. The reference image can be the design data of the sample to be evaluated, an image obtained by simulating the captured image using machine learning or other methods based on the design data, image statistics that represent the pixel-by-pixel distribution of pixel values ​​that can be taken in an image of a sample manufactured from the same design data, or a different region of an image of the same pattern as the sample.

[0003] In defect inspection, the captured images generated by the device may have variations in image quality, such as image distortion or misalignment caused by light source adjustment, sensor imaging, or sample-related factors such as the sample's pattern or characteristics, as well as image blur due to uneven brightness or defocus. For example, in the inspection of semiconductor circuits using a scanning electron microscope (SEM), image quality varies due to factors such as variations in electron beam control, differences between imaging devices, sample charging, and sample contamination. In such cases, differences that are not representative of defects or other abnormalities occur between the inspection image and the reference image, and an originally normal part is judged to be abnormal.

[0004] Therefore, for example, a technology described in Patent Document 1 has been proposed. Patent Document 1 discloses that parameters of a semiconductor manufacturing process and an imaging process are input to a machine learning model, and image statistics of a normal image generated from design data are varied. Furthermore, Patent Document 2 discloses that, for inspection robust to individual differences in images, a partial region of an inspection image is cut out, and from an image obtained by removing the center of the cut out region, a normal pattern of the removed center is generated by a machine learning model, and the normal pattern is compared with the removed center image for inspection.

[0005] International Publication No. 2021-255819 International Publication No. 2019-146538

[0006] Differences that occur between an inspection image and a reference image can be due to defects or other abnormalities, or they can be due to variations in image quality. In comparative inspections, it is desirable to obtain only differences due to defects or other abnormalities, and it is necessary to eliminate differences due to variations in image quality. However, image processing that simply converts images to reduce the differences between the reference image and the inspection image cannot distinguish between variations in image quality and differences due to defects or other abnormalities. Therefore, it is necessary to be able to correct only the differences in image quality and detect only abnormalities.

[0007] In Patent Literature 1, in an inspection in which image statistics of a normal image generated from design data are compared with an inspection image, parameters of the manufacturing process and the imaging process are included in the input of a machine learning model that generates the image statistics, thereby enabling inspection that takes into account variations in image quality caused by imaging. However, because the sample state, including the circuit pattern, is not taken into account, performance degradation may occur due to variations in image quality caused by the sample, whose occurrence position and appearance vary depending on the sample state.

[0008] In Patent Document 2, a normal pattern for a portion to be compared and inspected is generated using a machine learning model. However, if the portion to be compared and inspected cannot be uniquely determined with respect to the surrounding area, there is a risk of a decrease in the accuracy of generating the normal pattern. Therefore, in semiconductor inspection, which involves inspecting fine and complex patterns, it is desirable to inspect using a reference image that includes the normal pattern.

[0009] Therefore, the present invention provides an image inspection device and an image processing method that are robust against variations in image quality and can detect defects and other abnormalities with high accuracy.

[0010] In order to solve the above-mentioned problems, the image inspection device of the present invention is an image inspection device for comparing a first image and a second image, obtaining the image difference, and inspecting for abnormalities that may exist in either the first image or the second image, and is equipped with an image quality feature extraction unit that extracts image quality features, which are feature vectors that express the image quality for each partial region of the first image; an image quality correction unit that brings the image quality of the first image and / or the second image relatively closer for each partial region based on the feature vector obtained by the image quality feature extraction unit and the second image; and a comparison processing unit that compares one of the images corrected by the image quality correction unit with the other image, or the corrected first image and second image, to obtain the image difference, wherein the image quality feature extraction unit has an area division processing unit that divides the first image into partial regions in units larger than the size of a predetermined abnormality, and a feature extraction processing unit that extracts feature vectors that express the image quality of each partial region from other partial regions of the first image.

[0011] Furthermore, an image processing method according to the present invention is an image processing method for comparing a first image and a second image, obtaining image differences, and processing the images to inspect for abnormalities that may exist in either the first image or the second image, wherein an image quality feature extraction unit extracts image quality features, which are feature vectors expressing the image quality for each partial region of the first image; an image quality correction unit brings the image quality of the first image and / or the second image relatively closer for each partial region based on the feature vectors and the second image; a comparison processing unit compares one of the images corrected by the image quality correction unit with the other image, or the corrected first image and second image, to obtain image differences; an area division processing unit divides the first image into partial regions each larger than a predetermined size of an abnormality; and a feature extraction processing unit extracts feature vectors expressing the image quality of each partial region from other partial regions of the first image.

[0012] According to the present invention, it is possible to provide an image inspection device and an image processing method that are robust against variations in image quality and can detect abnormalities such as defects with high accuracy. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiments.

[0013] 3 is a diagram showing an example of a case where an image difference is calculated using two images having a difference due to variation in image quality. FIG. 4 is a diagram showing an example of a case where an image difference is calculated using two images without variation in image quality. FIG. 5 is a functional block diagram showing an example of the overall configuration of an image inspection device according to Example 1 of the present invention. FIG. 6 is a functional block diagram showing the configuration of an inference processing unit constituting the image inspection device shown in FIG. 2. FIG. 7 is a diagram showing an example of the processing of an image quality feature extraction unit constituting the inference processing unit shown in FIG. 3. FIG. 8 is a flowchart showing the processing operation of an inference processing unit constituting the image inspection device according to Example 1. FIG. 9 is a flowchart showing the processing operation of a learning processing unit constituting the image inspection device according to Example 1. FIG. 10 is a functional block diagram showing the configuration of an inference processing unit according to Example 2 of the present invention. FIG. 11 is a diagram showing an example of variation in image quality according to Example 2. FIG. 12 is a diagram showing an example of image quality correction amount according to Example 2. FIG. 13 is a diagram showing an example of correction processing of a probability distribution according to Example 3 of the present invention.

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In each drawing, the same components are designated by the same reference numerals, and detailed descriptions of overlapping parts will be omitted. The image inspection device exemplified in this specification relates to an image inspection device and an image processing method for reducing the difference caused by variations in image quality between a first image and a second image, thereby reducing false detections of abnormalities and improving inspection performance.

[0015] In this specification, a semiconductor circuit imaged by a scanning electron microscope (SEM) is used as an example of a sample and its captured image, but the present invention is not limited to this. It goes without saying that the present invention can also be applied to images captured by other imaging devices, such as optical inspection devices. Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] FIG. 1A shows an example of calculating the image difference using two line pattern images with differences due to image quality variations. FIG. 1B shows an example of calculating the image difference using two line pattern images with no image quality variations. As shown in FIG. 1A, the first image contains a defect. Furthermore, the white band at the boundary of the line pattern is thicker than in the second image, and there is a local brightness unevenness that darkens the upper right corner of the image. In image inspection using an SEM, these image quality variations can be caused by factors such as variations in electron beam control, differences between imaging devices, sample charging, or sample contamination. In this case, the image difference obtained by comparing the first and second images includes differences in image quality in addition to differences due to defects, making it difficult to detect only the defect abnormality. On the other hand, as shown in FIG. 1B, if there are no differences between the first and second images other than the defect abnormality, only the defect abnormality can be detected.

[0017] 1A shows the example of the variation in image quality, which is the phenomenon of a thick white band and the brightness unevenness where the image is locally darkened, but is not limited to these. For example, the variation in image quality may include uniform blurring or uneven blurring in the image, the phenomenon of a thin white band, the phenomenon of locally brightening the image, the phenomenon of overall differences in brightness and darkness in the image, and the misalignment of patterns between images due to image distortion.

[0018] 2 is a functional block diagram showing an example of the overall configuration of an image inspection device according to this embodiment. As shown in Fig. 2, the image inspection device 200 has a learning processing unit 203, an inference processing unit 208, a comparison processing unit 210, and an abnormality determination unit 212. Here, the learning processing unit 203, the inference processing unit 208, the comparison processing unit 210, and the abnormality determination unit 212 are realized, for example, by a processor such as a CPU (not shown), a ROM for storing various programs, a RAM for temporarily storing data in the calculation process, and a storage device such as an external storage device, and the processor such as the CPU reads and executes the various programs stored in the ROM, and stores the execution results, which are the calculation results, in the RAM, the external storage device, or cloud storage via a network connection, etc.

[0019] The learning processing unit 203 uses the first learning image 201, the corresponding second learning image 202, and a predetermined abnormality size 205 representing the size of an abnormality such as a predetermined defect to learn a machine learning model for performing a conversion process to bring the image quality of the first learning image 201 and / or the second learning image 202 closer to the image quality of the other, and creates an image quality correction model 204.

[0020] The inference processing unit 208 uses a first image 206, which is an inspection image in which an abnormality is to be found, a second image 207, which is used as a reference image for comparison inspection, a predetermined abnormality size 205, and an image quality correction model 204 created by the learning processing unit 203, to perform a conversion so that the image quality of the second image 207 approaches the image quality of the first image 206, and creates a corrected second image 209.

[0021] The comparison processing unit 210 calculates an image difference 211 using the corrected second image 209 and first image 206. An example of the image difference 211 is the absolute error in pixel values ​​between the two images as illustrated in Figures 1A and 1B, but it is not limited to this and may be any index that can represent the difference or similarity between the two images. For example, it may be the difference in edge positions obtained by performing a process to extract pattern edges for each image, or the cosine similarity of images calculated for each small region.

[0022] The abnormality determination unit 212 identifies an area having an abnormality in the first image 206 and / or the second image 207 from the image difference 211 through an analysis process including threshold processing for detecting an abnormality such as a defect.

[0023] Here, the processes performed by the learning processing unit 203, the inference processing unit 208, the comparison processing unit 210, and the abnormality determination unit 212 may be performed simultaneously or individually. Furthermore, as long as the computers that execute each unit are capable of transmitting and receiving output via a network connection, each unit may be configured to be executed by a different computer.

[0024] In this embodiment, the first image 206 is illustrated as the inspection image and the second image 207 as the reference image. However, the second image 207 may be the inspection image and the first image 206 as the reference image. Alternatively, the inference processing unit 208 may convert the image quality of the first image 206 to approximate that of the second image 207, thereby creating a corrected first image. In this case, the comparison processing unit 210 calculates the image difference 211 using the corrected first image and the second image 207. It is also possible to correct both the first image and the second image shown in FIG. 1A . For example, a correction may be performed to increase the luminance value of the dark (low luminance) region in the upper right corner of the first image shown in FIG. 1A by 50%, and a correction may be performed to decrease the luminance value of the region in the second image corresponding to the dark (low luminance) region in the upper right corner of the first image by 50%. The degree of correction is not limited to 50% and may be set as appropriate. In other words, the image quality of the first image and / or the second image is relatively approximated for each partial region.

[0025] In this embodiment, the first image 206 and the second image 207 are illustrated as including multiple variations in image quality, but it is not necessary that each image is a raw image obtained by capturing and have multiple variations in image quality. As pre-processing, any image processing such as contrast correction, shading correction, or template matching may be applied, and image quality that can be corrected in advance may be corrected.

[0026] Fig. 3 is a functional block diagram showing an example of the configuration of the inference processing unit 208 that constitutes the image inspection apparatus shown in Fig. 2. As shown in Fig. 3, the inference processing unit 208 of this embodiment includes an image quality feature extraction unit 301 and an image quality correction unit 305. The image quality feature extraction unit 301 has a region division processing unit 302 and a feature extraction processing unit 303, and calculates image quality feature amounts 304 that are results of calculating, for a plurality of partial regions from the first image 206, feature vectors that represent the image quality of each partial region.

[0027] The region segmentation processing unit 302 divides the first image 206 into N×M units (N rows and M columns) of partial regions based on a predetermined anomaly size 205 representing the size of a predetermined defect or other anomaly. Here, the predetermined anomaly size 205 is the number of vertical and / or horizontal pixels when a predetermined anomaly is resolved in an inspection image. The predetermined anomaly size 205 is determined by a user's input based on the type and size of the defect to be detected by inspection, or by calculation by analyzing one or more abnormal sample images. The size N×M (N rows and M columns) of the partial region is set to a value greater than the predetermined anomaly size 205. The region segmentation processing unit 302 can perform region segmentation by recording the coordinate values ​​and pixel values ​​of the divided partial regions, or by a process known as patch embedding, which linearly projects information about the partial regions into a feature vector that aggregates the information about the partial regions into one pixel.

[0028] The feature extraction processing unit 303 calculates a feature vector representing the image quality of one of the partial regions divided by the region division processing unit 302 by focusing on analyzing the features of the surrounding region other than the partial region. This process is repeated for the partial region to be inspected among the divided partial regions, or for all partial regions, to calculate an image quality feature 304.

[0029] The image quality correction unit 305 uses the image quality feature 304 obtained by the image quality feature extraction unit 301 and the second image 207 to perform conversion so that the image quality of the second image 207 approaches that of the first image 206. This conversion process is performed using a neural network included in the image quality correction model 204 obtained by the learning processing unit 203 and that outputs a corrected second image 209, but is not limited to this. For example, it is also possible to estimate correction parameters for making the image quality approach that of the first image 206 and correct the second image 207 based on the estimated correction parameters. A specific example will be described in Example 2.

[0030] FIG. 4 is a diagram illustrating an example of how the region segmentation processing unit 302 and feature extraction processing unit 303, which constitute the image quality feature extraction unit 301, calculate the image quality feature 304. As shown in FIG. 4 , the region segmentation processing unit 302 segments the first image into N×M units (units of N rows and M columns) larger than the predetermined abnormality size 205. The feature extraction processing unit 303 then masks one of the segmented regions, region a, with a value such as 0, and calculates a feature vector representing the image quality of region a from the surrounding region excluding region a. This process is repeated for the other regions to obtain the image quality feature 304. Here, the feature extraction processing unit 303 performs calculations using a neural network included in the image quality correction model 204 obtained by the learning processing unit 203 for each segmented region, but this is not limited to this. For example, pixel values ​​can be analyzed for each small region, and statistics representing the brightness, structure, and contrast of the image can be used.

[0031] Furthermore, as a method for focusing on analyzing the features of surrounding regions other than the partial region a for which a feature vector is calculated, a method of masking the partial region a by filling it with zero values ​​or the like, thereby making it possible to refer to only the pixel values ​​of the surrounding region, has been mentioned, but this is not limited to this. For example, a weighting process can be performed to reduce the values ​​of the partial region a and increase the values ​​of the surrounding region, thereby making the values ​​of the surrounding region have a greater influence on the feature vector than the partial region a. Furthermore, if the region segmentation processing unit 302 is implemented using a process that aggregates information about partial regions into one pixel, such as patch embedding, a similar process can be implemented by using a neural network whose receptive field has regions that are not subject to feature extraction, and setting the regions that are not subject to feature extraction as partial region a.

[0032] As shown in FIG. 4 , the feature vector representing partial region a, calculated from the surrounding region excluding partial region a, can represent pattern changes that occur beyond partial region a and across the surrounding region, such as variations in image quality, because the surrounding region can be referenced. However, pattern changes smaller than the size N×M (N rows and M columns) of partial region a, such as defects, cannot be represented because partial region a cannot be referenced. Therefore, the feature vector representing partial region a calculated in this embodiment represents the image quality of the first image and does not have features that represent abnormalities such as defects. Therefore, even if this feature vector is used to correct the second image, it will not correct defects contained in the first image, and only the differences due to variations in image quality can be corrected.

[0033] Fig. 5 is a flowchart showing the learning processing operation of the learning processing unit 203 shown in Fig. 2. As shown in Fig. 5, when the learning processing starts, in step S501, the first learning image 201, the second learning image 202, and a predetermined abnormality size 205 are input to the learning processing unit 203. In step S502, the learning processing unit 203 divides the first learning image 201 into partial regions by processing similar to that of the region division processing unit 302 described above.

[0034] In step S503, the learning processing unit 203 selects one of the divided partial regions by the same processing as that performed by the above-described feature extraction processing unit 303. In step S504, the learning processing unit 203 performs the same processing as that performed by the above-described feature extraction processing unit 303, focusing on analyzing the surrounding area of ​​the selected partial region, and calculates (extracts) a feature vector representing the image quality corresponding to the selected partial region.

[0035] In step S505, if the processes from step S503 to step S504 have not been performed on all partial regions that are the subject of the learning process and whose image quality should be corrected (NO), the learning processing unit 203 returns to step S503. On the other hand, if the processes from step S503 to step S504 have been performed (YES), the learning processing unit 203 outputs the image quality feature to the image quality correction unit 305. In step S506, the learning processing unit 203 uses the image quality feature output in step S505 and the second learning image 202 to perform correction processing to relatively approximate the image quality of the first learning image 201 and / or the second learning image 202 by processing similar to that of the image quality correction unit 305 described above.

[0036] In step S507, the learning processing unit 203 evaluates the error function of the image quality correction model using a pair of an image corrected in step S505 and an uncorrected image from among the first learning image 201 and the second learning image 202. Here, the error function of the image quality correction model 204 ( FIG. 2 ) evaluates the difference between the image whose image quality has been corrected in step S505 and the uncorrected image, and examples of this include the absolute error or squared error of pixel values.

[0037] In step S508, the learning processing unit 203 calculates the amount of update of the weight parameters of the image quality correction model 204 based on the evaluation result in step S507 so as to reduce the value of the error function of the image quality correction model 204, and updates the parameters according to the amount of update. This update is performed by, for example, the stochastic gradient descent method.

[0038] In step S509, the learning processing unit 203 determines whether or not the learning termination condition has been reached, and if it is determined that the learning termination condition has been reached (YES), the process proceeds to step S510, where the image quality correction model 204 is saved and the learning process is terminated. On the other hand, if it is determined that the learning termination condition has not been reached (NO), the process returns to step S501 and the processes from step S501 onwards are executed again.

[0039] The conditions for terminating the learning include whether the processing from step S501 to step S509 has been repeated a predetermined number of times or more, or whether the value of the error function calculated in step S507 does not become smaller even after the processing from step S501 to step S509 has been repeated a predetermined number of times, and therefore it is determined that the learning of the image quality correction model 204 has converged.

[0040] 6 is a flowchart showing the processing operation of the inspection process of the image inspection device 200 according to this embodiment. As shown in Fig. 6, when the inspection process starts, in step S601, a first image 206 which is an inspection image, a second image 207 which is a reference image, a predetermined abnormality size 205, and an image quality correction model 204 created by the learning processing unit 203 are input to the inference processing unit 208.

[0041] In step S602, the region division processing unit 302 of the image quality feature extraction unit 301 constituting the inference processing unit 208 divides the first image 206 into partial regions based on the predetermined abnormality size 205. In step S603, the feature extraction processing unit 303 of the image quality feature extraction unit 301 selects one of the divided partial regions.

[0042] In step S604, the feature extraction processing unit 303 of the image quality feature extraction unit 301 focuses on analyzing the surrounding area of ​​the selected partial area and calculates (extracts) a feature vector representing the image quality corresponding to the selected partial area. In step S605, if the processes from step S603 to step S604 have not been performed for all partial areas that are the subject of inspection processing and whose image quality should be corrected (NO), the image quality feature extraction unit 301 returns to step S603. On the other hand, if the processes from step S603 to step S604 have been performed (YES), the image quality feature extraction unit 301 outputs the image quality feature 304 to the comparison processing unit 210.

[0043] In step S606, the image quality correction unit 305 constituting the inference processing unit 208 uses the image quality feature 304 output in step S605 and the second image 207 to perform conversion so that the image quality of the second image 207 approaches the image quality of the first image 206. The inference processing unit 208 outputs the corrected second image 209 to the comparison processing unit 210. In step S607, the comparison processing unit 210 calculates an image difference 211 using the corrected second image 209 and the first image 206 output in step S606. In step S608, the comparison processing unit 210 outputs the image difference 211 to the abnormality determination unit 212 ( FIG. 2 ).

[0044] In step S609, the abnormality determination unit 212 uses the image difference 211 output in step S608 to identify abnormal areas in the first image 206 and / or the second image 207 through analysis processing including threshold processing for detecting abnormalities such as defects, outputs the inspection results, and ends the inspection processing.

[0045] As described above, this embodiment provides an image inspection device and an image processing method that are robust against variations in image quality and can detect defects and other abnormalities with high accuracy. Furthermore, this embodiment provides the image quality feature 304 output by the image quality feature extraction unit 301, which represents only image features spanning each partial region determined by the predetermined abnormality size 205, and does not include features representing defects and other abnormalities. Therefore, even if the image quality correction unit 305 corrects the second image 207 using the image quality feature 304, it is possible to correct only differences due to variations in image quality without correcting the image to reproduce defects contained in the first image 206. As a result, when the comparison processing unit 210 compares the corrected second image 209 with the first image 206, it is possible to obtain only image differences resulting from defects and other abnormalities, thereby reducing false detection of abnormalities due to variations in image quality.

[0046] An image inspection apparatus and an image processing method according to a second embodiment of the present invention will be described with reference to FIGS. 7, 8A, and 8B. Hereinafter, components similar to those in the first embodiment will be designated by the same reference numerals, and redundant description will be omitted. FIG. 7 is a functional block diagram showing the configuration of an inference processing unit 208 according to this embodiment. As shown in FIG. 7, the image quality correction unit 305 according to this embodiment includes a correction amount estimation processing unit 701 and a correction amount reflection processing unit 702. The correction amount estimation processing unit 701 and the correction amount reflection processing unit 702 are implemented, for example, by a processor such as a CPU (not shown), a ROM (read-only memory) for storing various programs, a RAM (random access memory) for temporarily storing data during the calculation process, and a storage device such as an external storage device. The processor such as the CPU reads and executes the various programs stored in the ROM, and stores the execution results in the RAM, an external storage device, or cloud storage via a network connection. The correction amount estimation processing unit 701 estimates one or more image processing parameters as image quality correction amounts for each factor that causes variation in image quality. The correction amount reflection processing unit 702 applies image processing to one of the first image 206 and the second image 207 using the image quality correction amount estimated by the correction amount estimation processing unit 701, thereby performing correction to bring the image quality closer to that of the other image.

[0047] Fig. 8A is a diagram illustrating factors that cause variations in image quality in this embodiment, and Fig. 8B is a diagram illustrating image quality correction amounts estimated in this embodiment for the case of Fig. 8A. As shown in Fig. 8A, a case will be described in which differences due to variations in three image qualities, namely, line pattern positional misalignment, image blur, and brightness unevenness, exist between a first image and a second image. In this case, as shown in Fig. 8B, the image quality correction amounts are estimated as a pixel value correction amount for correcting brightness unevenness, a positional misalignment correction amount for correcting positional misalignment, and an image blur correction amount for correcting image blur.

[0048] The pixel value correction amount is the difference in pixel values ​​between the first and second images that occurs across the partial regions divided by the region division processing unit 302, and is a parameter for brightness fluctuations that represent the brightness and darkness of the image, such as brightness unevenness and contrast fluctuations that occur locally or globally. An example format is an image format with the same size as the second image, where the correction value is stored for each pixel. In this case, to output the corrected second image y, the correction amount reflection processing unit 702 performs a linear mapping of y = αx + β using the pixel value x of the second image and the pixel value correction amounts α and β. In the example of pixel value correction amounts shown on the left side of Figure 8B, when the value is "+0", α = 1 and β = 0. When the value is "-30", α = 1 and β = -30.

[0049] The misalignment correction amount represents the pattern misalignment between the first and second images and serves as a parameter for geometric transformation. The central diagram in FIG. 8B illustrates an example of the downward misalignment amount of pixel values, but this is not limiting. For example, the misalignment amount may be expressed in the form of a two-dimensional vector (dx, dy) indicating the horizontal and vertical misalignment amounts, and the correction amount reflection processing unit 702 may perform a geometric transformation in which the pixel values ​​of the second image are shifted by (dx, dy) pixels. Furthermore, when correcting translational or rotational misalignment occurring across the entire image, the misalignment correction amount serves as a parameter matrix for affine transformation, and the correction amount reflection processing 702 performs affine transformation processing. The central diagram in FIG. 8B illustrates an example of a downward correction by five pixels.

[0050] The image blur correction amount represents the diffusion of pixel values ​​in the difference between the first and second images. Variations in image quality due to the diffusion of pixel values ​​include, for example, blurring of pattern edges, or the thickening or thinning of white bands. The right diagram in FIG. 8B illustrates the kernel of a convolution filter for a 5×5 (5 rows and 5 columns) image as an example of the image blur correction amount. In this case, the correction amount estimation processing unit 701 estimates kernel elements k00 to k44 for each image or each subregion. The correction amount reflection processing unit 702 then uses the estimated kernels to perform convolution processing for each image or each subregion, correcting the diffusion of pixel values. As another example, if the convolution filter is assumed to be a Gaussian filter, the variance, which is a parameter of the Gaussian filter, can be used as the image blur correction amount.

[0051] As described above, according to this embodiment, it is possible to select and correct the image quality to be corrected based on the variation in image quality between the first image and the second image. An example of a case in which this embodiment is applied is a case in which it is assumed that image brightness fluctuations, such as brightness unevenness, do not occur due to the characteristics of the imaging device or the sample, and the occurrence of image brightness fluctuations is abnormal and should not be corrected. Similarly, if the occurrence of pattern misalignment is considered abnormal, the processing contents of the correction amount estimation processing unit 701 and the correction amount reflection processing unit 702 can be changed so that the image quality is corrected using an image quality correction amount other than the misalignment correction amount.

[0052] An image inspection apparatus and an image processing method according to this embodiment will be described with reference to Fig. 9. In this embodiment, the second image 207 is one or more images having, as pixel values, parameters of a probability distribution of brightness values ​​of the captured image, calculated from design data of the sample by simulation or machine learning such as a neural network, and the image quality correction unit 305 (Fig. 3) corrects the parameters of the probability distribution, and the corrected second image 209 is a conditional probability distribution conditioned by image quality, which is different from the first embodiment.

[0053] As shown in Fig. 9, the image inspection device 200 of this embodiment sets an inspection image I as the first image and a probability distribution image p(x|D) obtained from design data D as the second image. Here, the probability distribution image p(x|D) is defined for each coordinate (i, j) of the image and is expressed as one image for each parameter. In Fig. 9, the probability distribution is assumed to be a normal distribution having two parameters, the mean and the standard deviation, and is expressed as a mean image and a standard deviation image.

[0054] In this embodiment, the image quality correction unit 305 (FIG. 3) receives the inspection image I as input, and uses the image quality feature Q output by the image quality feature extraction unit 301 (FIG. 3) and the probability distribution image p(x|D) to correct the probability distribution into a conditional probability distribution p(x|D,Q) that can express the image quality of the inspection image.

[0055] The correction method may be the conversion using a neural network as exemplified in the above-mentioned Example 1, or the method of calculating the correction amount for each of the average image and the standard deviation image as exemplified in Example 2. Furthermore, although the use of a probability distribution image as the second image has been exemplified, if the design data D is obtained in an image format, the design data D may be used as the second image, and in this case, the image quality to be corrected includes the deviation of the pattern shapes between the inspection image I and the design data D, the change in the range of the luminance values, etc.

[0056] In this embodiment, when the corrected second image 209 is expressed by a conditional probability distribution p(x|D,Q), the comparison processing unit 210 ( FIG. 2 ) calculates the image difference 211 between the inspection image I and the conditional probability distribution p(x|D,Q). As illustrated in FIG. 9 , assuming the probability distribution is a normal distribution, the image difference 211 can be determined as the degree of abnormality expressed as |x-μ| / σ, using the mean μ and standard deviation σ corresponding to the pixel value x at coordinates (i,j). The abnormality determination unit 212 ( FIG. 2 ) may then determine a pixel having an abnormality degree exceeding a specified threshold as a defect. Furthermore, the error function of the image quality correction model in step S507 ( FIG. 5 ) of the learning processing unit 203 can be the negative log-likelihood of the conditional probability distribution p(x|D,Q), which is the corrected second image, with respect to the inspection image I, which is the first image.

[0057] As described above, according to this embodiment, the corrected second image, which is the reference image during the comparison process, has a conditional probability distribution conditioned on the image quality of the first image, which is the inspection image, so that the pattern variations due to manufacturing tolerances contained in the first image and the pixel value variations due to imaging noise can be evaluated as variation components of a probability distribution such as the standard deviation of a normal distribution, thereby enabling image inspection that is robust to noise.

[0058] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.

[0059] 200...image inspection device, 201...first image for learning, 202...second image for learning, 203...learning processing unit, 204...image quality correction model, 205...predetermined abnormality size, 206...first image, 207...second image, 208...inference processing unit, 209...corrected second image, 210...comparison processing unit, 211...image difference, 212...abnormality determination unit, 301...image quality feature extraction unit, 302...region segmentation processing unit, 303...feature extraction processing unit, 304...image quality feature, 305...image quality correction unit, 701...correction amount estimation processing unit, 702...correction amount reflection processing unit

Claims

1. An image inspection device for comparing a first image and a second image, obtaining the image differences, and inspecting for abnormalities that may exist in either the first image or the second image, comprising: an image quality feature extraction unit that extracts image quality features, which are feature vectors expressing the image quality for each partial region of the first image; an image quality correction unit that brings the image quality of the first image and / or the second image relatively closer for each partial region based on the feature vectors obtained by the image quality feature extraction unit and the second image; and a comparison processing unit that compares either of the images corrected by the image quality correction unit with the other image, or the corrected first image and second image, and obtains the image differences, wherein the image quality feature extraction unit has: a region division processing unit that divides the first image into partial regions larger than a predetermined size of an abnormality; and a feature extraction processing unit that extracts feature vectors expressing the image quality of each partial region from other partial regions of the first image.

2. An image inspection device as described in claim 1, comprising a learning processing unit that uses a first learning image and a second learning image corresponding to the first learning image to create a model for performing conversion to relatively approximate the image quality of the first image and / or the second image for each partial region, and the image quality feature extraction unit and the image quality correction unit use the model created by the learning processing unit.

3. An image inspection device as described in claim 2, wherein the learning processing unit corrects the image quality of the first learning image and / or the second learning image to bring the image quality closer relatively for each partial area, compares the corrected image with another image, or the corrected first image and second image, and learns a model so that the difference is small.

4. An image inspection device as described in claim 1, wherein the feature extraction processing unit has a pre-processing unit that weights the pixel values ​​of the subregion and surrounding subregions so as to extract a feature vector expressing the image quality of the subregion with more emphasis on other surrounding subregions than on the subregion itself.

5. An image inspection device as described in claim 2, wherein the area segmentation processing unit is a patch embedding process that aggregates information about the surrounding areas into one pixel, and the feature extraction processing unit is a process using a neural network whose receptive field has areas that are not subject to feature extraction.

6. An image inspection device as described in claim 1, wherein the second image is one or more images having parameters of a probability distribution of the luminance values ​​of the captured image as pixel values, and the object to be corrected in the image quality correction unit is the parameters of the probability distribution, and the corrected second image becomes a conditional probability distribution conditioned by the image quality.

7. An image inspection device as described in claim 1, wherein the image quality correction unit has: a correction amount estimation processing unit that estimates parameters corresponding to one or more image processes as image quality correction amounts for each factor that generates variation in image quality; and a correction amount reflection processing unit that performs correction to relatively bring the image quality closer for each partial area by applying image processing using the image quality correction amount estimated by the correction amount estimation processing unit to the first image and / or the second image.

8. An image inspection device as described in claim 7, wherein one of the image quality correction amounts is a vector amount or a parameter expressing a positional misalignment of the patterns of the first image and the second image including image distortion, and the correction amount reflection processing unit includes a process of aligning the position of one of the patterns of the first image and the second image to the position of the other pattern based on the image quality correction amount.

9. An image inspection device as described in claim 7, wherein one of the image quality correction amounts is a value expressing a difference between the pixel values ​​of the first image and the second image, including brightness fluctuations, that has an effect over a range larger than the size of a predetermined abnormality, and the correction amount reflection processing unit reduces the difference between the pixel value of one of the first image and the second image and the pixel value of the other image by changing the pixel value of the other image based on the image quality correction amount.

10. An image inspection device as described in claim 7, wherein one of the image quality correction amounts is a parameter of an image processing filter representing a difference in diffusion of pixel values ​​between the first image including image blur and the second image, and the correction amount reflection processing unit includes a process of reducing the difference in the degree of diffusion of pixel values ​​by applying an image processing filter to one of the first image and the second image based on the image quality correction amount.

11. An image processing method for processing images to compare a first image and a second image, obtain image differences, and inspect for abnormalities that may exist in either the first image or the second image, comprising: an image quality feature extraction unit extracts image quality features, which are feature vectors expressing the image quality for each partial region of the first image; an image quality correction unit brings the image quality of the first image and / or the second image relatively closer for each partial region based on the feature vector and the second image; a comparison processing unit compares one of the images corrected by the image quality correction unit with the other image, or the corrected first image and second image, to obtain image differences; a region division processing unit divides the first image into partial regions in units larger than a predetermined size of an abnormality; and a feature extraction processing unit extracts feature vectors expressing the image quality of each partial region from other partial regions of the first image.

12. An image processing method as described in claim 11, characterized in that a learning processing unit uses a first learning image and a second learning image corresponding to the first learning image to create a model for performing conversion to relatively approximate the image quality of the first image and / or the second image for each partial region, and the image quality feature extraction unit and the image quality correction unit use the model created by the learning processing unit.

13. An image processing method as described in claim 12, characterized in that the learning processing unit corrects the image quality of the first learning image and / or the second learning image to bring the image quality closer relatively for each partial region, compares the corrected image with another image, or the corrected first image and second image, and learns a model so that the difference is small.

14. An image processing method as claimed in claim 11, characterized in that the second image is one or more images having parameters of a probability distribution of the luminance values ​​of the captured image as pixel values, and the object of correction in the image quality correction unit is the parameters of the probability distribution, and the corrected second image becomes a conditional probability distribution conditioned by image quality.

15. An image processing method as claimed in claim 11, characterized in that a correction amount estimation processing unit estimates parameters corresponding to one or more image processes as image quality correction amounts for each factor that generates variation in image quality, and a correction amount reflection processing unit applies image processing using the image quality correction amount estimated by the correction amount estimation processing unit to the first image and / or the second image, thereby performing correction to relatively bring the image quality closer for each partial region.

Images