Image inspection apparatus and image processing method

Abstract

The invention provides an image inspection device and an image processing method, which are capable of detecting abnormalities such as defects and the like in a robust and high-precision manner with respect to variations in image quality. An image inspection device for comparing a first image with a second image, acquiring a difference between the images, and inspecting an abnormality that may exist in either the first image or the second image, the image inspection device being provided with: an image quality feature amount extraction unit that extracts an image quality feature amount for each partial region of the first image; extracting a feature quantity vector representing image quality, namely an image quality feature quantity; an image quality correction unit that relatively approaches the image quality of the first image and / or the second image for each partial region on the basis of the feature quantity vector obtained by the image quality feature quantity extraction unit and the second image; and a comparison processing unit that compares one of the images corrected by the image quality correction unit with the other image, or the first image corrected by the image quality correction unit with the second image to obtain a difference between the images, the image quality feature amount extraction unit being provided with: a region division processing unit that divides the difference between the images; the first image segmentation unit is used for segmenting the first image into unit partial regions larger than a predetermined abnormal size; and a feature quantity extraction processing unit that extracts, from another partial region of the first image, a feature quantity vector representing the image quality of each partial region.

Description

Technical Field

[0001] The present invention relates to an image processing technology for processing image data, and in particular to an image inspection device and an image processing method suitable for inspection using image data. Background Art

[0002] To evaluate semiconductor circuit defects, a method is used to compare inspection images with reference images. The reference image includes the design data of the sample being evaluated, an image simulated using the design data through machine learning, or other methods, an image statistic representing, for each pixel, the distribution of pixel values ​​achievable in images of samples manufactured using the same design data, and other areas of the captured image with the same pattern as the sample.

[0003] During defect inspection, the captured images generated by the device sometimes exhibit image quality variations, such as distortion, positional shifts, uneven brightness, and blurring due to defocusing caused by adjustments to the light source, sensor imaging, or sample characteristics, such as the sample's pattern or characteristics. For example, during the inspection of semiconductor circuits using a scanning electron microscope (SEM), image quality variations can occur primarily due to variations in electron beam control, mechanical differences between imaging devices, sample electrification, and sample contamination. In such cases, a difference between the inspection image and the reference image, different from anomalies such as defects, can occur, leading to the judgment that a previously normal area is abnormal.

[0004] Therefore, a technique has been proposed, for example, as described in Patent Document 1. Patent Document 1 discloses using parameters of a semiconductor manufacturing process or an imaging process as inputs to a machine learning model to vary image statistics of a normal image generated based on design data.

[0005] In addition, Patent Document 2 discloses the following: In order to perform a robust inspection for individual differences in an image, a partial area of ​​the inspection image is cut out, and based on an image with the center of the cut-out area removed, a normal pattern of the removed center is generated by a machine learning model, and the normal pattern is compared with the image of the removed center for inspection.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: International Publication No. 2021-255819

[0009] Patent Document 2: International Publication No. 2019-146538 Summary of the Invention

[0010] Problems to be solved by the invention

[0011] The difference between the inspection image and the reference image includes differences due to defects and other anomalies, as well as differences due to image quality variations. In comparative inspection, it is desirable to only obtain differences due to defects and other anomalies, and differences due to image quality variations must be eliminated. However, image processing that simply reduces the difference between the reference and inspection images cannot distinguish between differences due to image quality variations and those due to defects and other anomalies. Therefore, only image quality differences can be corrected, requiring only detection of anomalies.

[0012] In Patent Document 1, when comparing image statistics of a normal image generated from design data with a test image, the machine learning model that generates the image statistics includes parameters related to the manufacturing and imaging processes as inputs, enabling inspection that takes into account image quality variations caused by imaging. However, since the state of the sample containing the circuit pattern is not considered, performance may be degraded due to image quality variations caused by samples whose pattern positions and shapes vary depending on the sample state.

[0013] Patent Document 2 uses a machine learning model to generate a normal pattern for the location to be inspected for comparison. However, if the location to be inspected cannot be uniquely determined for the surrounding area, there is a concern that the accuracy of the normal pattern generation may decrease. Therefore, in semiconductor inspections involving fine and complex patterns, it is preferable to use a reference image that includes a normal pattern.

[0014] Therefore, the present invention provides an image inspection device and an image processing method that are robust to variations in image quality and can detect abnormalities such as defects with high accuracy.

[0015] Means for solving problems

[0016] In order to solve the above-mentioned problems, the image inspection device of the present invention is used to compare a first image with a second image, obtain the difference between the images and detect abnormalities that may exist in either the first image or the second image, wherein the image inspection device comprises: an image quality feature extraction unit, which extracts a feature vector representing the image quality, i.e., an image quality feature, for each partial area of ​​the first image; an image quality correction unit, which makes the image quality of the first image and / or the second image relatively close according to each partial area based on the feature vector obtained by the image quality feature extraction unit and the second image; and a comparison processing unit, which compares any image corrected by the image quality correction unit with another image, or the corrected first image with the second image to obtain the difference between the images, wherein the image quality feature extraction unit comprises: an area segmentation processing unit, which divides the first image into partial areas of units larger than the size of a predetermined abnormality; and a feature extraction processing unit, which extracts feature vectors representing the image quality of each partial area from other partial areas of the first image.

[0017] In addition, the image processing method of the present invention compares a first image and a second image to obtain the difference between the images, and processes the images in order to detect abnormalities that may exist in either the first image or the second image, wherein the image quality feature extraction unit extracts a feature vector representing the image quality, i.e., the image quality feature, for each partial area of ​​the first image, and the image quality correction unit makes the image quality of the first image and / or the second image relatively close to each partial area based on the feature vector and the second image, the comparison processing unit compares any image corrected by the image quality correction unit with another image, or the corrected first image with the second image to obtain the difference between the images, the region segmentation processing unit divides the first image into partial areas of units larger than the size of a predetermined abnormality, and the feature extraction processing unit extracts feature vectors representing the image quality of each partial area from other partial areas of the first image.

[0018] Effects of the Invention

[0019] According to the present invention, it is possible to provide an image inspection apparatus and an image processing method that are robust against variations in image quality and can detect abnormalities such as defects with high accuracy.

[0020] Other problems, structures, and effects than those described above will become clear from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A This diagram shows an example of calculating an image difference using two images having a difference due to variations in image quality.

[0022] Figure 1B This diagram shows an example of calculating image differences using two images with no difference in image quality.

[0023] Figure 2 This is a functional block diagram showing an example of the overall configuration of an image inspection apparatus according to the first embodiment of the present invention.

[0024] Figure 3 It means composition Figure 2 The figure shows a functional block diagram of the structure of the estimation processing unit of the image inspection device.

[0025] Figure 4 It means composition Figure 3 FIG. 1 is a diagram showing an example of a processing method of the image quality feature value extraction section of the estimation processing section.

[0026] Figure 5 This is a flowchart showing the processing operations of the estimation processing unit constituting the image inspection apparatus of the first embodiment.

[0027] Figure 6 This is a flowchart showing the processing operation of the learning processing unit constituting the image inspection apparatus of the first embodiment.

[0028] Figure 7 This is a functional block diagram showing the configuration of an estimation processing unit according to a second embodiment of the present invention.

[0029] Figure 8A This is a diagram showing an example of the variation in image quality in Example 2.

[0030] Figure 8B 1 is a diagram showing an example of the image quality correction amount in the second embodiment.

[0031] Figure 9 This is a diagram showing an example of the probability distribution correction process in the third embodiment. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present invention will be described using the accompanying drawings. In the drawings, the same components are denoted by the same reference numerals, and detailed descriptions of the overlapping components will be omitted.

[0033] The image inspection apparatus exemplified in this specification relates to an image inspection apparatus and an image processing method for reducing a difference caused by a deviation in image quality between a first image and a second image to reduce false detection of abnormalities and thereby improve inspection performance.

[0034] In this specification, semiconductor circuits photographed by a scanning electron microscope (SEM) are exemplified as samples and images thereof. However, the present invention is not limited thereto and is also applicable to images photographed by other imaging devices such as optical inspection equipment.

[0035] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0036] Example 1

[0037] Figure 1A This diagram shows an example of calculating image differences using two line pattern images having a difference due to variations in image quality. Figure 1B : is a diagram showing an example of calculating the difference of an image using two line pattern images with no difference in image quality. Figure 1A As shown, the first image has an abnormality of a defect. Moreover, compared with the second image, the white band on the boundary surface of the line pattern is thicker, and there is a local uneven brightness that makes the upper right corner of the image darker. In image inspection using SEM, these deviations in image quality sometimes occur due to deviations in electron beam control, mechanical differences between imaging devices, charging of the sample, or contamination of the sample. In this case, the difference in the image obtained by comparing the first image and the second image includes not only the difference based on the defect, but also the difference in image quality, making it difficult to detect only the abnormality of the defect. On the other hand, as Figure 1B As shown, when there is no difference between the first image and the second image other than the abnormality of the defect, only the abnormality of the defect can be detected.

[0038] Here, as Figure 1A The examples of image quality variations shown in FIG. 1 and FIG. 2 show thickening of white bands and uneven brightness resulting from localized darkening of the image, but are not limited thereto. Examples include uniform blurring across the image, uneven blurring, thinning of white bands, localized brightening of the image, overall brightness differences in the image, and positional shifts of patterns between images due to image distortion.

[0039] Figure 2 This is a functional block diagram showing an example of the overall configuration of the image inspection apparatus according to this embodiment.

[0040] like Figure 2As shown, the image inspection apparatus 200 includes a learning processing unit 203, an estimation processing unit 208, a comparison processing unit 210, and an abnormality determination unit 212. Here, the learning processing unit 203, the estimation processing unit 208, the comparison processing unit 210, and the abnormality determination unit 212 are implemented by, for example, a processor (not shown) such as a CPU, a ROM storing various programs, a RAM temporarily storing data during calculations, and a storage device such as an external storage device. The CPU or other processor reads and executes the various programs stored in the ROM, and stores the resulting calculation results in a cloud storage via the RAM, the external storage device, or a network connection.

[0041] The learning processing unit 203 uses the first learning image 201, the corresponding second learning image 202, and a predetermined abnormality size 205 indicating the size of an abnormality such as a defect to learn a machine learning model for performing conversion processing to make the image quality of the first learning image 201 and / or the second learning image 202 close to the image quality of the other party, and produces an image quality correction model 204.

[0042] The inference processing unit 208 uses the inspection image in which the abnormality is to be discovered, namely the first image 206, the second image 207 used as a reference image for comparative inspection, the predetermined abnormality size 205, and the image quality correction model 204 generated by the learning processing unit 203, to convert the image quality of the second image 207 so that the image quality is close to that of the first image 206, and generates a corrected second image 209.

[0043] The comparison processing unit 210 calculates an image difference 211 using the corrected second image 209 and the first image 206 .

[0044] As an example of the image difference 211, there is Figure 1A and Figure 1B The absolute error of pixel values ​​between the two images is exemplified in FIG. However, any indicator that can represent the difference or similarity between the two images is not limited thereto. For example, edge position differences obtained by extracting pattern edges from each image, and cosine similarity between images calculated for each small area can be used.

[0045] The abnormality determination unit 212 specifies an abnormal region in the first image 206 and / or the second image 207 based on the image difference 211 through analysis processing including threshold processing for detecting abnormalities such as defects.

[0046] Here, the various processes performed by the learning processing unit 203, the estimation processing unit 208, the comparison processing unit 210, and the abnormality determination unit 212 may be performed simultaneously or independently. Furthermore, each component may be executed by a separate computer as long as the computer executing each component can send and receive output via a network connection.

[0047] In this embodiment, the first image 206 is exemplified as the inspection image and the second image 207 is exemplified as the reference image. However, the second image 207 may be the inspection image and the first image 206 may be the reference image. In addition, the estimation processing unit 208 may perform a conversion so that the image quality of the first image 206 approaches the image quality of the second image 207 to generate a corrected first image. In this case, the comparison processing unit 210 calculates the image difference 211 using the corrected first image and the second image 207. Alternatively, the comparison processing unit 210 may be configured to compare the first image 206 with the second image 207. Figure 1A For example, the correction can be performed on both the first image and the second image. Figure 1A In the example shown, the brightness of the dark (low brightness) area in the upper right corner of the first image is increased by 50%, while the brightness of the area in the second image corresponding to the dark (low brightness) area in the upper right corner of the first image is decreased by 50%. The degree of correction is not limited to 50% and can be set appropriately. In other words, the image quality of the first and / or second images is made relatively close for each partial area.

[0048] In this embodiment, a case where the first image 206 and the second image 207 include multiple deviations in image quality is illustrated, but it is not necessarily required that each image have multiple deviations in image quality in the original image obtained by shooting. As a preprocessing, arbitrary image processing such as contrast correction, shading correction, and template matching can also be applied to correct the image quality that can be corrected in advance.

[0049] Figure 3 It means composition Figure 2 FIG. 2 is a functional block diagram of a configuration example of the estimation processing unit 208 of the image inspection apparatus shown in FIG.

[0050] like Figure 3 As shown, the estimation processing unit 208 of this embodiment includes an image quality feature value extraction unit 301 and an image quality correction unit 305 .

[0051] The image quality feature extraction unit 301 includes a region segmentation unit 302 and a feature extraction unit 303 , and calculates an image quality feature 304 based on the first image 206 , which is a feature vector representing the image quality of each of the plurality of partial regions.

[0052] The region segmentation processing unit 302 segments the first image 206 into N×M units (N rows and M columns) of partial regions based on a predetermined abnormality size 205, which indicates the size of a predetermined abnormality such as a defect. The predetermined abnormality size 205 is the number of vertical and / or horizontal pixels used when analyzing the inspection image for the predetermined abnormality. It is determined by user input based on the type and size of the defect to be inspected, or by analyzing and calculating one or more abnormal sample images. The size of the partial region, N×M (N rows and M columns), is set to a value larger than the predetermined abnormality size 205.

[0053] In addition, in addition to the method of recording the coordinate values ​​and pixel values ​​of the divided partial area, the area segmentation method in the area segmentation processing unit 302 can also be implemented by a process called patch embedding, etc., which linearly projects the information of the partial area into a feature vector of 1 pixel.

[0054] The feature extraction processing unit 303 calculates a feature vector representing the image quality of one of the partial regions segmented by the region segmentation processing unit 302 by focusing on analyzing the features of the surrounding area outside the partial region. This process is repeated for the partial region or all the partial regions to be inspected after the segmentation, and the image quality feature 304 is calculated.

[0055] The image quality correction unit 305 uses the image quality feature 304 obtained by the image quality feature extraction unit 301 and the second image 207 to convert the image quality of the second image 207 so that it approaches the image quality of the first image 206. This conversion process is performed using the neural network included in the image quality correction model 204 obtained by the learning processing unit 203, which outputs the corrected second image 209, but is not limited to this. For example, correction parameters for bringing the image quality closer to that of the first image 206 can be estimated, and the second image 207 can be corrected based on these correction parameters. A specific example will be described in Example 2.

[0056] Figure 4 1 is a diagram showing an example of a method of calculating the image quality feature value 304 by the region segmentation processing unit 302 and the feature value extraction processing unit 303 constituting the image quality feature value extraction unit 301 .

[0057] like Figure 4As shown, the first image is segmented by the region segmentation processing unit 302 into N×M units (N rows and M columns) larger than the predetermined abnormality size 205. Then, the feature extraction processing unit 303 masks a portion a, one of the segmented portions, with a value such as zero. Based on the surrounding area excluding portion a, a feature vector representing the image quality of portion a is calculated. Similar processing is performed on the remaining portions to obtain image quality feature values ​​304.

[0058] Here, the feature extraction processing unit 303 performs calculations using the neural network included in the image quality correction model 204 obtained by the learning processing unit 203 for each segmented partial region, but the present invention is not limited to this. For example, it is also possible to analyze the pixel values ​​for each small region and use statistics that express the brightness, structure, and contrast of the image.

[0059] In addition, as a method for focusing on analyzing the feature values ​​of the surrounding area other than the partial area a for calculating the feature value vector, a method is listed in which the partial area a is masked by filling it with a value of 0, etc., so that only the pixel values ​​of the surrounding area can be referenced, but it is not limited to this. For example, by performing a weighted processing to reduce the value of the partial area a and increase the value of the surrounding area, the value of the surrounding area has a greater influence on the feature value vector than the partial area a. Furthermore, in the case where the region segmentation processing unit 302 aggregates the information of the partial area into a single pixel through a process such as block embedding, the same processing can be achieved by using a neural network that has an area that is not the object of feature value extraction in the receptive field and setting the area that is not the object of feature value extraction as the partial area a.

[0060] like Figure 4 As shown, regarding the feature vector representing partial area a calculated based on the surrounding area excluding partial area a, pattern changes that occur beyond partial area a and across the surrounding area, such as deviations in image quality, can be represented by reference to the surrounding area. However, pattern changes smaller than the size N×M (N rows and M columns) of partial area a, such as defects, cannot be represented by reference to partial area a. Therefore, the feature vector representing partial area a calculated by this embodiment becomes a feature vector representing the image quality of the first image and does not have features that represent abnormalities such as defects. Therefore, even if it is used to correct the second image, defects contained in the first image will not be corrected, and only differences caused by deviations in image quality can be corrected.

[0061] Figure 5 Yes Figure 2 The flowchart of the learning processing operation of the learning processing unit 203 is shown in FIG. Figure 5As shown, when the learning process starts, in step S501 , the first learning image 201 , the second learning image 202 , and the predetermined abnormality size 205 are input to the learning processing unit 203 .

[0062] In step S502 , the learning processing unit 203 divides the first learning image 201 into partial regions by the same processing as that of the above-described region division processing unit 302 .

[0063] In step S503 , the learning processing unit 203 selects one from the divided partial regions through the same processing as that of the feature extraction processing unit 303 .

[0064] In step S504 , the learning processing unit 203 performs the same processing as the feature extraction processing unit 303 to focus on analyzing the surrounding area of ​​the selected partial area and calculate (extract) a feature vector representing the image quality corresponding to the selected partial area.

[0065] In step S505, if the learning processing unit 203 has not executed the processing of steps S503 to S504 for all the partial regions that are the target of the learning processing and for which image quality correction is required (No), the process returns to step S503. On the other hand, if the processing of steps S503 to S504 has been executed (Yes), the image quality feature value is output to the image quality correction unit 305.

[0066] In step S506, the learning processing unit 203 uses the image quality feature output in the above-mentioned step S505 and the second image for learning 202, and performs correction processing to make the first image for learning 201 and / or the second image for learning 202 relatively close to the image quality through the same processing as the above-mentioned image quality correction unit 305.

[0067] In step S507 , the learning processing unit 203 evaluates the error function of the image quality correction model using the pair of the image corrected in step S505 and the uncorrected image in the first learning image 201 and the second learning image 202 .

[0068] Here, the image quality correction model 204 ( Figure 2 ) is a function for evaluating the difference between the image whose image quality is corrected in the above step S505 and the uncorrected image, and examples thereof include the absolute error and square error of pixel values.

[0069] In step S508, based on the evaluation results of step S507, the learning unit 203 calculates an update amount for the weight parameters of the image quality correction model 204 so as to reduce the value of the error function of the image quality correction model 204, and updates the parameters accordingly. This update is performed using, for example, stochastic gradient descent.

[0070] In step S509, the learning processing unit 203 determines whether the learning end condition has been met. If it is determined that the learning end condition has been met (Yes), the process proceeds to step S510, where the image quality correction model 204 is saved and the learning process ends. On the other hand, if it is determined that the learning end condition has not been met (No), the process returns to step S501 and the process from step S501 onward is repeated.

[0071] The conditions for ending learning include the following: the processing from step S501 to step S509 is repeated more than a predetermined number of times, or the value of the error function calculated in step S507 does not decrease even if the processing from step S501 to step S509 is repeated a predetermined number of times, and the learning of the image quality correction model 204 is judged to have converged.

[0072] Figure 6 FIG. 2 is a flowchart showing the processing actions of the inspection process of the image inspection apparatus 200 of this embodiment. Figure 6 As shown, when the inspection process starts, in step S601, the first image 206 as the inspection image, the second image 207 as the reference image, the predetermined abnormality size 205 and the image quality correction model 204 generated by the learning processing unit 203 are input into the estimation processing unit 208.

[0073] In step S602 , the region division processing unit 302 constituting the image quality feature extraction unit 301 of the estimation processing unit 208 divides the first image 206 into partial regions based on the predetermined abnormality size 205 .

[0074] In step S603 , the feature extraction processing unit 303 of the image quality feature extraction unit 301 selects one partial region from the divided partial regions.

[0075] In step S604 , the feature extraction processing section 303 of the image quality feature extraction section 301 analyzes the peripheral area of ​​the selected partial area with emphasis, and calculates (extracts) a feature vector representing the image quality corresponding to the selected partial area.

[0076] In step S605, if the image quality feature extraction unit 301 has not executed the processing of steps S603 to S604 for all of the partial regions that are the subject of inspection processing and for which image quality correction is required (No), the process returns to step S603. On the other hand, if the processing of steps S603 to S604 has been executed (Yes), the image quality feature 304 is output to the comparison processing unit 210.

[0077] In step S606, the image quality correction unit 305 constituting the estimation processing unit 208 uses the image quality feature 304 output in step S605 and the second image 207 to convert the image quality of the second image 207 so that the image quality approaches that of the first image 206. The estimation processing unit 208 outputs the corrected second image 209 to the comparison processing unit 210.

[0078] In step S607, the comparison processing unit 210 calculates the image difference 211 using the corrected second image 209 and the first image 206 output in step S606. In step S608, the comparison processing unit 210 outputs the image difference 211 to the abnormality determination unit 212 ( Figure 2 ).

[0079] In step S609, the abnormality determination unit 212 uses the difference 211 of the image output in step S608 to determine the abnormal area of ​​the first image 206 and / or the second image 207 through an analytical process including threshold processing for discovering abnormalities such as defects, outputs the inspection result, and ends the inspection process.

[0080] As described above, according to the present embodiment, it is possible to provide an image inspection apparatus and an image processing method that are robust against variations in image quality and can detect abnormalities such as defects with high accuracy.

[0081] Furthermore, according to this embodiment, the image quality feature 304 output by the image quality feature extraction unit 301 only represents the characteristics of the image spanning each partial region determined by the predetermined abnormality size 205, and does not include characteristics that represent abnormalities such as defects. Therefore, even when the image quality correction unit 305 uses the image quality feature 304 to correct the second image 207, it can only correct the differences caused by variations in image quality, without correcting the image that reproduces the defects included in the first image 206. As a result, when the comparison processing unit 210 compares the corrected second image 209 with the first image 206, it can only obtain the image differences caused by abnormalities such as defects, thereby reducing the possibility of false detection of abnormalities caused by variations in image quality.

[0082] Example 2

[0083] use Figure 7 、 Figure 8A as well as Figure 8B , an image inspection apparatus and an image processing method according to a second embodiment of the present invention will be described. Hereinafter, the same components as those in the first embodiment are denoted by the same reference numerals, and repeated descriptions will be omitted.

[0084] Figure 7 2 is a functional block diagram showing the configuration of the estimation processing unit 208 according to this embodiment.

[0085] like Figure 7 As shown, the image quality correction unit 305 of this embodiment includes a correction amount estimation processing unit 701 and a correction amount reflection processing unit 702. Furthermore, the correction amount estimation processing unit 701 and the correction amount reflection processing unit 702 are implemented by, for example, a processor (not shown) such as a CPU, a ROM storing various programs, a RAM temporarily storing data during calculations, and a storage device such as an external storage device. The CPU or other processor reads and executes the various programs stored in the ROM, and stores the calculation results obtained as a result of the execution in a cloud storage device via the RAM, an external storage device, or a network connection.

[0086] The correction amount estimation processing unit 701 estimates parameters corresponding to one or more image processing as image quality correction amounts for each factor causing variation in image quality.

[0087] The correction amount reflection processing unit 702 applies image processing using the image quality correction amount estimated by the correction amount estimation processing unit 701 to one of the first image 206 and the second image 207 to thereby perform correction to approximate the image quality of the other image.

[0088] Figure 8A FIG. 1 is a diagram illustrating the cause of the variation in image quality in this embodiment. Figure 8B is exemplified in Figure 8A FIG. 5 is a diagram of the image quality correction amount estimated in this embodiment when .

[0089] like Figure 8A As shown in FIG, the difference between the first image and the second image is caused by the deviation of the three image qualities of the line pattern position shift, image blur and brightness unevenness. In this case, the image quality correction amount is as follows: Figure 8B As shown, a pixel value correction amount for correcting brightness unevenness, a positional shift correction amount for correcting positional shift, and an image blur correction amount for correcting image blur are estimated.

[0090] The pixel value correction amount is the difference in pixel values ​​between the first image and the second image, which is generated across the partial area divided by the area segmentation processing unit 302. It is a parameter representing the brightness change of the image, such as brightness unevenness or contrast change generated partially or globally. As a form, an example is given of storing the correction value for each pixel in the form of an image with the same size as the image size of the second image. At this time, in order to output the corrected second image y, the correction amount reflection processing unit 702 uses the pixel value x of the second image and the pixel value correction amounts α and β to perform a linear mapping of y=αx+β. Figure 8B In the example of the pixel value correction value shown in the left figure of , in the case of "+0", α=1, β=0. In the case of "-30", α=1, β=-30.

[0091] The position shift correction amount represents the position shift of the pattern in the difference between the first image and the second image and is a parameter of the geometric transformation. Figure 8B In the central figure of , the position offset of the pixel value in the downward direction is illustrated, but it is not limited to this. For example, it is expressed in the form of a two-dimensional vector (dx, Sdy) representing the position offset of each of the horizontal and vertical directions, and the correction amount reflection processing unit 702 becomes a geometric transformation that moves the pixel value of the second image by the amount of (dx, Sdy) pixels. In addition, when correcting the translation offset and rotation offset generated in the entire image, the position offset correction amount is the parameter matrix of the affine transformation, and the correction amount reflection processing unit 702 is an affine transformation process. Figure 8B In the example shown in the center figure of , a correction of 5 pixels downward is shown.

[0092] The image blur correction amount represents the spread of pixel values ​​in the difference between the first image and the second image. The deviation in image quality caused by the spread of pixel values ​​is, for example, a phenomenon in which the edge of a pattern becomes unclear due to blurring or a white band becomes thicker or thinner. Figure 8B The right figure shows the kernel of a convolution filter for a 5×5 (5 rows and 5 columns) image as an example of an image blur correction amount. In this case, the correction amount estimation processing unit 701 estimates kernel elements k00 to k44 for each image or each partial region. The correction amount reflection processing unit 702 then performs convolution processing on each image or each partial region using the estimated kernel to correct for the spread of pixel values. As another example, if the convolution filter is assumed to be a Gaussian filter, the variance value, a parameter of the Gaussian filter, can also be used as the image blur correction amount.

[0093] As described above, according to the present embodiment, it is possible to select and correct the image quality to be corrected according to the deviation in image quality between the first image and the second image.

[0094] As an example of a situation in which this embodiment can be applied, there is a premise that image brightness fluctuations such as brightness unevenness do not occur, depending on the characteristics of the imaging device and the sample. For example, the occurrence of image brightness fluctuations is abnormal and therefore undesirable for correction. Similarly, if the occurrence of pattern positional shift is considered abnormal, the processing contents of the correction amount estimation processing unit 701 and the correction amount reflection processing unit 702 can be modified to correct image quality using an image quality correction amount other than the positional shift correction amount.

[0095] Example 3

[0096] use Figure 9 , describing the image inspection device and image processing method of this embodiment.

[0097] This embodiment differs from the first embodiment in that the second image 207 is one or more images having parameters of the probability distribution of the brightness values ​​of the captured image as pixel values, which are calculated based on the design data of the sample by simulation or machine learning such as neural network. Figure 3 ), the object of correction is the parameter of the probability distribution, and the corrected second image 209 becomes a conditional probability distribution with the image quality added as a condition.

[0098] like Figure 9 As shown, the image inspection apparatus 200 of this embodiment sets the inspection image I as the first image and the probability distribution image p(x|D) obtained from the design data D as the second image. Here, the probability distribution image p(x|D) is defined as each coordinate (i, Sj) of the image, and is expressed as one image for each parameter. Figure 9 In , the probability distribution is assumed to be a normal distribution with two parameters, the mean and the standard deviation, and is expressed as a mean image and a standard deviation image.

[0099] In this embodiment, the image quality correction unit 305 ( Figure 3 ) uses the inspection image I as input and is extracted by the image quality feature quantity extraction unit 301 ( Figure 3 ) outputs the image quality feature Q and probability distribution image p(x|D), and corrects the probability distribution to the conditional probability distribution p(x|D, SQ) that can represent the image quality of the inspection image.

[0100] The correction method may be the conversion using the neural network described in the first embodiment, or the method of calculating the correction amount described in the second embodiment for each of the average image and the standard deviation image.

[0101] In addition, the case of using a probability distribution image as the second image is illustrated, but when the design data D is obtained in the form of an image, the design data D can also be used as the second image. In this case, the corrected image quality includes the deviation of the pattern shape between the inspection image I and the design data D, the change in the range of brightness values, etc.

[0102] In this embodiment, when the corrected second image 209 is represented by the conditional probability distribution p(x|D, SQ), the comparison processing unit 210 ( Figure 2 ) uses the inspection image I and the conditional probability distribution p(x|D, SQ) to calculate the image difference 211. Figure 9 As shown in the example, if the probability distribution is assumed to be a normal distribution, the average μ and standard deviation σ corresponding to the pixel value x at the coordinate (i, Sj) can be used to set the abnormality represented by |x-μ| / σ as the image difference 211. Figure 2 ) A method can be cited in which pixels having an abnormality exceeding a specified threshold are regarded as defects.

[0103] Furthermore, as step S507 ( Figure 5 ) can be set as the negative log-likelihood of the conditional probability distribution p(x|D, SQ) of the corrected second image relative to the first image, i.e., the inspection image I.

[0104] As described above, according to this embodiment, the corrected second image, used as the reference image during comparison processing, forms a conditional probability distribution that conditions the image quality of the first image, used as the inspection image. This allows variations in the pattern due to manufacturing tolerances contained in the first image to be evaluated as variations in pixel values ​​due to imaging noise, as deviation components of a probability distribution similar to the standard deviation of a normal distribution. This results in image inspection that is robust to noise.

[0105] The present invention is not limited to the above-described embodiments and includes various variations. For example, the above-described embodiments are examples described in detail to facilitate understanding of the present invention and are not necessarily limited to having all the described structures. Furthermore, a portion of the structure of one embodiment may be replaced with the structure of another embodiment, and the structure of another embodiment may be added to the structure of one embodiment.

[0106] Explanation of symbols

[0107] 200…Image inspection device, 201…First image for learning, 202…Second image for learning, 203…Learning processing unit, 204…Image quality correction model, 205…Predetermined abnormality size, 206…First image, 207…Second image, 208…Estimation processing unit, 209…Corrected second image, 210…Comparison processing unit, 211…Image difference, 212…Abnormality determination unit, 301…Image quality feature extraction unit, 302…Region segmentation processing unit, 303…Feature extraction processing unit, 304…Image quality feature, 305…Image quality correction unit, 701…Correction amount estimation processing unit, 702…Correction amount reflection processing unit.

Claims

1. An image inspection device for comparing a first image with a second image, obtaining a difference between the images, and detecting an abnormality that may exist in either the first image or the second image, characterized in that: The image inspection device comprises: an image quality feature extraction unit for extracting, for each partial region of the first image, a feature vector representing image quality, namely, an image quality feature; an image quality correction unit for making the image quality of the first image and / or the second image relatively close for each partial region based on the feature vector obtained by the image quality feature extraction unit and the second image; as well as a comparison processing unit that compares any one image corrected by the image quality correction unit with another image, or the corrected first image with the second image to obtain an image difference; The image quality feature extraction unit includes: a region division processing unit that divides the first image into partial regions having a unit size larger than a predetermined abnormality size; and The feature extraction processing unit extracts feature vectors representing image quality of each partial region from other partial regions of the first image.

2. The image inspection device according to claim 1, wherein The image inspection device includes: a learning processing unit that uses a first learning image and a second learning image corresponding to the first learning image to generate a model for converting the image quality of the first image and / or the second image so that the image quality is relatively close for each partial area; The image quality feature extraction section and the image quality correction section use the model generated by the learning processing section.

3. The image inspection device according to claim 2, wherein The learning processing unit corrects the image quality of the first learning image and / or the second learning image so that the image quality is relatively close in each partial area, and compares the corrected image with another image, or the corrected first image with the second image, to learn the model in a manner that reduces their difference.

4. The image inspection device according to claim 1, wherein The feature extraction processing unit includes a pre-processing unit that extracts a feature vector by weighting the pixel values ​​of the partial area and the surrounding partial areas in a manner that gives more importance to the pixel values ​​of the partial area and the surrounding partial areas than to the feature vector representing the image quality of the partial area.

5. The image inspection device according to claim 2, wherein The area division processing unit is a block embedding process that collects information of the surrounding area in one pixel. The feature extraction processing unit is a process using a neural network having an area not to be extracted as a feature in a receptive field.

6. The image inspection device according to claim 1, wherein The second image is one or more images having parameters of a probability distribution of brightness values ​​of a captured image as pixel values, and The object of correction by the image quality correction unit is the parameters of the probability distribution, and the corrected second image becomes a conditional probability distribution conditioned by the image quality.

7. The image inspection device according to claim 1, wherein The image quality correction unit includes: a correction amount estimation processing unit that estimates parameters corresponding to one or more image processing as image quality correction amounts for each cause of image quality deviation; and The correction amount reflection processing unit applies image processing using the image quality correction amount estimated by the correction amount estimation processing unit to the first image and / or the second image, thereby performing correction to make the image quality relatively close for each partial area.

8. The image inspection device according to claim 7, wherein: One of the image quality correction amounts is a vector or parameter representing a positional offset between patterns of the first image and the second image including image distortion. The correction amount reflection processing unit includes processing for aligning a position of a pattern in one of the first image and the second image with a position of a pattern in the other image based on the image quality correction amount.

9. The image inspection device according to claim 7, wherein: One of the image quality correction amounts is a value representing a difference in pixel values ​​between the first image and the second image including brightness fluctuations, which difference has an influence on a range larger than a predetermined abnormal size. The correction amount reflection processing unit changes the pixel value of one of the first image and the second image according to the image quality correction amount, thereby reducing the difference in pixel value between the first image and the other image.

10. The image inspection device according to claim 7, wherein One of the image quality correction amounts is a parameter of an image processing filter representing a difference in spread between pixel values ​​of the first image and the second image including image blur. The image inspection apparatus includes a process in which the correction amount reflection processing unit applies an image processing filter to one of the first image and the second image based on the image quality correction amount, thereby reducing a difference in the degree of spread of pixel values.

11. An image processing method comprising comparing a first image and a second image to obtain a difference between the images, and processing the images to detect an abnormality that may exist in either the first image or the second image, wherein: The image quality feature extraction unit extracts a feature vector representing image quality, that is, an image quality feature, for each partial region of the first image. The image quality correction unit makes the image quality of the first image and / or the second image relatively close to that of each partial region based on the feature vector and the second image. The comparison processing unit compares one image corrected by the image quality correction unit with another image, or the corrected first image with the second image to obtain a difference between the images. The region division processing unit divides the first image into partial regions of a unit larger than a predetermined abnormal size. The feature extraction processing unit extracts feature vectors representing image quality of each partial region from other partial regions of the first image.

12. The image processing method according to claim 11, wherein: The learning processing unit generates a model for converting the image quality of the first image and / or the second image so that the image quality is relatively close for each partial area, using the first learning image and the second learning image corresponding to the first learning image. The image quality feature extraction section and the image quality correction section use the model generated by the learning processing section.

13. The image processing method according to claim 12, wherein: The learning processing unit corrects the image quality of the first learning image and / or the second learning image so that the image quality is relatively close in each partial area, and compares the corrected image with another image, or the corrected first image with the second image, to learn the model in a manner that reduces their difference.

14. The image processing method according to claim 11, wherein: The second image is one or more images having parameters of a probability distribution of brightness values ​​of a captured image as pixel values, and In the image quality correction unit, the object of correction is the parameter of the probability distribution, and the second image after correction becomes a conditional probability distribution with the image quality added as a condition.

15. The image processing method according to claim 11, wherein: The correction amount estimation processing unit estimates parameters corresponding to one or more image processing as image quality correction amounts for each cause of image quality deviation. The correction amount reflection processing unit applies image processing using the image quality correction amount estimated by the correction amount estimation processing unit to the first image and / or the second image, thereby performing correction to make the image quality relatively close for each partial area.

Images