Optical defect detection method, device and equipment and storage medium

Abstract

The invention provides an optical defect detection method, apparatus and device, and a storage medium. The method comprises the steps of obtaining a target image of a to-be-detected target and a reference image of at least one reference target adjacent to the to-be-detected target; performing differential processing on the target image and the reference image to obtain a differential image; determining a defect region of interest of the difference image; identifying and filtering a pseudo-defect region in the defect region of interest based on a deep learning network model to obtain an updated defect region of interest; the pseudo defect area is an area which does not need to be identified as a defect; and identifying the defect of the to-be-detected target image based on the updated defect region of interest. By adopting the scheme, the defect detection accuracy and the detection efficiency are improved, and the detection cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of semiconductor detection technology, and in particular to an optical defect detection method, device, equipment and storage medium. Background Art

[0002] In the chip (die) manufacturing process, defect detection of chips is an essential step, which can discover various defects in the chip production process, thereby improving the reliability and performance of the product. Among them, optical detection is the most commonly used detection method. The existing optical detection technology compares the image of the chip to be detected with the standard image (default is no defects), constructs a differential image, and obtains the defect area of ​​interest of the differential image by setting the threshold, so as to perform defect detection. Although this detection method is efficient, its shortcomings are also obvious. For example, under the influence of various imaging reasons such as ambient light, it may cause false detection, and detect images that are not defects as defects. At this time, manual review is required. The review operation has high labor costs and low efficiency.

[0003] Therefore, it is necessary to provide a more efficient and accurate chip defect detection technology to solve the above-mentioned problems. Summary of the invention

[0004] The present application provides an optical defect detection method, device, equipment and storage medium to solve at least one of the technical problems of high false detection rate and high manual review cost in the existing optical defect detection technology.

[0005] According to a first aspect of the present application, there is provided an optical defect detection method, the method comprising:

[0006] Acquire a target image of a target to be detected and a reference image of at least one reference target adjacent to the target to be detected;

[0007] Performing differential processing on the target image and the reference image to obtain a differential image;

[0008] determining a defect region of interest of the differential image;

[0009] Based on the deep learning network model, the pseudo-defect region in the defect region of interest is identified and filtered out to obtain an updated defect region of interest; the pseudo-defect region is a region that does not need to be identified as a defect;

[0010] Defects of the target image to be inspected are identified based on the updated defect region of interest.

[0011] In some embodiments, when the reference target is greater than one, the method further comprises:

[0012] Compute the intersection with the reference image;

[0013] Selecting the area corresponding to the intersection in each reference image as the intersection reference image;

[0014] The step of performing differential processing on the target image and the reference image to obtain a differential image comprises:

[0015] The target image is subjected to difference processing with the intersection reference image respectively to obtain a difference image.

[0016] In some embodiments, when the reference target is greater than one, determining the defect region of interest of the differential image includes:

[0017] Calculating the intersection of the difference images to obtain a difference intersection image;

[0018] Calculating the mean and variance of the difference intersection image;

[0019] Determining a filtering threshold based on the mean and the variance;

[0020] The difference intersection image is binarized and segmented based on the filtering threshold to obtain a defect region of interest of the difference intersection image.

[0021] In some embodiments, the method further comprises: performing differential processing on a reference image of the reference target to obtain a reference differential image;

[0022] The calculating the intersection of the difference images comprises:

[0023] An intersection of the difference image and the reference difference image is calculated.

[0024] In some embodiments, the step of identifying and filtering out pseudo-defect regions in the defect region of interest based on a deep learning network model to obtain an updated defect region of interest comprises:

[0025] Determine whether there is a suspected pseudo-defect region in the defect region of interest; wherein, an area in the defect region of interest that satisfies a specified adjacent relationship with a main connected area of ​​a target graphic in a target image and occupies at least two quadrants of four quadrants is identified as a suspected pseudo-defect region; wherein the four quadrants are quadrants divided by taking the center of the main connected area of ​​the target graphic in the target image as the coordinate origin;

[0026] If so, regenerate the sub-region image of the reference image based on the trained deep learning network model using the target image and the reference image, so that the size of the target figure in the regenerated sub-region image of the reference image is consistent with the size of the target figure in the target image;

[0027] Performing differential processing on the target image and the sub-region image of the regenerated reference image to obtain an updated differential image;

[0028] An updated defect region of interest is obtained based on the updated differential image.

[0029] According to a second aspect of the present application, an optical defect detection device is provided, the device comprising:

[0030] An acquisition unit, used to acquire a target image of a target to be detected and a reference image of at least one reference target adjacent to the target to be detected;

[0031] A differential processing unit, used for performing differential processing on the target image and the reference image to obtain a differential image;

[0032] A defect region of interest determining unit, used to determine the defect region of interest of the differential image;

[0033] A pseudo-defect recognition and filtering unit, used to identify and filter out pseudo-defect regions in the defect region of interest based on a deep learning network model to obtain an updated defect region of interest; the pseudo-defect region is a region that does not need to be identified as a defect;

[0034] The defect detection unit is used to identify defects of the target image to be detected based on the updated defect region of interest.

[0035] In some embodiments, the pseudo-defect identification and filtering unit further comprises:

[0036] A judgment subunit is used to judge whether there is a suspected pseudo-defect area in the defect region of interest; wherein, an area in the defect region of interest that satisfies a specified adjacent relationship with a main connected area of ​​a target figure in a target image and occupies at least two quadrants of four quadrants is identified as a suspected pseudo-defect area; wherein the four quadrants are quadrants divided by taking the center of the main connected area of ​​the target figure in the target image as the coordinate origin;

[0037] A filtering subunit, configured to, when the judgment result of the judgment unit is that there is a suspected pseudo-defect area, regenerate a sub-region image of the reference image based on the trained deep learning network model using the target image and the reference image, so that the size of the target figure in the sub-region image of the regenerated reference image is consistent with the size of the target figure in the target image;

[0038] A differential subunit, used for performing differential processing on the target image and the sub-region image of the regenerated reference image to obtain an updated differential image;

[0039] The defect region of interest updating unit is used to obtain an updated defect region of interest based on the updated differential image.

[0040] According to a third aspect of the present application, a computer program product is provided, wherein the computer program product comprises a computer program, wherein when the computer program is executed by a processor, the steps of the method described in any embodiment of the present application are implemented.

[0041] According to a fourth aspect of the present application, a computer device is provided, comprising: a processor and a memory storing a computer program; when the processor executes the computer program, the steps of the method described in any embodiment of the present application are implemented.

[0042] According to a fifth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any embodiment of the present application are implemented.

[0043] In summary, the optical defect detection method, device, equipment and storage medium provided by the present application have at least the following features:

[0044] Beneficial effects:

[0045] The optical defect detection method described in the embodiment of the present application can perform differential processing on the target image of the target to be detected and at least one reference image to obtain a differential image. It does not rely on a single reference image, and will not cause false detection due to defects in a single reference image, which can reduce the high false detection rate caused by this. In addition, after obtaining the defect region of interest based on the differential image, the pseudo-defect region in the defect region of interest can be automatically identified and filtered out based on the deep learning network model, without the need for manual review, thereby reducing the cost of manual review. The method of filtering pseudo-defect regions based on the deep learning network model is more accurate and efficient than the method of filtering using a fixed threshold. Therefore, the present application improves the accuracy and efficiency of defect detection and reduces the detection cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present application, the drawings required for use in the specific implementation modes will be briefly introduced below in conjunction with the accompanying drawings. It is obvious that the drawings described below are some implementation modes of the present application. For those skilled in the art, other drawings or solutions can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of an optical defect detection method provided according to an embodiment of the present application;

[0048] FIG. 2( a ) is an example of a target image test provided by an embodiment of the present application;

[0049] FIG2( b ) is an example of a reference image ref1 provided by an embodiment of the present application;

[0050] FIG2( c ) is an example of a reference image ref2 provided by an embodiment of the present application;

[0051] FIG3( a ) is an example of a differential image D1 between test and ref1 provided in one embodiment of the present application;

[0052] FIG3( b ) is an example of a difference image D2 between test and ref2 provided in one embodiment of the present application;

[0053] Figure 4 A flow chart of a method for determining a defect region of interest of a differential image provided in an embodiment of the present application;

[0054] FIG5( a ) is an example of a target image test provided by another embodiment of the present application;

[0055] FIG5( b ) is an example of a reference image ref1 provided in another embodiment of the present application;

[0056] FIG5( c ) is an example of a reference image ref2 provided by another embodiment of the present application;

[0057] FIG6( a ) is an example of a differential image between test and ref1 provided in another embodiment of the present application;

[0058] FIG6( b ) is an example of a differential image of test and ref2 provided in another embodiment of the present application;

[0059] Figure 7 A flow chart of a method for identifying and filtering out pseudo-defect areas based on a deep learning network model provided in an embodiment of the present application;

[0060] Figure 8 A schematic diagram of the relationship between the suspected pseudo-defect area and the main connected area of ​​the target graphic provided in the embodiment of the present application;

[0061] Fig. 9 A schematic diagram of regenerating a reference image based on a deep learning network model provided in an embodiment of the present application;

[0062] Fig.10 A schematic diagram of the structure of an optical defect detection device provided in one embodiment of the present application;

[0063] Fig.11 A schematic diagram of a computer program product in one embodiment of the present application;

[0064] Fig.12 A structural diagram of a computer device in one embodiment of the present application;

[0065] Fig.13 This is a structural diagram of a computer-readable storage medium in one embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the above and other features and advantages of the present application more clear, the present application is further described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.

[0067] In the following description, many specific details are set forth to provide a thorough understanding of the present application. However, it is apparent to those skilled in the art that specific details need not be adopted to practice the present application. In other cases, well-known steps or operations are not described in detail to avoid blurring the present application.

[0068] The inventors found in the research process that some existing technologies use optical detection methods to detect defects in chips by comparing the image of the chip to be detected with the standard image, constructing a differential image, and obtaining the defect region of interest of the differential image by setting a fixed threshold, thereby performing defect detection. This defect detection method relies on the standard image. If the standard image has defects, it will cause systematic deviations in the detection results, and there will be problems such as inaccurate defect detection results due to image alignment errors.

[0069] To overcome the above problems, the present application adopts a method of comparing the chip (die) to be detected with at least one adjacent die (a preferred embodiment is two adjacent dies), which does not rely on a single reference image and will not cause false detection due to defects in a single reference image.

[0070] In addition, after determining the defect region of interest, the present application further identifies and filters out pseudo-defect regions (regions that do not need to be identified as defects) in the defect region of interest according to the method of the present application, and performs defect detection based on the filtered defect region of interest. This method can effectively reduce the false detection rate and does not require manual review, thereby reducing the cost of manual review while ensuring the accuracy and efficiency of defect detection.

[0071] The optical defect detection method provided in the embodiment of the present application can be executed by the optical defect detection device provided in the embodiment of the present application, and the device can be configured in an electronic device.

[0072] refer to Figure 1 A flowchart of an optical defect detection method provided in an embodiment of the present application, the method comprising the following steps:

[0073] S10, acquiring a target image of a target to be detected and a reference image of at least one reference target adjacent to the target to be detected;

[0074] S11, performing difference processing on the target image and the reference image to obtain a difference image;

[0075] S12, determining a defect region of interest of the differential image;

[0076] S13, identifying and filtering out pseudo-defect regions in the defect region of interest based on a deep learning network model to obtain an updated defect region of interest; the pseudo-defect region is a region that does not need to be identified as a defect;

[0077] S14: Identify defects of the target image to be detected based on the updated defect region of interest.

[0078] The optical defect detection method described in the embodiment of the present application can perform differential processing on the target image of the target to be detected and at least one reference image to obtain a differential image. It does not rely on a single reference image and will not cause false detection due to defects in a single reference image, which can reduce the problem of high false detection rate caused by this. After obtaining the defect region of interest based on the differential image, the pseudo-defect area in the defect region of interest can be automatically identified and filtered out based on the deep learning network model, without the need for manual review, thereby reducing the cost of manual review. And the method of filtering pseudo-defect areas based on the deep learning network model is more accurate and efficient than the method of filtering with a fixed threshold. Therefore, the present application improves the accuracy and efficiency of defect detection and reduces the detection cost.

[0079] In one embodiment, the target to be detected described in S10 may be a chip, and the chip may be one of the multiple units into which the wafer is divided. After the wafer is divided into multiple units and made into chips, each qualified chip should be consistent in theory. In order to detect whether each chip has defects, an optical device is used to collect images of each chip, and the collected images are processed accordingly to obtain a target image of the target to be detected (chip). The process of processing the collected images is not specifically limited in the embodiment of the present application, and can be implemented using existing technologies.

[0080] An embodiment of the present application simultaneously obtains a reference image of at least one reference target adjacent to the target to be detected while obtaining a target image of the target to be detected, wherein a preferred embodiment obtains reference images of two reference targets adjacent to the target to be detected. The following embodiments of the present application are explained by taking the acquisition of reference images of two reference targets as an example. For the convenience of description, the target image of the target to be detected in the embodiment of the present application is referred to as test, and an image example is shown in Figure 2(a). The reference images corresponding to the two adjacent reference targets are referred to as ref1 and ref2, and the reference image examples are shown in Figures 2(b) and 2(c), respectively.

[0081] In one embodiment, when the reference target is greater than one, the method further includes: calculating the intersection of the reference images; selecting the area corresponding to the intersection in each reference image as the intersection reference image. Then the step of performing differential processing on the target image and the reference image in S11 to obtain the differential image includes: performing differential processing on the target image and the intersection reference image respectively to obtain the differential image. As shown in FIG2(b) and FIG2(c), there are two reference images ref1 and ref2. As can be seen from the figure, the lower left corner of ref2 is a cross-connected area which is not displayed in ref1, so this part of the area does not belong to the intersection of the reference images. The selection of the area corresponding to the intersection in each reference image as the intersection reference image is to remove the image after the difference area in each reference image is removed as the intersection reference image. That is, for ref2, when performing differential processing with test, the lower left corner is a cross-connected area which does not participate in the differential processing.

[0082] The embodiment of the present application obtains a differential image by performing a differential calculation between the intersection reference image and the target image, thereby avoiding the problem of inaccurate inspection results caused by defects in a single reference image.

[0083] It is understandable that the embodiment of the present application can also first perform differential processing on the target image and each reference image respectively, obtain two differential images, and then calculate the intersection of the two differential images as the final differential image, which can also avoid the false detection problem caused by the defects of a single reference image.

[0084] Since the target graphic (commonly known as pattern) area in the test image in the dark field image must be a connected area with regular shapes, in order to improve the detection efficiency, when performing differential processing, differential calculation is performed according to the geometric shape, and pixel-by-pixel calculation is not required, which can speed up the calculation of the differential image. The binary segmentation method can be used to obtain the geometric shape in the target image, and then the differential calculation is further performed based on the geometric shape.

[0085] In one embodiment, the difference processing can be combined with the intersection-and-union ratio in the geometric calculation method and the Relu activation function in the neural network function to obtain a difference image. Among them, the value taking method of the Relu activation function may include: difference modulus method, direct difference method, difference calculation and normalization processing, etc. As shown in Figure 3(a), the difference image D1 after test and ref1 perform ReLu, and as shown in Figure 3(b), the difference image D2 after test and ref2 perform ReLu. It can be seen from the figure that D2 does not contain the cross-connected area in the lower left corner of the ref2 image.

[0086] It can be understood that in the embodiment of the present application, when performing differential processing on the target image and the reference image to obtain the differential image, only pixels with a difference value greater than 0 are retained, and all pixels with a difference value less than 0 are set to 0. Non-zero pixel points indicate inconsistent points in the two differential images, which may be defective points.

[0087] Dark field optical inspection image equipment generally detects defects as brighter areas in the image. Ideally, the contents of the three images (test, ref1, ref2) should be exactly the same. The areas where the test image is brighter than the two differential images of ref1 and ref2 are the test map containing the defect area of ​​interest DOI. And the target graphic (commonly known as pattern) area in the test image must be a connected area with a regular shape, such as the cross-connected area shown in Figure 2 (a). Therefore, only the connected areas where the test image is brighter than ref1 and ref2 can be counted.

[0088] Based on this idea, the method flow chart of determining the defect region of interest of the differential image in step S12 is as follows: Figure 4 As shown, the following steps are included:

[0089] S121. Calculate the intersection of the difference images to obtain a difference intersection image.

[0090] As described above, by performing a difference between the target image and each intersection reference image, the same number of difference images as the reference target can be obtained. In the case where the reference image includes two images, ref1 and ref2, two difference images D1 and D2 can be obtained. The calculation of the intersection of the difference images in this embodiment is to calculate the intersection of all the difference images to obtain a difference intersection image.

[0091] In one embodiment, when the reference target is greater than one, the reference image may be subjected to differential processing to obtain a reference differential image. For example, the reference image includes two reference images, ref1 and ref2, and the two reference images are subjected to differential processing to obtain a reference differential image D3. In this case, calculating the intersection of the differential images to obtain the differential intersection image includes calculating the intersection of all differential images and the reference differential image, that is, calculating the intersection of D1, D2, and D3.

[0092] S122, calculating the mean and variance of the difference intersection image.

[0093] The method for calculating the mean and variance of the difference intersection image can be implemented using existing technology, and this application does not impose any specific restrictions on this.

[0094] S123, determining a filtering threshold based on the mean and variance;

[0095] In one implementation manner, the method for determining the filtering threshold based on the mean and variance can be implemented using the following formula:

[0096] threshold=mean+k*std

[0097] Among them, threshold represents the filtering threshold, mean represents the mean, std represents the variance, and k is the empirical value.

[0098] Considering that the function of the filtering threshold is a first-order linear function of the mean and the variance, other multi-order functions may also be used to calculate the filtering threshold.

[0099] S124 . Binarize and segment the difference intersection image based on the filtering threshold to obtain a defect region of interest of the difference intersection image.

[0100] Specifically, performing binary segmentation on the difference intersection image based on the filtering threshold may include: for each pixel in the difference intersection image, the area consisting of the pixels whose grayscale value is greater than the corresponding pixel of the filtering threshold is taken as the defect region of interest. It is understandable that the grayscale value may be a value processed by various algorithms.

[0101] The embodiment of the present application calculates an adaptive filtering threshold based on the mean and variance of the differential intersection image, and performs binary segmentation on the differential intersection image based on the filtering threshold, so as to obtain a more accurate defect region of interest. Compared with the method of using a fixed threshold for binary segmentation, the present application can obtain a filtering threshold that better matches the current image (test, ref1, ref2), thereby further improving the accuracy of the binary segmentation and effectively ensuring the recognition accuracy of the defect region of interest.

[0102] In some scenarios, the size (width or height) of the pattern in test, ref1, and ref2 may be inconsistent due to imaging and other reasons, such as Figure 5(a) , 5(b) As shown in Figures 5(a) and 5(c), they are examples of ref1, test and ref2 images respectively. For the above image examples, the corresponding two differential images are shown in Figures 6(a) and 6(b). The lower left box in Figures 6(a) and 6(b) is the real defect area, that is, the true defect, and the area formed due to the difference in pattern size in the differential image is called a pseudo defect area in this embodiment of the application, which is not a defect to be identified, such as the false nuisance shown in Figures 6(a) and 6(b). If the pseudo defect area is identified as a defect, the probability of false detection will increase. Therefore, after obtaining the defect area of ​​interest, the embodiment of the application automatically identifies the pseudo defect area therein and filters it out.

[0103] It is understandable that the parameters required for the corresponding optical detection process can be set in the pre-scanning process, including setting a method for identifying a pseudo-defect area and a filtering algorithm adopted after the pseudo-defect area is identified.

[0104] In one embodiment, the method flow of step S13 for identifying and filtering out pseudo defect regions in the defect region of interest based on the deep learning network model to obtain an updated defect region of interest is as follows: Figure 7 As shown, the method includes:

[0105] S131, determining whether there is a suspected pseudo-defect region in the defect region of interest.

[0106] In one embodiment of the present application, the area in the defect region of interest that satisfies a specified adjacent relationship with the main connected area of ​​the target pattern in the target image and occupies at least two of the four quadrants is identified as a suspected pseudo-defect area; wherein the four quadrants are quadrants divided by taking the center of the main connected area of ​​the target pattern in the target image as the coordinate origin. Figure 8The white cross connected area shown in is the pattern in the target image, and the center of the pattern area in the target image is used as the coordinate origin to divide the four quadrants, namely the 1st quadrant, the 2nd quadrant, the 3rd quadrant and the 4th quadrant. The main connected area of ​​the target pattern in the target image satisfies the specified adjacent relationship, including: using a specified adjacent relationship analysis method to determine the area adjacent to the main connected area of ​​the pattern in the target image, the specified adjacent relationship analysis method includes but is not limited to: a four-neighborhood method or an eight-neighborhood method, or analyzing whether the distance between the centroid of the minimum circumscribed rectangle of the main connected area of ​​the pattern in the target image and the centroid of the connected area to be identified is less than a specified distance threshold, or whether the inclination angles of the long and short sides of the main connected area of ​​the pattern in the target image are the same as the inclination angles of the long and short sides of the connected area to be identified. The pseudo-defect area identified by the pseudo-defect area identification method can ensure that the defect interest area is identified because of the difference in size of the main connected area of ​​the pattern of the target image and the reference image, that is, to ensure that the identified pseudo-defect area is a defect area that does not really need to be concerned about, thereby reducing the false detection rate.

[0107] like Figure 8 The white cross-connected area is the pattern in the target image, and the shaded area around it is adjacent to the pattern in the target image and occupies four quadrants. Therefore, the method of the present application will identify the shaded area as a pseudo-defect area. However, the defect area shown in the lower left box diagram in Figures 6(a) and 6(b) does not meet the above-mentioned pseudo-defect area identification conditions. It is not adjacent to the pattern in the target image and only occupies one quadrant. Therefore, the defect area will not be identified as a pseudo-defect area. Since the pseudo-defect area is an area that does not need to be paid attention to, after the pseudo-defect area is identified, it is necessary to filter out the pseudo-defect area from the determined defect area of ​​interest.

[0108] S132. If there is a suspected pseudo-defect area, the sub-area image of the reference image is regenerated using the target image and the reference image based on the trained deep learning network model, so that the size of the target graphic in the sub-area image of the regenerated reference image is consistent with the size of the target graphic in the target image.

[0109] For the trained deep learning network model, the reference images ref1, ref2 and the target image test are input in sequence, and the reference images ref1 and ref2 having the same size as the target image test are output. That is, the embodiment of the present application utilizes the trained deep learning network model to regenerate the reference image, thereby overcoming the problem of false detection due to the inconsistency of the pattern size between the target image and the reference image.

[0110] The schematic diagram of the sub-region image of the reference image regenerated based on the deep learning network model is as follows Fig. 9 As shown in the figure, the left side is the sub-region image of the original reference image (original patch image), and the right side is the sub-region image of the reference image generated based on the deep learning network model (generated patch image). It can be seen from the figure that the pattern sizes in the patch images ref1 and ref2 regenerated based on the deep learning network model are basically consistent with the pattern size in test.

[0111] It can be understood that the size of the target graphic in the reference image described in the embodiment of the present application is consistent with the size of the target graphic in the target image, including the case where the error in the size between the target graphics is less than a set threshold.

[0112] S133: Perform differential processing on the target image and the sub-region image of the regenerated reference image to obtain an updated differential image.

[0113] The specific method of performing differential processing on the target image and the sub-region image of the regenerated reference image to obtain an updated differential image is the same as that described in the above embodiment and will not be repeated here.

[0114] The specific method for obtaining the updated defect region of interest is the same as described in the above embodiment.

[0115] S134: Obtain an updated defect region of interest based on the updated differential image.

[0116] If the original suspected pseudo-defect area still exists in the updated differential image, it means that it is not a defect caused by inconsistent pattern size, but a real defect; if the suspected pseudo-defect area does not exist, it means that it is a defect caused by inconsistent pattern size, which is a pseudo-defect and has been filtered out through the above steps S132 and S133.

[0117] That is, the embodiment of the present application obtains the final defect region of interest by regenerating the pattern in the reference image and re-performing the image difference processing and the process of identifying the defect region of interest to filter out the pseudo defect region of interest.

[0118] Since the finally obtained updated defect region of interest does not include a pseudo-defect region, defect detection based on the updated defect region of interest will effectively improve the accuracy of defect detection and reduce the false detection rate.

[0119] In one implementation manner, the identifying the defect of the target image to be detected based on the updated defect region of interest in step S14 may be identifying the updated defect region of interest as the defect of the target image to be detected.

[0120] The present application also provides an optical defect detection device, such as Fig.10 The schematic diagram of the structure of the device is shown, and the device comprises:

[0121] An acquisition unit 100 is used to acquire a target image of a target to be detected and a reference image of at least one reference target adjacent to the target to be detected;

[0122] The differential processing unit 101 is used to perform differential processing on the target image and the reference image to obtain a differential image;

[0123] A defect region of interest determining unit 102, configured to determine a defect region of interest of the differential image;

[0124] The pseudo-defect recognition and filtering unit 103 is used to identify and filter out pseudo-defect regions in the defect region of interest based on a deep learning network model to obtain an updated defect region of interest; the pseudo-defect region is a region that does not need to be identified as a defect;

[0125] The defect detection unit 104 is configured to identify defects of the target image to be detected based on the updated defect region of interest.

[0126] In some embodiments, when the reference target is greater than one, the apparatus further comprises:

[0127] The reference image intersection calculation unit 105 is used to calculate the intersection of the reference images and select the area corresponding to the intersection in each reference image as the intersection reference image.

[0128] The difference processing unit 101 is further used to perform difference processing on the target image and the intersection reference image respectively to obtain a difference image.

[0129] In some implementations, when the reference target is greater than one, the defect region of interest determination unit 102 further includes:

[0130] The first subunit 1021 calculates the intersection of the difference images to obtain a difference intersection image;

[0131] The second subunit 1022 calculates the mean and variance of the difference intersection image;

[0132] The third subunit 1023 determines a filtering threshold based on the mean and variance;

[0133] The fourth subunit 1024 performs binary segmentation on the difference intersection image based on the filtering threshold to obtain a defect region of interest of the difference intersection image.

[0134] In some embodiments, the differential processing unit 101 is further used to: perform differential processing on a reference image of a reference target to obtain a reference differential image; and the first subunit 1021 is further used to: calculate the intersection of the differential image and the reference differential image.

[0135] In some implementations, the defect region of interest determining unit 102 is further configured to:

[0136] An area in the defect region of interest that satisfies a specified adjacent relationship with the main connected area of ​​the target graphic in the target image and occupies at least two of the four quadrants is identified as a pseudo-defect area; wherein the four quadrants are quadrants divided with the center of the main connected area of ​​the target graphic in the target image as the coordinate origin.

[0137] In some embodiments, the pseudo-defect identification and filtering unit 103 further includes:

[0138] The judging subunit 1031 is used to judge whether there is a suspected pseudo-defect region in the defect region of interest; wherein, the region in the defect region of interest that satisfies a specified adjacent relationship with the main connected region of the target figure in the target image and occupies at least two quadrants of the four quadrants is identified as the suspected pseudo-defect region; wherein the four quadrants are quadrants divided by taking the center of the main connected region of the target figure in the target image as the coordinate origin;

[0139] The filtering subunit 1032 is used to regenerate a sub-region image of the reference image based on the trained deep learning network model using the target image and the reference image when the judgment result of the judgment unit 1031 is that there is a suspected pseudo-defect area, so that the size of the target figure in the sub-region image of the regenerated reference image is consistent with the size of the target figure in the target image;

[0140] A differential subunit 1033, configured to perform differential processing on the target image and the sub-region image of the regenerated reference image to obtain an updated differential image;

[0141] The defect region of interest updating unit 1034 is configured to obtain an updated defect region of interest based on the updated differential image.

[0142] In summary, the optical defect detection device described in the embodiment of the present application can perform differential processing on the target image of the target to be detected and at least one reference image to obtain a differential image, which is independent of a single reference image and will not cause false detection due to defects in a single reference image, thereby reducing the problem of high false detection rate caused by this. In addition, after obtaining the defect region of interest based on the differential image, the pseudo-defect region in the defect region of interest can be automatically identified and filtered out based on the deep learning network model, without the need for manual review, thereby reducing the cost of manual review, and the method of filtering pseudo-defect regions based on the deep learning network model is more accurate and efficient than the method of filtering using a fixed threshold, so the present application improves the accuracy and efficiency of defect detection and reduces the detection cost.

[0143] It should be understood that the specific features, operations and details described hereinabove with respect to the method of the present application may also be similarly applied to the device of the present application, or vice versa. In addition, each step of the method of the present application described above may be performed by the corresponding components or units of the device of the present application.

[0144] It should be understood that each module / unit of the device of the present application can be implemented in whole or in part by software, hardware, firmware or a combination thereof. Each module / unit can be embedded in the processor of the electronic device or independent of the processor in the form of hardware or firmware, or can be stored in the memory of the electronic device in the form of software for the processor to call to perform the operation of each module / unit. Each module / unit can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.

[0145] like Fig.11 As shown, an embodiment of the present application provides a computer program product 400, which includes a computer program, wherein when the computer program is executed by a processor, the steps of the optical defect detection method in the above embodiments are implemented.

[0146] like Fig.12 As shown, an embodiment of the present application provides a computer device 500, which includes a processor 501 and a memory 502 storing a computer program. When the processor 501 executes the computer program, each step of the above-mentioned optical defect detection method is implemented. The computer device 500 can be generally a server, a terminal, or any other computer device with necessary computing and / or processing capabilities.

[0147] In one embodiment, the computer device 500 may include a processor, a memory, a network interface, a communication interface, etc. connected through a system bus. The processor of the computer device 500 may be used to provide necessary computing, processing and / or control capabilities. The memory of the computer device 500 may include a non-volatile storage medium and an internal memory. The non-volatile storage medium may store an operating system, a computer program, etc. The internal memory may provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the computer device 500 may be used to connect and communicate with external devices through a network. When the computer program is executed by the processor, the steps of the method of the present application are executed.

[0148] In addition, if Fig.13 As shown, the present application provides a computer-readable storage medium 600, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned optical defect detection method is implemented.

[0149] It will be appreciated by those skilled in the art that the method steps of the present application may be completed by instructing related hardware such as a computer device 500 or a processor through a computer program, and the computer program may be stored in a non-temporary computer-readable storage medium, which causes the steps of the present application to be executed when the computer program is executed. Depending on the circumstances, any reference to memory, storage, or other media herein may include non-volatile or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0150] The various technical features described above can be combined arbitrarily. Although all possible combinations of these technical features are not described, any combination of these technical features should be considered to be covered by this specification as long as there is no contradiction in such combination.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An optical defect detection method, characterized in that: The method comprises: Acquire a target image of a target to be detected and a reference image of at least one reference target adjacent to the target to be detected; Performing differential processing on the target image and the reference image to obtain a differential image; determining a defect region of interest of the differential image; Based on the deep learning network model, the pseudo-defect region in the defect region of interest is identified and filtered out to obtain an updated defect region of interest; the pseudo-defect region is a region that does not need to be identified as a defect; Defects of the target image to be inspected are identified based on the updated defect region of interest.

2. The method according to claim 1, characterized in that When the reference target is greater than one, the method further includes: Compute the intersection with the reference image; Selecting the area corresponding to the intersection in each reference image as the intersection reference image; The step of performing differential processing on the target image and the reference image to obtain a differential image comprises: The target image is subjected to difference processing with the intersection reference image respectively to obtain a difference image.

3. The method according to claim 1 or 2, characterized in that: When the reference target is greater than one, determining the defect region of interest of the differential image includes: Calculating the intersection of the difference images to obtain a difference intersection image; Calculating the mean and variance of the difference intersection image; Determining a filtering threshold based on the mean and the variance; The difference intersection image is binarized and segmented based on the filtering threshold to obtain a defect region of interest of the difference intersection image.

4. The method according to claim 3, characterized in that The method further comprises: performing differential processing on a reference image of the reference target to obtain a reference differential image; The calculating the intersection of the difference images comprises: An intersection of the difference image and the reference difference image is calculated.

5. The method according to claim 1, characterized in that The step of identifying and filtering out pseudo-defect regions in the defect region of interest based on the deep learning network model to obtain an updated defect region of interest comprises: Determine whether there is a suspected pseudo-defect region in the defect region of interest; wherein, an area in the defect region of interest that satisfies a specified adjacent relationship with a main connected area of ​​a target graphic in a target image and occupies at least two quadrants of four quadrants is identified as a suspected pseudo-defect region; wherein the four quadrants are quadrants divided by taking the center of the main connected area of ​​the target graphic in the target image as the coordinate origin; If so, regenerate the sub-region image of the reference image based on the trained deep learning network model using the target image and the reference image, so that the size of the target figure in the regenerated sub-region image of the reference image is consistent with the size of the target figure in the target image; Performing differential processing on the target image and the sub-region image of the regenerated reference image to obtain an updated differential image; An updated defect region of interest is obtained based on the updated differential image.

6. An optical defect detection device, characterized in that: The device comprises: An acquisition unit, used to acquire a target image of a target to be detected and a reference image of at least one reference target adjacent to the target to be detected; A differential processing unit, used for performing differential processing on the target image and the reference image to obtain a differential image; A defect region of interest determining unit, used to determine the defect region of interest of the differential image; A pseudo-defect recognition and filtering unit, used to identify and filter out pseudo-defect regions in the defect region of interest based on a deep learning network model to obtain an updated defect region of interest; the pseudo-defect region is a region that does not need to be identified as a defect; The defect detection unit is used to identify defects of the target image to be detected based on the updated defect region of interest.

7. The device according to claim 6, characterized in that The pseudo-defect recognition and filtering unit further comprises: A judgment subunit is used to judge whether there is a suspected pseudo-defect area in the defect region of interest; wherein, an area in the defect region of interest that satisfies a specified adjacent relationship with a main connected area of ​​a target figure in a target image and occupies at least two quadrants of four quadrants is identified as a suspected pseudo-defect area; wherein the four quadrants are quadrants divided by taking the center of the main connected area of ​​the target figure in the target image as the coordinate origin; A filtering subunit, configured to, when the judgment result of the judgment unit is that there is a suspected pseudo-defect area, regenerate a sub-region image of the reference image based on the trained deep learning network model using the target image and the reference image, so that the size of the target figure in the sub-region image of the regenerated reference image is consistent with the size of the target figure in the target image; A differential subunit, used for performing differential processing on the target image and the sub-region image of the regenerated reference image to obtain an updated differential image; The defect region of interest updating unit is used to obtain an updated defect region of interest based on the updated differential image.

8. A computer program product, characterized in that The computer program product comprises a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer device, characterized in that: include: a processor and a memory storing a computer program; When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which implements the steps of the method according to any one of claims 1 to 5 when executed by a processor.

Images