Detection method, detection system and storage medium for scribe line defects

Abstract

The present invention provides a method for detecting scribe lane defects, a system for detecting scribe lane defects, and a computer-readable storage medium. The method for detecting scribe lane defects comprises the following steps: obtaining first images of multiple scribe lanes between multiple grains on a sample to be tested; dividing each of the first images into multiple second images of scribe lane segments according to a preset size, and splicing the second images of the corresponding scribe lane segments of each of the scribe lanes into multiple third images; and analyzing each of the third images to respectively detect defects on each of the scribe lane segments of each of the scribe lanes. By reconstructing scribe lane images with a large aspect ratio, the present invention can effectively avoid the destruction of small defect features caused by image scaling, thereby improving the detection rate of scribe lane defects in the image.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor manufacturing and detection technology, and in particular to a method for detecting scribe line defects, a system for detecting scribe line defects, and a computer-readable storage medium. Background Art

[0002] Automatic Optical Inspection (AOI) technology can achieve fast, high-precision, non-destructive testing, and is therefore widely used in semiconductors, LEDs, pan-semiconductors, solar panels, and other fields. Semiconductor manufacturing usually involves a variety of different processes, such as chemical mechanical planarization (CMP), etching, deposition, and ion implantation. Wafer defect detection, classification, and filtering play an important role in improving yield.

[0003] Among the existing wafer defect detection technologies, there is detection for grains with similar lengths and widths. The processing method of such images is generally directly scaled to the input size required by the model. For example, the process method involved in patent CN111696077A establishes a wafer appearance defect dataset with an image data size of 640×640, then extracts wafer defect features through the EfficientNet neural network, and further extracts multi-scale features based on the Feature Pyramid Network (FPN), and then uses the RPN network to generate regions of interest on the extracted feature map, and finally uses a classification layer to identify the defect category of the region of interest. However, this method is only applicable to images with an aspect ratio close to 1. Images with an aspect ratio much greater than 1 need to be scaled, which means that compressing the defect size will inevitably destroy the structure of the defect and is not suitable for defect detection in the dicing lane. In addition, if a model with a specific input size adapted to a large aspect ratio is directly used for training, some defects on the dicing lane, such as scratches, contamination, and edge collapse, will be more prominent when the XY compression ratio is inconsistent, resulting in a significant decrease in the resolution of the model.

[0004] In addition, in the existing patent CN116486242A, two data enhancement strategies, Mosaic and Mixup, are used to enhance the detection images on the training set of small-scale phytoplankton target detection images. However, the Mosaic data processing measurement is randomly cropped according to the proportion, and the scaling caused by its cropping ratio will cause the defect characteristics to change, which is not suitable for defect detection in the scribe line. The Mixup data enhancement strategy mixes the grayscale values ​​of the two images, which will cause the specific pixel values ​​in the image to change. Therefore, the Mixup data enhancement strategy will destroy the relative relationship between the pixel value and the background pixel in its field, destroy the original defect characteristics, and is also not suitable for defect detection in the scribe line.

[0005] In order to overcome the above-mentioned defects in the prior art, an improved detection method for scribe line defects is urgently needed in the art to improve the detection rate of scribe line defects in scribe line images with a large aspect ratio. Summary of the invention

[0006] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or decisive elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0007] In order to overcome the above-mentioned defects in the prior art, the present invention provides a method for detecting scribe line defects, a system for detecting scribe line defects and a computer-readable storage medium, which can effectively avoid the destruction of small defect features caused by image scaling by reconstructing scribe line images with large aspect ratios, thereby improving the detection rate of scribe line defects in the image.

[0008] Specifically, the above-mentioned method for detecting scribe line defects provided according to the first aspect of the present invention includes the following steps: obtaining first images of multiple scribe lines between multiple grains on a sample to be tested; dividing each of the first images into multiple second images of scribe line segments according to a preset size, and splicing the second images of the corresponding scribe line segments of each of the scribe lines into multiple third images; and analyzing each of the third images to respectively detect defects on each of the scribe line segments of each of the scribe lines.

[0009] Furthermore, in some embodiments of the present invention, the step of acquiring first images of multiple scribing lanes between multiple grains on the sample to be tested includes: placing the sample to be tested on a moving stage of a defect detection device; and moving the multiple scribing lanes between multiple grains on the sample to be tested to an imaging area of ​​a microscopic imaging module of the defect detection device in sequence via the moving stage, so as to sequentially acquire first images of the multiple scribing lanes via the microscopic imaging module.

[0010] Further, in some embodiments of the present invention, the step of dividing each of the first images into a plurality of second images of scribing lane segments according to a preset size includes: determining the number of scribing lane segments to be divided according to the width of the first image and the preset width of the third image; and determining the net width of each scribing lane segment after removing the overlapping area according to the preset overlap area width and the number of scribing lane segments to be divided. The overlapping area is located between two adjacent scribing lane segments; and determining the segmentation position of each of the second images according to the net width of each scribing lane segment and the overlap area width, so as to perform the segmentation.

[0011] Furthermore, in some embodiments of the present invention, before dividing each of the first images into a plurality of second images of scribe line segments according to preset sizes, the detection method also includes the following steps: detecting the width and height of each of the first images; and in response to the width of the first image being smaller than the corresponding height, rotating the first image 90° in a clockwise or counterclockwise direction.

[0012] Further, in some embodiments of the present invention, the step of stitching the second images of the corresponding scribe line fragments of each of the scribe lines into a plurality of third images includes: determining the height of the image stitching unit according to the height of the first image and the preset height of the spacing area; determining the number of stitched scribe line fragments in each of the third images according to the preset height of the third image and the height of the image stitching unit; and determining the stitching position of each of the second images in the corresponding third image according to the height of the image stitching unit, so as to perform the stitching.

[0013] Further, in some embodiments of the present invention, the step of parsing each of the third images to respectively detect defects on each of the scribe line segments of each of the scribe lines comprises: inputting each of the third images into a pre-trained deep learning model to determine its corresponding defect detection result [y1, x1, y2, x2, cls, sco]. y1, x1, y2, x2 are the coordinates of two feature points of the defect in the third image. cls is the defect category with the highest predicted probability score. sco is the predicted probability score of the defect category to which the defect belongs to cls; and in response to the predicted probability score being greater than a preset score threshold, outputting the defect category and coordinate position of the defect.

[0014] Furthermore, in some embodiments of the present invention, the step of outputting the defect category and coordinate position of the defect includes: looking up a table based on the defect category data cls to determine and output text information of the defect category; and calculating and outputting the position of the defect on the sample to be tested based on the coordinate data (x1, y1) and (x2, y2) of the two feature points.

[0015] Further, in some embodiments of the present invention, the step of calculating and outputting the position of the defect on the sample to be tested according to the coordinate data (x1, y1) and (x2, y2) of the two feature points includes: determining a first X coordinate of the defect in the first image according to the serial number of the third image where the defect is located, and the coordinate data x1 and x2; determining the serial number of the image stitching unit of the defect in the third image according to the coordinate data y1 and y2, and then determining a first Y coordinate of the defect in the first image.

[0016] Further, in some embodiments of the present invention, the step of calculating and outputting the position of the defect on the sample to be tested according to the coordinate data (x1, y1) and (x2, y2) of the two feature points further includes: for the rotated and re-divided first image, inversely rotating the first X coordinate and the first Y coordinate to respectively determine a second X coordinate and a second Y coordinate of the defect in the first image.

[0017] Further, in some embodiments of the present invention, the deep learning model includes a feature extraction module, a feature fusion module and a prediction module. The step of training the deep learning model includes: preparing a plurality of first image samples of scribe lines between a plurality of grains on a plurality of semiconductor samples; dividing each of the first image samples into a plurality of second image samples of scribe line segments according to a preset size, and splicing each of the second image samples into a plurality of third image samples; marking the true value [y1, x1, y2, x2, cls] of the defect data of each of the third image samples respectively to construct a training set of the deep learning model. y1, x1, y2, x2 are the coordinates of the two feature points of the defect in the third image. cls is the defect category; inputting each of the third image samples into the deep learning model to be trained respectively to determine the predicted value [y1', x1', y2', x2', cls', sco'] of the defect detection result; and adjusting the learning parameters of the deep learning model according to the predicted value and its corresponding true value until the difference between the two is less than a preset loss threshold.

[0018] Furthermore, in some embodiments of the present invention, the step of splicing each of the second image samples into multiple third image samples includes: after completing the segmentation of each of the first image samples, randomly taking out a preset number of second image samples, and splicing them into one of the third image samples to expand the number of samples in the training set.

[0019] In addition, the above-mentioned detection system for scribe line defects provided according to the second aspect of the present invention includes a memory and a processor. The memory stores computer instructions. The processor is connected to the memory and is configured to execute the computer instructions stored in the memory to implement the detection method for scribe line defects provided according to the first aspect of the present invention.

[0020] Furthermore, in some embodiments of the present invention, the detection system further includes a defect detection device, which includes a moving stage and a microscopic imaging module. The moving stage is used to carry the sample to be tested, and to sequentially move multiple scribe lines between multiple grains on the moving stage to an imaging area of ​​the microscopic imaging module. The microscopic imaging module is used to sequentially capture first images of the multiple scribe lines.

[0021] In addition, the computer-readable storage medium provided in the third aspect of the present invention stores computer instructions, which, when executed by a processor, implement the method for detecting scribe line defects provided in the first aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above features and advantages of the present invention can be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or features may have the same or similar reference numerals.

[0023] Figure 1 A schematic structural diagram of a scribe line defect detection system provided according to some embodiments of the present invention is shown.

[0024] Figure 2 A schematic diagram of the algorithm architecture of a scribe line defect detection system provided according to some embodiments of the present invention is shown.

[0025] Figure 3 A schematic diagram of the principles of a deep learning model provided according to some embodiments of the present invention is shown.

[0026] Figure 4 A schematic diagram of a first image sample provided according to some embodiments of the present invention is shown.

[0027] Figure 5 A schematic diagram of dividing a longitudinal scribe line according to some embodiments of the present invention is shown.

[0028] Figure 6 A schematic diagram of divided scribe line segments according to some embodiments of the present invention is shown.

[0029] Figure 7 A schematic diagram of dividing a horizontal scribe line according to some embodiments of the present invention is shown.

[0030] Figure 8 A schematic diagram showing a plurality of third image samples stitched together according to some embodiments of the present invention is shown.

[0031] Fig. 9 A schematic diagram of a third image sample stitched together according to some embodiments of the present invention is shown.

[0032] Fig.10 A schematic flow chart of an online detection phase of a method for detecting scribe line defects provided according to some embodiments of the present invention is shown.

[0033] Fig.11 A schematic diagram of an image before detection provided according to some embodiments of the present invention is shown.

[0034] Fig.12 A schematic diagram of an image after detection provided according to some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0035] The following specific embodiments illustrate the implementation of the present invention, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Although the description of the present invention will be introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to this implementation. On the contrary, the purpose of introducing the invention in conjunction with the implementation is to cover other options or modifications that may be extended based on the claims of the present invention. In order to provide a deep understanding of the present invention, the following description will include many specific details. The present invention can also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present invention, some specific details will be omitted in the description.

[0036] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0037] In addition, the terms "upper", "lower", "left", "right", "top", "bottom", "horizontal" and "vertical" used in the following description should be understood as the directions shown in the paragraph and the related drawings. Such relative terms are only used for the convenience of description and do not mean that the device described therein must be manufactured or operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0038] It is understood that although the terms "first", "second", "third", etc. may be used herein to describe various components, regions, layers and / or parts, these components, regions, layers and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers and / or parts. Therefore, the first component, region, layer and / or part discussed below may be referred to as a second component, region, layer and / or part without departing from some embodiments of the present invention.

[0039] As mentioned above, existing wafer defect detection technologies include detection of grains with similar lengths and widths. Such images are generally processed by directly scaling them to the input size required by the model. However, this method is only applicable to images with an aspect ratio close to 1. Images with an aspect ratio much greater than 1 need to be scaled, which means that compressing the defect size will inevitably destroy the structure of the defect and is not suitable for defect detection in the dicing lane. In addition, if a model with a specific input size adapted to a large aspect ratio is directly used for training, certain defects on the dicing lane, such as scratches, stains, and edge collapse, will be more prominent when the XY compression ratio is inconsistent, resulting in a significant decrease in the model's resolution.

[0040] In addition, the Mosaic data processing measurement is randomly cropped according to the proportion, and the scaling caused by the cropping proportion will cause the defect characteristics to change, which is not suitable for defect detection in the scribe line. The Mixup data enhancement strategy mixes the grayscale values ​​of two images, which will cause the specific pixel values ​​in the image to change. Therefore, the Mixup data enhancement strategy will destroy the relative relationship between the pixel value and the background pixel in its field, destroying the original defect characteristics, and is also not suitable for defect detection in the scribe line.

[0041] In order to overcome the above-mentioned defects in the prior art, the present invention provides a method for detecting scribe line defects, a system for detecting scribe line defects and a computer-readable storage medium, which can effectively avoid the destruction of small defect features caused by image scaling by reconstructing scribe line images with large aspect ratios, thereby improving the detection rate of scribe line defects in the image.

[0042] In some non-limiting embodiments, the above-mentioned detection method of the scribe line defect provided in the first aspect of the present invention can be implemented based on the above-mentioned detection system of the scribe line defect provided in the second aspect of the present invention. The detection system includes a memory and a processor. The memory includes but is not limited to the above-mentioned computer-readable storage medium provided in the third aspect of the present invention, on which computer instructions are stored. The processor is connected to the memory and is configured to execute the computer instructions stored on the memory to implement the detection method of the scribe line defect provided in the first aspect of the present invention.

[0043] In addition, please refer to Figure 1 . Figure 1 A schematic structural diagram of a scribe line defect detection system provided according to some embodiments of the present invention is shown.

[0044] exist Figure 1In the illustrated embodiment, the above-mentioned scribe line defect detection system provided by the second aspect of the present invention further includes a defect detection device. Here, the defect detection device includes a moving platform 11 and a microscopic imaging module 12. The moving platform 11 is used to carry a sample to be tested 13, and to sequentially move multiple scribe lines between multiple grains on the moving platform to an imaging area of ​​the microscopic imaging module. The microscopic imaging module 12 is used to sequentially capture first images of multiple scribe lines.

[0045] Specifically, the microscopic imaging module 12 includes a light source 121, an illumination control unit 122, an imaging system 123, and a sensor 124. Here, the emitted light of the light source 121 passes through the illumination control unit 122 to form the illumination light required for detection, and projects the light beam onto the sample to be tested 13. The reflected light or scattered light of the illuminated area on the sample to be tested 13 is collected by the imaging system 123, and then forms a detection image via the sensor 124.

[0046] In addition, please refer to Figure 2 . Figure 2 A schematic diagram of the algorithm architecture of a scribe line defect detection system provided according to some embodiments of the present invention is shown.

[0047] exist Figure 2 In the illustrated embodiment, the method for detecting scribe line defects can be divided into two parts: an offline training phase of the model and an online detection phase of scribe line defects, and the scribe line defect detection system can be independently operated by different actors controlling an offline training module 101 and an online prediction module 102.

[0048] The following will first describe the working principle of the above-mentioned detection system for scribe line defects in conjunction with some examples of the offline training stage of the detection model. Those skilled in the art will understand that the embodiments of these detection methods are only some non-restrictive implementation methods provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit all functions or all working modes of the detection system. Similarly, the detection system for scribe line defects is also only a non-restrictive implementation method provided by the present invention, and does not constitute a limitation on the execution subject or execution order of each step in these detection methods for scribe line defects.

[0049] Please refer to Figure 2 and Figure 3 , Figure 3 A schematic diagram of the principles of a deep learning model provided according to some embodiments of the present invention is shown.

[0050] exist Figure 2 and Figure 3In the illustrated embodiment, the present invention may first train the deep learning model 1013 in an offline training process via the offline training module 101 of the detection system and using the labeled third image sample 1011 .

[0051] Specifically, the above-mentioned deep learning model 1013 can adopt the existing deep learning model architecture, which includes a feature extraction module 31, a feature fusion module 32 and a prediction module 33. Further, in some embodiments, the feature extraction module 31 can use a feature extraction network that incorporates an attention mechanism, which adds a se_block module 312 on the basis of the CSPDarknet53-Tiny network 311 to enhance the feature extraction of small defects. The feature fusion module 32 can use a feature pyramid structure FPN. The prediction module 33 can use a YoloHead network, which contains some convolutional layers to achieve the final prediction of the detection results.

[0052] Please refer to further Figure 4~Figure 7 . Figure 4 A schematic diagram of a first image sample provided according to some embodiments of the present invention is shown. Figure 5 A schematic diagram of dividing a longitudinal scribe line according to some embodiments of the present invention is shown. Figure 6 A schematic diagram of divided scribe line segments according to some embodiments of the present invention is shown. Figure 7 A schematic diagram of dividing a horizontal scribe line according to some embodiments of the present invention is shown.

[0053] like Figure 4 As shown, in the process of training the above-mentioned deep learning model, the present invention can first prepare a first image sample of a plurality of scribe lines between a plurality of grains on a plurality of semiconductor samples. Here, the first image sample includes a plurality of transverse scribe lines 41 and a plurality of longitudinal scribe lines 42.

[0054] Afterwards, the present invention can divide each first image sample into a plurality of second image samples of scribe line segments according to a preset size, and splice each second image sample into a plurality of third image samples 1011 .

[0055] Here, the present invention may first determine the number scolN of scribe line segments to be divided according to the width SL_w of the first image sample and the preset width Img_W of the third image sample 1011:

[0056]

[0057] Here, the operation " ” stands for division downwards.

[0058] Afterwards, the present invention can determine the net width cutScribleL of each scribe line segment after removing the overlapping area according to the preset overlapping area width Overlap_W (for example, 50 pixels) and the number of scribe line segments to be segmented scolN:

[0059]

[0060] Here, the overlap region is located between two adjacent scribe line segments, and is used to avoid missed detection due to segmentation defects.

[0061] In addition, in some embodiments, the present invention can also calculate the total width of the scribe line segments to avoid it being larger than the input size of the deep learning model. , it can be determined that the width of the second image sample obtained by clipping is larger than the input size of the prediction model, so scolN is increased by 1, and then the net width cutScribleL is recalculated in the above manner. Here, if Figure 5 and Figure 7 As shown, the net width CutScribleL is the length of the actual cut of the scribe line excluding the overlapping area, that is, the length of the white area in the scribe line segments 411-413 and scribe line segments 421-423, and the black area width is the overlapping area width Overlap_W. In this way, when the total width of the scribe line segments is larger than the input size of the deep learning model, it is necessary to divide it into scolN+1 segments so that the total width of the re-divided scribe line segments is smaller than the input size of the deep learning model.

[0062] Those skilled in the art can understand that the above-mentioned embodiments of setting the overlapping area between two adjacent dicing lane segments are only some non-limiting implementation methods provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit the scope of protection of the present invention.

[0063] Optionally, in some other embodiments, those skilled in the art may not set the overlap area, but directly determine the width of the scribe line segment according to the preset width of the third image sample.

[0064] After that, if Figure 5 and Figure 7 As shown, the present invention can determine the segmentation position offset_x1 or offset_x2 of each second image sample according to the net width cutScribleL of each scribe line segment and the overlap area width Overlap_W, so as to perform segmentation:

[0065]

[0066]

[0067] In addition, Figure 5 In the illustrated embodiment, before dividing the first image sample into a plurality of second image samples of scribe lane segments, the present invention can also preferably unify all scribe lanes in a horizontal direction.

[0068] Specifically, the present invention can detect the width and height of each first image sample. In response to the width of the first image sample being smaller than the corresponding height, the present invention can rotate the first image sample 90° in a clockwise or counterclockwise direction to convert the longitudinal scribing lane 42 into the transverse scribing lane 41.

[0069] Alternatively, in some embodiments, the present invention may also unify all scribe lines into a longitudinal direction, and in the algorithm, define the width parameter of the longitudinal scribe line 42 as SL_h, and define the height parameter as SL_w.

[0070] Please refer to the following for further reference Figure 8 and Fig. 9 . Figure 8 A schematic diagram showing a plurality of third image samples stitched together according to some embodiments of the present invention is shown. Fig. 9 A schematic diagram of a third image sample stitched together according to some embodiments of the present invention is shown.

[0071] like Figure 8 and Fig. 9 As shown, the present invention can splice the second image samples of the corresponding scribe lane segments of each scribe lane into multiple third image samples 1011 for batch prediction of multiple scribe lanes to improve prediction efficiency.

[0072] Specifically, the present invention can determine the height SL_Ext_H of the image splicing unit according to the height SL_h of the first image sample and the preset spacing area height boundaryExt (for example, 10 pixels in height), so as to facilitate the deep learning network to identify the scribe lane segments:

[0073]

[0074] Those skilled in the art will appreciate that the above-mentioned embodiments of setting the spacing region 80 between each dicing lane segment are only some non-limiting implementation modes provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, but are not intended to limit the scope of protection of the present invention.

[0075] Optionally, in some other embodiments, those skilled in the art may also only set one spacing area between two longitudinally adjacent scribe line segments, or set a specific linear dividing line between two longitudinally adjacent scribe line segments.

[0076] Afterwards, the present invention can determine the number srowN of spliced ​​scribe lane segments in each third image sample 1011 according to the preset height Img_H of the third image sample and the height SL_Ext_H of the image splicing unit:

[0077]

[0078] Afterwards, the present invention can determine the stitching position offset_y of the first second image sample in the corresponding third image sample 1011 according to the height SL_Ext_H of the image stitching unit:

[0079]

[0080] Afterwards, the present invention can stitch the second image samples of the corresponding scribe line segments of each scribe line from top to bottom according to the stitching quantity parameter srowN and the stitching position parameter offset_y, and fill the interval area with black to make it easier for the deep learning network to identify the scribe line segments.

[0081] Afterwards, the present invention can respectively mark the true value [y1, x1, y2, x2, cls] of the defect data of each third image sample 1011 to construct a training set for the deep learning model 1013. Here, y1, x1, y2, x2 are the coordinates of two feature points of the defect (for example, the upper left corner and the lower right corner, or the upper right corner and the lower left corner) in the third image sample 1011, and cls is the defect category.

[0082] Furthermore, in some embodiments, in addition to splicing the second image samples of the corresponding scribe lane segments of each scribe lane, the present invention can also randomly take out a preset number (i.e., srowN calculated in the previous lane) of second image samples and splice them into a third image sample 1011, and then annotate the true value [y1, x1, y2, x2, cls] of the defect data of the third image sample 1011 as described above to expand the number of samples in the training set. Here, the second image sample can be the second image samples of multiple scribe lane segments at different positions of multiple scribe lanes (or even the same scribe lane).

[0083] Thereafter, the present invention can encode each third image sample 1011 through the slicing line encoding module 1012, and input them into the deep learning model 1013 to be trained respectively to determine the defect detection result, and output the corresponding prediction value [y1', x1', y2', x2', cls', sco'] through the decoding module 1014, and then adjust the learning parameters of the deep learning model according to the predicted value and its corresponding true value until the preset number of training times is reached, or the difference between the predicted value and its corresponding true value is less than the preset loss threshold.

[0084] In some embodiments, the model optimizer of the deep learning model 1013 can use the Adam method, its learning rate can be set to 0.001, the decay rate can be set to 0.0001, the loss function can use the cross entropy loss function, the single batch training number (Batchsize) can be set to 32, and the number of training times can be set to 200.

[0085] By executing the above steps, a person skilled in the art can obtain a detection model 1023 that determines the coordinates, type, and predicted probability score of at least one scribe line defect according to the reconstructed third image 1021. Afterwards, the technician can store it in a CD, a disk, a network disk, or various other storage media for access and call by an online detection module of a scribe line defect detection system.

[0086] The following will further describe the working principle of the above-mentioned detection system for scribe line defects in combination with some embodiments of online detection of scribe line defects based on the above-mentioned detection model. Those skilled in the art will understand that the embodiments of these detection methods are only some non-restrictive implementation methods provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit all functions or all working modes of the detection system. Similarly, the detection system for scribe line defects is also only a non-restrictive implementation method provided by the present invention, and does not constitute a limitation on the execution subject or execution order of each step in these detection methods for scribe line defects.

[0087] Please refer to Figure 2 , Figure 4 and Fig.10 . Fig.10 A schematic flow chart of an online detection phase of a method for detecting scribe line defects provided according to some embodiments of the present invention is shown.

[0088] like Figure 2 , Figure 4 and Fig.10As shown, the algorithm architecture of the processor 10 of the above-mentioned scribing defect detection system provided by the present invention may also include an online prediction module 102. In the process of detecting scribing defects on the sample 13 to be tested, the processor 10 may first obtain a first image of multiple scribing lanes between multiple grains on the sample 13 to be tested. Here, the first image includes multiple transverse scribing lanes 41 and multiple longitudinal scribing lanes 42.

[0089] Specifically, the technician can place the sample to be tested 13 on the moving table 11 of the defect detection device, and move the multiple dicing lanes between the multiple grains on the sample to be tested 13 to the imaging area of ​​the microscopic imaging module 12 of the defect detection device in sequence via the moving table 11, so as to collect the first images of the multiple dicing lanes in sequence via the microscopic imaging module 12.

[0090] Afterwards, if Figure 5 and Figure 7 As shown, the processor 10 can divide each first image into a second image of a plurality of scribing road segments according to a preset size. Specifically, the processor 10 can first use the above formula (1) to determine the number scolN of scribing road segments to be divided according to the width SL_w of the first image and the preset width Img_W of the third image, and then use the above formula (2) to determine the net width cutScribleL of each scribing road segment after removing the overlapping area according to the preset overlap area width Overlap_W (for example: 50 pixels width) and the number scolN of scribing road segments to be divided. Here, the overlapping area is located between two adjacent scribing road segments to avoid missed detection due to segmentation defects.

[0091] Furthermore, in some embodiments, the processor 10 may also calculate the total width of the scribe line segments to avoid it being larger than the input size of the deep learning model.

[0092] Specifically, in response to , the processor 10 may determine that the width of the cropped second image is greater than the input size of the prediction model, thereby adding 1 to scolN, and then recalculating the net width cutScribleL in the above manner. Figure 5 and Figure 7 As shown, the net width CutScribleL is the length of the overlapped area not included in the actual cut of the scribe line, that is, the length of the white area in 411-413 and 421-423, and the black area width is the overlapped area width Overlap_W. In this way, when the total width of the scribe line segment is larger than the input size of the deep learning model, it needs to be divided into scolN+1 segments so that the total width of the re-divided scribe line segment is smaller than the input size of the deep learning model.

[0093] Those skilled in the art can understand that the above-mentioned embodiments of setting the overlapping area between two adjacent dicing lane segments are only some non-limiting implementation methods provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit the scope of protection of the present invention.

[0094] Optionally, in some other embodiments, those skilled in the art may not set the overlap area, but directly determine the width of the scribe line segment according to the preset width of the third image.

[0095] After that, if Figure 5 and Figure 7 As shown, the processor 10 can use the above formulas (3) and (4) to determine the segmentation position offset_x1 or offset_x2 of each second image according to the net width cutScribleL of each scribe line segment and the overlap area width Overlap_W for segmentation.

[0096] In addition, Figure 5 In the illustrated embodiment, before dividing the first image into a plurality of second images of the scribing lane segments, the processor 10 may also preferably unify all scribing lanes into a horizontal orientation. Specifically, the processor 10 may detect the width and height of each first image. In response to the width of the first image being smaller than the corresponding height, the processor 10 may rotate the first image 90° in a clockwise or counterclockwise direction to convert the longitudinal scribing lane 42 into a horizontal orientation.

[0097] Alternatively, in some embodiments, the processor 10 may also unify all the scribe lines into a vertical direction, and in the algorithm, define the width parameter of the vertical scribe line as SL_h and the height parameter thereof as SL_w to achieve the same effect.

[0098] Afterwards, if Figure 8 and Fig. 9 As shown, the processor 10 may stitch the second images of the corresponding scribe lane segments of each scribe lane into a plurality of third images 1021 to perform batch prediction on the plurality of scribe lanes, thereby improving prediction efficiency.

[0099] Specifically, the processor 10 can use the above formula (5) to determine the height SL_Ext_H of the image stitching unit according to the height SL_h of the first image and the preset height boundaryExt of the spacing area 80 (for example, 10 pixels in height), so as to facilitate the deep learning network to identify the dicing lane segments.

[0100] Those skilled in the art will appreciate that the above-mentioned embodiments of setting the spacing region 80 between each dicing lane segment are only some non-limiting implementation modes provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, but are not intended to limit the scope of protection of the present invention.

[0101] Optionally, in some other embodiments, those skilled in the art may also only set one spacing area between two longitudinally adjacent scribe line segments, or set a specific linear dividing line between two longitudinally adjacent scribe line segments.

[0102] Afterwards, the processor 10 can use the above formula (6) to determine the number srowN of spliced ​​scribe line segments in each third image according to the preset height Img_H of the third image and the height SL_Ext_H of the image splicing unit, and then use the above formula (7) to determine the splicing position offset_y of each second image in the corresponding third image according to the height SL_Ext_H of the image splicing unit.

[0103] Afterwards, the processor 10 may stitch the second images of the corresponding scribe line segments of the scribe line according to the stitching quantity parameter srowN and the stitching position parameter offset_y, and fill the interval area with black to facilitate the deep learning network to identify the scribe line segments.

[0104] Afterwards, the processor 10 may encode the spliced ​​third images through the scribe line encoding module 1022, and input them into the detection model obtained by the previous training for analysis, so as to detect defects on each scribe line segment of each scribe line. Here, the scribe line encoding module 1022 may have the same encoding rules as the scribe line encoding module 1012 in the offline training module 101.

[0105] Specifically, in response to acquiring the third image 1021, the pre-trained detection model 1023 can perform neural network inference based on it, and determine and output the corresponding defect detection result [y1, x1, y2, x2, cls, sco] via the corresponding decoding module 1024. Here, the decoding module 1024 can have the same decoding rules as the scribe line decoding module 1014 in the offline training module 101. y1, x1, y2, x2 are the coordinates of two feature points of the defect (for example: the upper left corner and the lower right corner, or the upper right corner and the lower left corner) in the third image, cls is the defect category with the highest predicted probability score, and sco is the predicted probability score of the defect category to which the defect belongs to cls.

[0106] Please refer to Fig.11 and Fig.12 . Fig.11 A schematic diagram of an image before detection provided according to some embodiments of the present invention is shown. Fig.12 A schematic diagram of an image after detection provided according to some embodiments of the present invention is shown.

[0107] like Fig.11 and Fig.12 As shown, in response to the predicted probability score being greater than the preset score threshold, the processor 10 can look up the table according to the defect category data cls obtained by decoding to determine and output the text information of the defect category, and output the defect category of the defect. In addition, the processor 10 can also mark the third image 1021 with a box according to the coordinate positions of the two feature points.

[0108] Further, in order to facilitate the user to intuitively understand the position of the defect on the sample 13 to be tested, the processor 10 can also preferably determine the first X coordinates shift_x1 and shift_x2 of the defect in the first image according to the sequence number col_i of the third image where the defect is located, and the coordinate data x1 and x2:

[0109]

[0110]

[0111] Afterwards, the processor 10 may determine the sequence number row_j of the image stitching unit where the defect is located in the third image according to the coordinate data y1 and y2:

[0112]

[0113]

[0114] Afterwards, the processor 10 may determine the first Y coordinates shift_y1 and shift_y2 of the defect in the first image:

[0115]

[0116]

[0117] Furthermore, for the rotated and segmented first image, the processor 10 may also preferably reversely rotate the first X coordinate and the first Y coordinate to respectively determine the second X coordinates Rotate_x1, Rotate_x2 and the second Y coordinates Rotate_y1, Rotate_y2 of the defect in the first image:

[0118]

[0119]

[0120]

[0121]

[0122] Afterwards, technicians in this field can verify the effectiveness of the detection method for dicing lane defects provided by the first aspect of the present invention. After the training of the above-mentioned deep learning model 1013 is completed, no further parameter adjustment is required to effectively detect small defects on dicing lanes with an aspect ratio much greater than 1, and the average precision mAP of the test data reaches more than 0.9.

[0123] Therefore, by executing the above steps of analyzing the third image and converting the coordinate position, the present invention can efficiently and accurately determine the type of each defect in the scribe line image and its exact position on the sample 13 to be tested.

[0124] Those skilled in the art will understand that the above-mentioned embodiments of training the deep learning model using the image samples processed by the slicing lane coding module and using it to perform defect detection on each image stitching unit are only some non-limiting implementation methods provided by the present invention, which are intended to clearly demonstrate the main concepts of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit the scope of protection of the present invention.

[0125] Optionally, in other embodiments, those skilled in the art may also use a common machine learning model to respectively calculate the overall similarity of each image stitching unit in each third image to determine whether there is a defect in each image stitching unit.

[0126] In summary, the above-mentioned scribe line defect detection method, scribe line defect detection system and computer-readable storage medium provided by the present invention can effectively avoid the destruction of small defect features caused by image scaling by reconstructing scribe line images with large aspect ratios, thereby improving the detection rate of scribe line defects in the image.

[0127] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0128] Those skilled in the art will appreciate that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips cited throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or optical particles, or any combination thereof.

[0129] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0130] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium may be integrated into a processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0131] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0132] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but should be granted the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting scribe line defects, characterized in that: The following steps are involved: Acquire a first image of a plurality of scribe lines between a plurality of grains on a sample to be tested; Determining the number of scribe line segments to be segmented and the net width of each scribe line segment according to the width of each of the first images and the preset width of the third image to be spliced; According to the net width of each of the scribe line segments, each of the first images is divided into a plurality of second images of the scribe line segments; Determining a height of an image stitching unit according to a height of the first image; Determining the number of scribe street segments to be spliced ​​in each of the third images according to a preset height of the third image and a height of the image splicing unit; Determining the stitching positions of the second images in the corresponding third images according to the height of the image stitching unit, so as to stitch the second images of the corresponding scribe line segments of the scribe lines into a plurality of the third images; and Each of the third images is analyzed to detect defects on each of the scribe line segments of each of the scribe lines.

2. The method for detecting scribe line defects according to claim 1, wherein: The step of acquiring a first image of a plurality of scribe lines between a plurality of grains on the sample to be tested comprises: Placing the sample to be tested on a moving platform of a defect detection device; and The plurality of scribing lanes between the plurality of grains on the sample to be tested are sequentially moved to the imaging area of ​​the microscopic imaging module of the defect detection device via the moving stage, so that the first images of the plurality of scribing lanes are sequentially acquired via the microscopic imaging module.

3. The method for detecting scribe line defects according to claim 1, wherein: The step of determining the net width of each of the scribe line segments comprises: determining the net width of each of the scribe line segments after removing the overlapped area according to a preset overlapped area width and the number of the scribe line segments to be divided, wherein the overlapped area is located between two adjacent scribe line segments, The step of dividing each of the first images into a plurality of second images of the scribe line segments according to the net width of each of the scribe line segments comprises: determining the division position of each of the second images according to the net width of each of the scribe line segments and the width of the overlapping area to perform the division.

4. The method for detecting scribe line defects according to claim 3, wherein: Before performing the segmentation, the detection method further comprises the following steps: detecting the width and height of each of the first images; and In response to the width of the first image being smaller than the corresponding height, the first image is rotated 90° in a clockwise or counterclockwise direction.

5. The method for detecting scribe line defects according to claim 3, wherein: The step of determining the height of the image stitching unit according to the height of the first image comprises: The height of the image stitching unit is determined according to the height of the first image and the preset height of the spacing area.

6. The method for detecting scribe line defects according to claim 5, wherein: The step of analyzing each of the third images to respectively detect defects on each of the scribe line segments of each of the scribe lines comprises: Each of the third images is input into a pre-trained deep learning model to determine its corresponding defect detection result [ y 1, x 1, y 2, x 2, cls , sco ],in, y 1, x 1, y 2, x 2 are the coordinates of the two characteristic points of the defect in the third image, cls is the defect category with the highest predicted probability score, sco The defect belongs to cls The predicted probability score of the defect category; and In response to the predicted probability score being greater than a preset score threshold, a defect category and a coordinate position of the defect are output.

7. The method for detecting scribe line defects according to claim 6, wherein: The step of outputting the defect type and coordinate position of the defect comprises: According to defect category data cls Look up a table to determine and output text information of the defect category; and According to the coordinate data of the two feature points ( x 1, y 1) and ( x 2, y 2) Calculate and output the position of the defect on the sample to be tested.

8. The method for detecting scribe line defects according to claim 7, wherein: The coordinate data of the two feature points ( x 1, y 1) and ( x 2, y 2) The step of calculating and outputting the position of the defect on the sample to be tested includes: According to the sequence number of the third image where the defect is located, and the coordinate data x 1 and x 2. Determine a first X coordinate of the defect in the first image; According to the coordinate data y 1 and y 2. Determine the serial number of the image stitching unit of the defect in the third image, and then determine the first Y coordinate of the defect in the first image.

9. The method for detecting scribe line defects according to claim 8, wherein: The coordinate data of the two feature points ( x 1, y 1) and ( x 2, y 2), the step of calculating and outputting the position of the defect on the sample to be tested also includes: For the rotated and segmented first image, the first X coordinate and the first Y coordinate are inversely rotated to respectively determine a second X coordinate and a second Y coordinate of the defect in the first image.

10. The method for detecting scribe line defects according to claim 6, wherein: The deep learning model includes a feature extraction module, a feature fusion module and a prediction module, and the steps of training the deep learning model include: preparing first image samples of a plurality of scribe lines between a plurality of grains on a plurality of semiconductor samples; Dividing each of the first image samples into a plurality of second image samples of scribe line segments according to a preset size, and splicing each of the second image samples into a plurality of third image samples; The true value of the defect data of each of the third image samples is marked respectively. y 1, x 1, y 2, x 2, cls ] to construct a training set for the deep learning model, where: y 1, x 1, y 2, x 2 are the coordinates of the two characteristic points of the defect in the third image, cls is the defect category; Each of the third image samples is input into the deep learning model to be trained to determine the predicted value of the defect detection result [ y 1 ’ , x 1 ’ , y 2 ’ , x 2 ’ , cls' , sco' ];as well as According to the predicted value and its corresponding true value, the learning parameters of the deep learning model are adjusted until the difference between the two is less than a preset loss threshold.

11. The method for detecting scribe line defects according to claim 10, wherein: The step of splicing each of the second image samples into a plurality of third image samples comprises: After completing the segmentation of each of the first image samples, a preset number of second image samples are randomly taken out and spliced ​​into one of the third image samples to expand the number of samples in the training set.

12. A scribe line defect detection system, characterized in that: include: a memory having computer instructions stored thereon; and A processor is connected to the memory and is configured to execute computer instructions stored in the memory to implement the method for detecting scribe line defects as described in any one of claims 1 to 11.

13. The detection system according to claim 12, characterized in that The detection system also includes a defect detection device, which includes: A moving stage, used for carrying the sample to be tested and sequentially moving a plurality of scribing lanes between a plurality of grains on the sample to the imaging area of ​​the microscopic imaging module; and The microscopic imaging module is used to sequentially collect first images of the plurality of scribe lines.

14. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by the processor, the method for detecting scribe line defects according to any one of claims 1 to 11 is implemented.

Images