Semiconductor defect detection method and system and storage medium

Abstract

The invention provides a semiconductor defect detection method and system and a storage medium. The semiconductor defect detection method comprises the following steps: obtaining a detection image of a semiconductor device to be detected; performing target detection of preset defects on the detection image to determine at least one first candidate defect and a corresponding first confidence coefficient; performing feature comparison on the detection image and a preset template image so as to determine at least one second candidate defect and a second confidence coefficient corresponding to the second candidate defect; performing routing selection based on neural network reasoning on the detection image to determine a first score of the detection image about the target detection and a second score of the detection image about the feature comparison; and screening a defect detection result of the semiconductor device to be detected from each first candidate defect and each second candidate defect according to the first score, the second score, the first confidence coefficient of each first candidate defect and the second confidence coefficient of each second candidate defect.

Description

Technical Field

[0001] The present invention relates to the technical field of semiconductor defect detection, and in particular to a semiconductor defect detection method, a semiconductor defect detection system, and a computer-readable storage medium. Background Art

[0002] In semiconductor defect detection equipment, detection speed and accuracy are key performance indicators. Traditional defect detection algorithms compare the die to be tested with several adjacent die, or with a standard die. These algorithms rely on rules and feature extraction, and are insufficient for detecting complex data and defects with unclear features, resulting in insufficient accuracy and robustness. Furthermore, while detection methods using common deep learning models are suitable for some scenarios, they are prone to missed detections and false positives when processing low-confidence detection frames, and have poor generalization capabilities for small sample defects. Both technologies struggle to meet the demands of complex scenarios in terms of accuracy, robustness, and adaptability.

[0003] In order to overcome the above-mentioned defects of the existing technology, the field urgently needs an improved semiconductor defect detection method to improve the accuracy and efficiency of semiconductor defect detection in complex scenarios while ensuring the speed and accuracy of semiconductor defect detection. Summary of the Invention

[0004] The following is a brief summary of one or more aspects to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceivable aspects and is neither intended to identify key or critical elements of all aspects nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that will be provided later.

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a semiconductor defect detection method, a semiconductor defect detection system and a computer-readable storage medium. By integrating the advantages of traditional algorithms and deep learning models, and combining routing selection algorithms to optimize the final detection frame, the accuracy and efficiency of semiconductor defect detection in complex scenarios can be improved, while ensuring the speed and accuracy of semiconductor defect detection.

[0006] Specifically, the above-mentioned semiconductor defect detection method provided according to the first aspect of the present invention includes the following steps: obtaining a detection image of the semiconductor device to be tested; performing target detection of preset defects on the detection image to determine at least one first candidate defect and its corresponding first confidence; performing feature comparison between the detection image and a preset template image to determine at least one second candidate defect and its corresponding second confidence; performing routing selection based on neural network reasoning on the detection image to determine its first score with respect to the target detection and its second score with respect to the feature comparison; and screening the defect detection results of the semiconductor device to be tested from each first candidate defect and each second candidate defect based on the first score, the second score, the first confidence of each first candidate defect, and the second confidence of each second candidate defect.

[0007] Furthermore, in some embodiments of the present invention, before performing the target detection, the feature comparison and the route selection on the detection image, the detection method also includes the following steps: performing a first preprocessing of resizing and / or normalizing the acquired detection image to adapt it to the first input feature dimension of the target detection; and / or performing a second preprocessing of resizing and / or normalizing the acquired detection image to adapt it to the second input feature dimension of the feature comparison; and / or performing a third preprocessing of resizing and / or normalizing the acquired detection image to adapt it to the third input feature dimension of the route selection.

[0008] Furthermore, in some embodiments of the present invention, the step of performing target detection of preset defects on the detection image to determine at least one first candidate defect and its corresponding first confidence level includes: inputting the detection image into a pre-trained target detection model to obtain the first coordinates and first confidence level of at least one first candidate defect outputted by it.

[0009] Furthermore, in some embodiments of the present invention, the step of training the target detection model includes: preparing a training sample set. The training sample set includes multiple inspection image samples of the semiconductor device to be tested, and the first real coordinates and real confidences of all defects in each of the inspection image samples; constructing the target detection model, and inputting each of the inspection image samples in the training sample set into the target detection model one by one to obtain the first predicted coordinates and predicted confidence of at least one defect output by the target detection model; and determining the loss function value of the target detection model based on the difference between the first predicted coordinates and the first real coordinates of each defect in each of the inspection image samples, and the difference between the corresponding predicted confidence and the real confidence, and modifying the learning parameters of the target detection model accordingly.

[0010] Furthermore, in some embodiments of the present invention, the step of performing feature comparison between the detection image and a preset template image to determine at least one second candidate defect and its corresponding second confidence level includes: inputting the detection image into a pre-debugged feature comparison model to obtain the second coordinates of at least one second candidate defect outputted by it; and determining the second confidence level of each second candidate defect based on a preset value determined by debugging.

[0011] Furthermore, in some embodiments of the present invention, the step of debugging the feature comparison model includes: preparing a training sample set and a defect-free template image of the semiconductor device to be tested. The training sample set includes multiple inspection image samples of the semiconductor device to be tested and the second true coordinates of all defects in each of the inspection image samples; constructing the feature comparison model, and inputting each of the inspection image samples in the training sample set into the feature comparison model one by one to obtain the second predicted coordinates of at least one defect whose grayscale difference with the template image is greater than a preset first threshold and whose size is greater than a preset second threshold; and respectively correcting the first threshold and the second threshold of the feature comparison model based on the difference between the second predicted coordinates and the second true coordinates of each defect in each of the inspection images.

[0012] Further, in some embodiments of the present invention, the step of screening the defect detection results of the semiconductor device to be tested from each first candidate defect and each second candidate defect based on the first score, the second score, the first confidence of each first candidate defect, and the second confidence of each second candidate defect includes: constructing a candidate defect list based on the at least one first candidate defect determined by the target detection and the at least one second candidate defect determined by the feature comparison; calculating the weighted sum of the first confidence, first score, second confidence and second score of each candidate defect in the candidate defect list one by one, and comparing it with a preset third threshold; in response to the weighted sum being greater than or equal to the third threshold, determining the corresponding first candidate defect and / or second candidate defect as a detected defect of the semiconductor device to be tested; and in response to the weighted sum being less than the third threshold, screening out the corresponding first candidate defect and / or second candidate defect from the candidate defect list.

[0013] Furthermore, in some embodiments of the present invention, before calculating the weighted sum value of each candidate defect, the detection method also includes the following steps: arranging each candidate defect in the candidate defect list in order of confidence from high to low; and selecting the candidate defect with the highest confidence and that has not been selected from the candidate defect list one by one as the target candidate defect, so as to calculate the intersection and union ratio of the candidate defect with the coordinate range of the remaining candidate defects in the candidate defect list, and merging the candidate defects in the candidate defect list whose intersection and union ratio is greater than a preset fourth threshold into the target candidate defect, until the calculation and screening of all candidate defects in the candidate defect list are completed.

[0014] Furthermore, in some embodiments of the present invention, the step of performing neural network inference-based routing selection on the detection image to determine its first score with respect to the target detection and its second score with respect to the feature comparison includes: performing convolution processing and / or pooling processing on the acquired detection image to extract image features therefrom; inputting the image features into a first prediction unit of a pre-trained routing selection model to determine its first score with respect to the target detection; and inputting the image features into a second prediction unit of the routing selection model to determine its second score with respect to the feature comparison.

[0015] Furthermore, in some embodiments of the present invention, the step of training the routing model includes: preparing a training sample set, wherein the training sample set includes a plurality of inspection image samples of the semiconductor device to be tested, and the third real coordinates and real scores of all defects in each inspection image sample; constructing the routing model, and inputting each inspection image sample in the training sample set into the routing model one by one to obtain a first score output by its first prediction unit and a second score output by its second prediction unit; constructing a candidate defect list for each inspection image in the third image set based on at least one first candidate defect determined by performing target detection on each inspection image in the training sample set and at least one second candidate defect determined by performing feature comparison on each inspection image in the training sample set; and calculating a weighted sum of the first confidence, first score, second confidence, and second score of each candidate defect in each candidate defect list as a predicted score for each candidate defect, and modifying the learning parameters of the routing model based on the difference between the predicted score and the corresponding real score.

[0016] Furthermore, the semiconductor defect detection system provided in accordance with the second aspect of the present invention includes a memory and a processor. The memory stores computer instructions. The processor is configured to execute the computer instructions stored in the memory to implement the semiconductor defect detection method provided in accordance with the first aspect of the present invention.

[0017] Furthermore, in some embodiments of the present invention, the processor includes a target detection unit, a feature comparison unit, a routing selection unit, and a result calculation unit. The target detection unit is used to perform target detection of preset defects on the acquired detection image to determine at least one first candidate defect and its corresponding first confidence level. The feature comparison unit is used to perform feature comparison between the detection image and a preset template image to determine at least one second candidate defect and its corresponding second confidence level. The routing selection unit is used to perform routing selection based on neural network reasoning on the detection image to determine its first score with respect to the target detection and its second score with respect to the feature comparison. The result calculation unit is used to filter the defect detection results of the semiconductor device to be tested from each of the first candidate defects and each of the second candidate defects based on the first score, the second score, the first confidence level of each of the first candidate defects, and the second confidence level of each of the second candidate defects.

[0018] Furthermore, the computer-readable storage medium provided in accordance with the third aspect of the present invention stores computer instructions, which, when executed by a processor, implement the semiconductor defect detection method provided in accordance with the first aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above features and advantages of the present invention will be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or characteristics may have the same or similar reference numerals.

[0020] Figure 1 A schematic flow chart of an offline training phase of a semiconductor defect detection method provided according to some embodiments of the present invention is shown.

[0021] Figure 2 A schematic diagram of a process for training an object detection model according to some embodiments of the present invention is shown.

[0022] Figure 3 A schematic diagram of a process for debugging a feature comparison model provided according to some embodiments of the present invention is shown.

[0023] Figure 4 A schematic diagram of a process for training a routing selection model according to some embodiments of the present invention is shown.

[0024] Figure 5 A schematic diagram illustrating the principles of a training routing model provided according to some embodiments of the present invention is shown.

[0025] Figure 6A schematic flow chart of an online detection phase of a semiconductor defect detection method provided according to some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0026] The following specific embodiments illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Although the description of the present invention will be introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to this embodiment. On the contrary, the purpose of introducing the invention in conjunction with the embodiment is to cover other options or modifications that may be extended based on the claims of the present invention. In order to provide a deep understanding of the present invention, the following description will include many specific details. The present invention can also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present invention, some specific details will be omitted in the description.

[0027] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0028] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood to refer to the orientations depicted in that section and the accompanying drawings. These relative terms are used solely for convenience of description and do not necessarily imply that the devices described herein must be manufactured or operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0029] It will be understood that although the terms "first," "second," "third," etc. may be used herein to describe various components, regions, layers, and / or portions, these components, regions, layers, and / or portions should not be limited by these terms, and these terms are merely used to distinguish different components, regions, layers, and / or portions. Thus, a first component, region, layer, and / or portion discussed below may be referred to as a second component, region, layer, and / or portion without departing from some embodiments of the present invention.

[0030] As mentioned above, in semiconductor defect detection equipment, detection speed and accuracy are key performance indicators. Traditional defect detection algorithms compare the die to be tested with several adjacent die, or with a standard die, relying on rules and feature extraction. They are insufficiently capable of detecting complex data and defects with unclear features, and their accuracy and robustness are insufficient. Furthermore, while detection methods using common deep learning models are suitable for some scenarios, they are prone to missed detections and false positives when processing low-confidence detection frames, and have weak generalization capabilities for small sample defects. Both types of technologies struggle to meet the requirements of complex scenarios in terms of accuracy, robustness, and adaptability.

[0031] In order to overcome the above-mentioned defects of the prior art, the present invention provides a semiconductor defect detection method, a semiconductor defect detection system and a computer-readable storage medium. By integrating the advantages of traditional algorithms and deep learning models, and combining routing selection algorithms to optimize the final detection frame, the accuracy and efficiency of semiconductor defect detection in complex scenarios can be improved, while ensuring the speed and accuracy of semiconductor defect detection.

[0032] In some non-limiting embodiments, the semiconductor defect detection system provided in the second aspect of the present invention may be implemented based on the semiconductor defect detection method provided in the first aspect of the present invention.

[0033] Specifically, in some non-limiting embodiments, the semiconductor defect detection system provided in the second aspect of the present invention includes a memory and a processor. The memory includes, but is not limited to, the computer-readable storage medium provided in the third aspect, having computer instructions stored thereon. The processor is connected to the memory and configured to execute the computer instructions stored in the memory to implement the semiconductor defect detection method provided in the first aspect of the present invention.

[0034] Furthermore, the above-mentioned processor includes a target detection unit, a feature comparison unit, a routing selection unit and a result calculation unit. Here, the target detection unit is used to perform target detection of preset defects on the acquired detection image to determine at least one first candidate defect and its corresponding first confidence level. The feature comparison unit is used to perform feature comparison between the detection image and a preset template image to determine at least one second candidate defect and its corresponding second confidence level. The routing selection unit is used to perform routing selection based on neural network reasoning on the detection image to determine its first score for target detection and its second score for feature comparison. The result calculation unit is used to screen the defect detection results of the semiconductor device to be tested from each first candidate defect and each second candidate defect based on the first score, the second score, the first confidence level of each first candidate defect, and the second confidence level of each second candidate defect.

[0035] Furthermore, in some embodiments of the present invention, the semiconductor defect detection method may be divided into two parts: an offline model training phase and an online semiconductor defect detection phase, and these two parts may be independently performed by different actors.

[0036] The following will first describe the working principle of the above-mentioned semiconductor defect detection system in conjunction with some examples of the offline training stage of the detection model. Those skilled in the art will understand that these detection method embodiments are only some non-limiting implementation methods provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than limiting the full functions or all working methods of the semiconductor defect detection system. Similarly, the semiconductor defect detection system is also only a non-limiting implementation method provided by the present invention, and does not constitute a limitation on the execution subject and execution order of each step in these detection methods.

[0037] Please refer to Figure 1 . Figure 1 A schematic flow chart of an offline training phase of a semiconductor defect detection method provided according to some embodiments of the present invention is shown.

[0038] like Figure 1 As shown, the semiconductor defect detection system provided by the second aspect of the present invention can obtain a training sample set and perform image preprocessing on it. The detection system can then use the training sample set to train an object detection model and debug a feature comparison model. Furthermore, the detection system can then train a routing model based on the object detection model and feature comparison model.

[0039] First, please refer to Figure 2 . Figure 2 A schematic diagram of a process for training an object detection model according to some embodiments of the present invention is shown.

[0040] like Figure 2 As shown, the semiconductor defect detection system provided by the second aspect of the present invention can first train a target detection model (for example, a deep learning model such as Yolov8) in an offline training process.

[0041] Specifically, the inspection system may first prepare a training sample set, which includes a plurality of inspection image samples of the semiconductor device to be inspected, and the first real coordinates and real confidence (ie, 1) of all defects in each inspection image sample.

[0042] Then, in some preferred embodiments, the detection system may first perform image preprocessing on the detection image samples in the training sample set. Specifically, the detection system may perform a first preprocessing of resizing and / or normalizing the acquired detection image samples to adapt them to the first input feature dimension of target detection.

[0043] Thereafter, the detection system may construct a target detection model, and input each detection image sample in the training sample set into the target detection model one by one, so as to respectively obtain the first predicted coordinates and prediction confidence of at least one defect outputted by the model.

[0044] Afterwards, the detection system can determine the loss function value of the target detection model based on the difference between the first predicted coordinates and the first true coordinates of each defect in each detection image sample, as well as the difference between its corresponding predicted confidence and the true confidence, and correct the learning parameters of the target detection model accordingly.

[0045] Next, please refer to Figure 3 . Figure 3 A schematic diagram of a process for debugging a feature comparison model provided according to some embodiments of the present invention is shown.

[0046] like Figure 3 As shown, the semiconductor defect detection system provided by the second aspect of the present invention can also debug the feature comparison model (for example, grayscale comparison model) during the offline training process.

[0047] Specifically, the inspection system can prepare a training sample set and a defect-free template image of the semiconductor device to be inspected. Here, the training sample set includes multiple inspection image samples of the semiconductor device to be inspected and the second real coordinates of all defects in each inspection image sample.

[0048] Then, in some preferred embodiments, the detection system may first perform image preprocessing on the detection image samples in the training sample set. Specifically, the detection system may perform a second preprocessing of resizing and / or normalizing the acquired detection image samples to adapt them to the second input feature dimension of the feature comparison.

[0049] Afterwards, the detection system can construct a feature comparison model and input each detection image sample in the training sample set into the feature comparison model one by one to obtain the second predicted coordinates of at least one defect whose grayscale difference with the template image is greater than a preset first threshold (for example: 20) and whose size is greater than a preset second threshold (for example: 5 pixels).

[0050] Thereafter, the detection system may respectively correct the first threshold and the second threshold of the feature comparison model according to the difference between the second predicted coordinates and the second real coordinates of each defect in each detection image.

[0051] Please refer to Figure 4 and Figure 5 . Figure 4 A schematic diagram of a process for training a routing selection model according to some embodiments of the present invention is shown. Figure 5 A schematic diagram illustrating the principles of a training routing model provided according to some embodiments of the present invention is shown.

[0052] like Figure 4 As shown, after training the above-mentioned target detection model and debugging the above-mentioned feature comparison model, the above-mentioned semiconductor defect detection system provided by the second aspect of the present invention can train the routing selection model in the offline training process based on the above-mentioned target detection model and feature comparison model.

[0053] Specifically, the inspection system may first prepare a training sample set, which includes a plurality of inspection image samples of the semiconductor device to be inspected, and the third true coordinates and true scores (ie, 1) of all defects in each inspection image sample.

[0054] Then, in some preferred embodiments, the detection system may first perform image preprocessing on the detection image samples in the training sample set. Specifically, the detection system may perform a third preprocessing of resizing and / or normalizing the acquired detection image samples to adapt them to the third input feature dimension of the routing selection.

[0055] Thereafter, the detection system may construct a routing selection model and input each detection image sample in the training sample set into the routing selection model one by one to obtain the first score output by its first prediction unit and the second score output by its second prediction unit respectively.

[0056] Afterwards, the detection system can construct a candidate defect list for each detection image in the third image set based on at least one first candidate defect determined by target detection of each detection image in the training sample set, and at least one second candidate defect determined by feature comparison of each detection image in the training sample set.

[0057] Afterwards, in some preferred embodiments, the detection system may screen each candidate defect in the candidate defect list based on a non-maximum suppression algorithm (NMS).

[0058] Specifically, the detection system may first arrange the candidate defects in the candidate defect list in descending order of confidence. For example, in the candidate defect list, A(0.9)>B(0.8)>C(0.7)>D(0.6)>E(0.5).

[0059] Afterwards, the inspection system can select the candidate defect with the highest confidence and that has not been selected from the candidate defect list one by one as the target candidate defect (for example, A), calculate the intersection over union (IoU) of its coordinate range with the coordinate range of the remaining candidate defects in the selected defect list, and merge the candidate defects in the candidate defect list whose IoU is greater than a preset fourth threshold (for example, 0.6) into the target candidate defect until the calculation and screening of all candidate defects in the candidate defect list are completed. Here, the merged defect inherits the coordinates of the target candidate defect and has both the first confidence and second confidence of the two defects before the merger.

[0060] Afterwards, the detection system can calculate the weighted sum of the first confidence, first score, second confidence and second score of each candidate defect in each candidate defect list one by one as the predicted score of each candidate defect, and correct the learning parameters of the routing selection model based on the difference between the predicted score and the corresponding true score.

[0061] Here, the detection system can use the mean squared error (MSE) to determine the loss function value of the above predicted score and the actual score:

[0062]

[0063] Among them, y i For the real score, is the prediction score, and n is the total number of candidate defects.

[0064] Here, among the above-mentioned candidate defects, for the first candidate defect determined only by target detection, its second confidence is 0, and for the second candidate defect determined only by feature comparison, its first confidence is 0. Only the common candidate defects determined by target detection and feature comparison have both a first confidence and a second confidence greater than 0.

[0065] By following the above steps, a person skilled in the art can obtain a trained object detection model, feature comparison model, and routing selection model. The person can then store the trained model on a CD, disk, network drive, or other storage medium for access and use by the online inspection module of a semiconductor defect detection system.

[0066] The following will further describe the working principle of the above-mentioned semiconductor defect detection system in conjunction with some examples of online detection of semiconductor defects based on the above-mentioned detection model. Those skilled in the art will understand that these detection method examples are only some non-limiting implementation methods provided by the present invention, and are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit the full functions or all working methods of the detection system. Similarly, the semiconductor defect detection system is also only a non-limiting implementation method provided by the present invention, and does not constitute a limitation on the execution subject or execution order of each step in these semiconductor defect detection methods.

[0067] Please refer to Figure 6 . Figure 6 A schematic flow chart of an online detection phase of a semiconductor defect detection method provided according to some embodiments of the present invention is shown.

[0068] like Figure 6 As shown, the above detection system can first obtain a detection image of the semiconductor device to be detected.

[0069] Afterwards, in some preferred embodiments, before performing target detection on the detection image, the detection system may perform a first preprocessing of resizing and / or normalizing the acquired detection image to adapt it to the first input feature dimension of target detection.

[0070] The inspection system can then perform target detection for a preset defect on the inspection image via the target detection unit to determine at least one first candidate defect and its corresponding first confidence level. Specifically, the inspection system can input the inspection image into the previously trained target detection model to obtain the first coordinates and first confidence level of the at least one first candidate defect outputted by the target detection model.

[0071] Then, in some preferred embodiments, before performing feature comparison on the detection image, the detection system may perform a second preprocessing of resizing and / or normalizing the acquired detection image to adapt it to the second input feature dimension of the feature comparison.

[0072] The inspection system can then use a feature comparison unit to perform a feature comparison between the inspection image and a preset template image to determine at least one second candidate defect and its corresponding second confidence level. Specifically, the inspection system can input the inspection image into the previously trained feature comparison model to obtain the second coordinates of the at least one second candidate defect outputted by the model. The inspection system can then determine the second confidence level for each second candidate defect based on preset values ​​determined during debugging.

[0073] Here, the second confidence level can be adjusted and determined based on the defect type of the semiconductor device under test. If the target defect has a complex texture, the second confidence level can be lowered (for example, to 0.6) to prioritize the target detection results. Conversely, if the target defect has a unique morphology and there are fewer defect samples, the second confidence level can be increased (for example, to 0.8) to prioritize the feature comparison results.

[0074] Then, in some preferred embodiments, before routing the detection image, the detection system may perform a third preprocessing of resizing and / or normalizing the acquired detection image to adapt it to the third input feature dimension of routing (for example: 1024×1024×3).

[0075] Thereafter, the detection system may perform routing selection based on neural network reasoning on the detection image via the routing selection unit to determine its first score score1 regarding target detection and its second score score2 regarding feature comparison.

[0076] Specifically, the detection system may perform convolution and / or pooling processing on the acquired detection image to extract image features. The detection system may then input the image features into the first prediction unit of the routing model trained previously via a fully connected layer to determine its first score (score1) for target detection via a softmax function. Furthermore, the detection system may input the image features into the second prediction unit of the routing model via a fully connected layer to determine its second score (score2) for feature alignment via a softmax function.

[0077] Thereafter, the detection unit may filter the defect detection results of the semiconductor device to be tested from each first candidate defect and each second candidate defect through the result calculation unit according to the first score score1, the second score score2, the first confidence of each first candidate defect, and the second confidence of each second candidate defect.

[0078] Specifically, the detection system may construct a candidate defect list based on at least one first candidate defect determined by target detection and at least one second candidate defect determined by feature comparison.

[0079] Afterwards, in some preferred embodiments, the detection system may screen each candidate defect in the candidate defect list based on a non-maximum suppression algorithm (NMS).

[0080] Specifically, the detection system may first arrange the candidate defects in the candidate defect list in descending order of confidence. For example, in the candidate defect list, A(0.9)>B(0.8)>C(0.7)>D(0.6)>E(0.5).

[0081] Afterwards, the inspection system can select the candidate defect with the highest confidence and that has not been selected from the candidate defect list one by one as the target candidate defect (for example, A), calculate the intersection over union (IoU) of its coordinate range with the coordinate range of the remaining candidate defects in the selected defect list, and merge the candidate defects in the candidate defect list whose IoU is greater than a preset fourth threshold (for example, 0.6) into the target candidate defect until the calculation and screening of all candidate defects in the candidate defect list are completed. Here, the merged defect inherits the coordinates of the target candidate defect and has both the first confidence and second confidence of the two defects before the merger.

[0082] Thereafter, the detection system may calculate the weighted sum of the first confidence, the first score score1, the second confidence and the second score score2 of each candidate defect in the candidate defect list one by one, and compare it with a preset third threshold (eg, 0.5).

[0083] confidence=score1×confidence1+score2×confidence2

[0084] For example, the first score score1 = 0.9, the first confidence confidence1 = 0.5, the second score score2 = 0.1, and the second confidence confidence2 = 0.8. The calculated weighted sum value confidence = 0.9 × 0.5 + 0.1 × 0.8 = 0.53, which is considered to be the correct target.

[0085] Here, among the above-mentioned candidate defects, for the first candidate defect determined only by target detection, its second confidence is 0, and for the second candidate defect determined only by feature comparison, its first confidence is 0. Only the common candidate defects determined by target detection and feature comparison have both a first confidence and a second confidence greater than 0.

[0086] Thereafter, in response to the weighted sum value being greater than or equal to a third threshold, the inspection system may determine the corresponding first candidate defect and / or second candidate defect as a detected defect of the semiconductor device to be tested.

[0087] On the contrary, in response to the weighted sum value being less than the third threshold, the inspection system may screen out the corresponding first candidate defect and / or second candidate defect from the candidate defect list.

[0088] In summary, the semiconductor defect detection method, semiconductor defect detection system and computer-readable storage medium provided by the present invention can all optimize the final detection frame by integrating the advantages of traditional algorithms and deep learning models and combining routing selection algorithms to improve the accuracy and efficiency of semiconductor defect detection in complex scenarios, while ensuring the speed and accuracy of semiconductor defect detection.

[0089] Although the above methods are illustrated and described as a series of acts for simplicity of explanation, it is to be understood and appreciated that these methods are not limited by the order of the acts, as some acts may occur in a different order and / or concurrently with other acts from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art according to one or more embodiments.

[0090] Those skilled in the art will appreciate that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips cited throughout the foregoing description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0091] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. A skilled person may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0092] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0093] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside in a user terminal as discrete components.

[0094] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0095] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting semiconductor defects, characterized in that: The following steps are involved: Acquiring a test image of the semiconductor device to be tested; Performing target detection of preset defects on the inspection image to determine at least one first candidate defect and its corresponding first confidence level; Performing feature comparison on the inspection image and a preset template image to determine at least one second candidate defect and its corresponding second confidence level; Performing neural network inference-based routing on the detection image to determine a first score thereof with respect to the target detection and a second score thereof with respect to the feature comparison; as well as The defect detection result of the semiconductor device to be tested is screened from the first candidate defects and the second candidate defects according to the first score, the second score, the first confidence of each first candidate defect, and the second confidence of each second candidate defect.

2. The detection method according to claim 1, wherein Before performing the target detection, feature comparison, and route selection on the detection image, the detection method further includes the following steps: Performing a first preprocessing of resizing and / or normalizing the acquired detection image to adapt it to a first input feature dimension of the target detection; performing a second preprocessing of resizing and / or normalizing the acquired detection image to adapt it to a second input feature dimension of the feature comparison; A third pre-processing of resizing and / or normalizing the acquired detection image is performed to adapt the image to the third input feature dimension of the routing selection.

3. The detection method according to claim 1, wherein The step of performing target detection of preset defects on the detection image to determine at least one first candidate defect and its corresponding first confidence level includes: The detection image is input into a pre-trained target detection model to obtain the first coordinates and first confidence of at least one first candidate defect outputted by the model.

4. The detection method according to claim 3, wherein The steps of training the target detection model include: Preparing a training sample set, wherein the training sample set includes a plurality of inspection image samples of the semiconductor device to be tested, and first real coordinates and real confidences of all defects in each of the inspection image samples; Constructing the target detection model, and inputting each of the detection image samples in the training sample set into the target detection model one by one, so as to respectively obtain the first predicted coordinates and prediction confidence of at least one defect outputted by the model; and According to the difference between the first predicted coordinates and the first true coordinates of each defect in each detection image sample, and the difference between its corresponding predicted confidence and the true confidence, the loss function value of the target detection model is determined respectively, and the learning parameters of the target detection model are corrected accordingly.

5. The detection method according to claim 1, wherein The step of performing feature comparison between the detection image and a preset template image to determine at least one second candidate defect and its corresponding second confidence level includes: Inputting the inspection image into a pre-debugged feature comparison model to obtain the second coordinates of at least one second candidate defect outputted by the model; and A second confidence level of each of the second candidate defects is determined according to a preset value determined during debugging.

6. The detection method according to claim 5, wherein The steps of debugging the feature comparison model include: Preparing a training sample set and a defect-free template image of the semiconductor device to be tested, wherein the training sample set includes multiple inspection image samples of the semiconductor device to be tested and the second real coordinates of all defects in each of the inspection image samples; Constructing the feature comparison model and inputting each of the inspection image samples in the training sample set into the feature comparison model one by one to obtain second predicted coordinates of at least one defect whose grayscale difference with the template image is greater than a preset first threshold and whose size is greater than a preset second threshold; and According to the difference between the second predicted coordinates and the second real coordinates of each defect in each detection image, the first threshold and the second threshold of the feature comparison model are respectively corrected.

7. The detection method according to claim 1, wherein The step of screening the defect detection result of the semiconductor device to be tested from each of the first candidate defects and each of the second candidate defects according to the first score, the second score, the first confidence level of each of the first candidate defects, and the second confidence level of each of the second candidate defects includes: constructing a candidate defect list based on the at least one first candidate defect determined by the target detection and the at least one second candidate defect determined by the feature comparison; Calculating the first confidence, the first score, the weighted sum of the second confidence and the second score of each candidate defect in the candidate defect list one by one, and comparing the sum with a preset third threshold; In response to the weighted sum value being greater than or equal to the third threshold, determining the corresponding first candidate defect and / or second candidate defect as a detected defect of the semiconductor device under test; and In response to the weighted sum value being less than the third threshold, the corresponding first candidate defect and / or second candidate defect is screened out from the candidate defect list.

8. The detection method according to claim 7, wherein Before calculating the weighted sum of the candidate defects, the detection method further includes the following steps: Arrange the candidate defects in the candidate defect list in descending order of confidence; and Select the candidate defects with the highest confidence and that have not been selected from the candidate defect list one by one as the target candidate defects, calculate the intersection-and-union ratio of the coordinate range between the candidate defects and the remaining candidate defects in the selected defect list, and merge the candidate defects in the candidate defect list whose intersection-and-union ratio is greater than a preset fourth threshold into the target candidate defect until the calculation and screening of all candidate defects in the candidate defect list are completed.

9. The detection method according to claim 1, wherein The step of performing neural network inference-based routing on the detection image to determine a first score thereof with respect to the target detection and a second score thereof with respect to the feature comparison comprises: Performing convolution processing and / or pooling processing on the acquired detection image to extract image features therefrom; Inputting the image features into a first prediction unit of a pre-trained routing model to determine a first score thereof for detecting the object; and The image features are input into a second prediction unit of the routing model to determine a second score thereof with respect to the feature alignment.

10. The detection method according to claim 9, wherein The steps of training the routing model include: Preparing a training sample set, wherein the training sample set includes a plurality of inspection image samples of the semiconductor device to be tested, and the third true coordinates and true scores of all defects in each of the inspection image samples; Constructing the routing selection model, and inputting each of the detection image samples in the training sample set into the routing selection model one by one to respectively obtain a first score output by its first prediction unit and a second score output by its second prediction unit; constructing a candidate defect list for each of the inspection images in the third image set based on at least one first candidate defect determined by performing the target detection on each of the inspection images in the training sample set, and at least one second candidate defect determined by performing the feature comparison on each of the inspection images in the training sample set; and Calculate the first confidence, first score, second confidence and second score weighted sum of each candidate defect in the candidate defect list one by one as the predicted score of each candidate defect, and correct the learning parameters of the routing model according to the difference between the predicted score and the corresponding true score.

11. A semiconductor defect detection system, characterized in that: include: a memory having computer instructions stored thereon; as well as A processor is configured to execute computer instructions stored in the memory to implement the semiconductor defect detection method according to any one of claims 1 to 10.

12. The detection system according to claim 11, wherein: The processor includes: an object detection unit, configured to perform object detection of preset defects on the acquired inspection image to determine at least one first candidate defect and its corresponding first confidence level; a feature comparison unit, configured to perform feature comparison on the detection image and a preset template image to determine at least one second candidate defect and its corresponding second confidence level; a routing selection unit, configured to perform routing selection based on neural network reasoning on the detection image to determine a first score thereof with respect to the target detection and a second score thereof with respect to the feature comparison; and A result calculation unit is used to filter the defect detection result of the semiconductor device to be tested from each of the first candidate defects and each of the second candidate defects based on the first score, the second score, the first confidence of each of the first candidate defects, and the second confidence of each of the second candidate defects.

13. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the semiconductor defect detection method according to any one of claims 1 to 10 is implemented.

Images