Semiconductor defect analysis method and system for rapid positioning and joint semantic interpretation
By building the defect image diagnosis network of Faster R-CNN and LSTM decoder, the problem of difficulty in obtaining semantic information in wafer defect image analysis is solved, and the detailed positioning and description of wafer defects is achieved, and the accuracy of analysis and identification is improved. Sex and richness.
Patent Information
- Application Number
- CN202410976279.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-20
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-07-20
AI Technical Summary
The prior art is difficult to effectively obtain semantic information in wafer defect image analysis, and ignores the interactive characteristics of defects and background, resulting in insufficient detailed and accurate defect description, and it is difficult to support cross-modal retrieval, fault warning and diagnostic reports generation.
Using a semiconductor defect analysis method with fast positioning combined semantic explanation, a defect image diagnosis network consisting of Faster R-CNN encoder and an attention mechanism-based LSTM decoder is obtained and detailed diagnosis is generated. Reports, including defect location, shape, size, concaveness, process layer, and background information.
It improves the accuracy and richness of visual semantic embedding, can quickly locate and describe defects in the image, generate more accurate and detailed diagnostic reports, and improves the analysis and identification of wafer defects.
Smart Images

Figure CN118982499A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of semiconductor defect analysis, and in particular relates to a semiconductor defect analysis method and system for rapid positioning and combined semantic interpretation. Background Art
[0002] In recent years, problems combining image and language understanding, such as image captioning and visual question answering, have attracted a lot of research in the fields of computer vision and natural language processing. However, in the field of automated wafer image analysis, the acquisition of semantic information remains a complex and thorny problem. Due to the rigor of industrial production inspection, wafer defect image captions need to use coherent, concise professional language descriptions to accurately summarize the specific process layers of specific defects. This not only requires computer vision knowledge to identify dominant defects and detect the relationship between different defective parts and backgrounds in the same image, but also requires the model to parse the identified image features with sentences that are as clear as possible in grammar and as rich in vocabulary as possible. This defect image description task provides more specific and semantic information than traditional visual tasks, and provides stronger technical support in applications such as cross-modal retrieval, fault warning, and diagnostic report generation.
[0003] At present, the existing research on wafer defect images is still focused on defect classification, target recognition, segmentation, etc. These works usually only identify defects in defective SEM images and obtain defect class labels, ignoring other easily ignored features of defects, information interaction with the background, etc. The purpose of this invention is to generate a specified comprehensive sentence that summarizes the image content at the semantic level. Compared with traditional encoders, regional features can provide object-level information, identify the most prominent areas in the image and represent them by feature vectors, which can greatly reduce the difficulty of visual semantic embedding. Summary of the invention
[0004] In order to solve the problems in the prior art, the present invention provides a semiconductor defect analysis method and system for rapid positioning and joint semantic interpretation.
[0005] The technical solution adopted by the present invention is as follows:
[0006] In a first aspect, the present invention discloses a semiconductor defect analysis method for rapid positioning and combined semantic interpretation, comprising:
[0007] Acquire a wafer defect image diagnosis dataset including a wafer defect SEM image and a diagnosis report, wherein the diagnosis report includes a plurality of sentences for different defect diagnosis tasks;
[0008] A defect image diagnosis network is constructed, wherein the defect image diagnosis network includes a Faster R-CNN encoder and an LSTM decoder based on an attention mechanism for different defect diagnosis tasks; a wafer defect image diagnosis data set is used to train the defect image diagnosis network, a wafer defect SEM image is input into the Faster R-CNN encoder, a feature vector of the wafer defect SEM image is obtained, the feature vector is gradually decoded by the LSTM decoder to generate a word vector representation of a word corresponding to a corresponding defect diagnosis task, a word is generated according to the word vector representation, and all the words are sequentially spliced to form a sentence of the defect diagnosis task; and sentences of different defect diagnosis tasks are combined to obtain a diagnosis report;
[0009] During the training process of the defect image diagnosis network, the sentence of the defect diagnosis task is used as the input of the LSTM decoder to guide the LSTM decoder to generate the target sentence;
[0010] The trained defect image diagnosis network is used to analyze the SEM image of wafer defects to be processed and generate a diagnosis report.
[0011] Furthermore, the defect diagnosis task includes:
[0012] Defect location, shape, and size diagnosis tasks;
[0013] The task of diagnosing the convexity and deformation degree of defects;
[0014] The task of diagnosing the process layer where the defect is located;
[0015] The background and composition information diagnosis task of the defect;
[0016] Defect category diagnosis task;
[0017] Wherein, the defect diagnosis task includes at least one of the above tasks.
[0018] Furthermore, the sentences of the defect diagnosis tasks in the diagnosis report simultaneously meet the following conditions:
[0019] Except for the sentences corresponding to the defect category diagnosis task, a sentence should contain at least six words;
[0020] Do not use ambiguous words and grammar;
[0021] The content of the sentence contains all the flaws.
[0022] Furthermore, the wafer defect SEM images in the wafer defect image diagnosis data set are obtained by synthesizing wafer defect SEM images taken at different angles.
[0023] Furthermore, before training the defect image diagnosis network, the sentences in the diagnosis report need to be preprocessed, including:
[0024] Convert all uppercase letters in the sentence to lowercase letters, remove all punctuation marks in the sentence and perform word segmentation, convert each sentence into a list containing multiple words, and save the processing results in a dictionary. The key in the dictionary is the ID of the wafer defect SEM image, and the value in the dictionary is a list containing multiple words.
[0025] All sentences in the diagnosis report are subjected to the same preprocessing.
[0026] In a second aspect, the present invention discloses a semiconductor defect analysis system for rapid positioning and combined semantic interpretation, comprising:
[0027] A data acquisition module, which is used to acquire a wafer defect image diagnosis data set in a training phase, the wafer defect image diagnosis data set including a wafer defect SEM image and a diagnosis report, wherein the diagnosis report includes a plurality of sentences for different defect diagnosis tasks; and, in a prediction phase, to acquire a wafer defect SEM image to be analyzed;
[0028] A defect image diagnosis network module includes a Faster R-CNN encoder and an LSTM decoder based on an attention mechanism for different defect diagnosis tasks; a wafer defect SEM image is input into the Faster R-CNN encoder to obtain a feature vector of the wafer defect SEM image, the LSTM decoder is used to gradually decode the feature vector to generate a word vector representation of a word corresponding to a corresponding defect diagnosis task, a word is generated according to the word vector representation, and all the words are sequentially concatenated to form a sentence of the defect diagnosis task; sentences of different defect diagnosis tasks are combined to obtain a diagnosis report;
[0029] A network training module is used to train a defect image diagnosis network using a wafer defect image diagnosis dataset. During the training of the defect image diagnosis network, the sentence of the defect diagnosis task needs to be used as the input of the LSTM decoder to guide the LSTM decoder to generate the target sentence.
[0030] The defect analysis module is used to analyze the wafer defect SEM image to be processed using the trained defect image diagnosis network and generate a diagnosis report.
[0031] Compared with the prior art, the present invention has the following beneficial effects:
[0032] (1) Faster-Rcnn is applied to encoders in the field of image captioning, so that attention can be calculated on objects and other important image regions. In the method of the present invention, each region of the image region of the selected target box has an associated feature vector, and the feature weight is determined by Faster-Rcnn, which extracts richer information than the traditional encoder with fixed feature size.
[0033] (2) Faster-RCNN is an object detection model that aims to identify object instances belonging to a specific category and locate them using bounding boxes. While outputting a description of each SEM image, it can quickly locate one or more defects in the image, their corresponding labels and their probabilities.
[0034] (3) In order to obtain multi-scale information, the three grayscale images collected, which are illuminated from 45 degrees to the left, directly above, and 45 degrees to the right, correspond to the three-channel information of the encoder's initial input image.
[0035] (4) Compared with the traditional convolutional neural network (CNN) image modeling and Vision transformer (ViT), the present invention adopts the method of faster-rcnn target detection to extract the target frame in the encoding stage. For each image, the target frame is first screened, which establishes a closer connection between vision and language tasks and the latest progress in target detection. The accuracy of the output sentence can be greatly improved compared with the traditional method. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A flowchart of the semiconductor defect analysis method for rapid positioning combined with semantic interpretation of the present invention;
[0037] Figure 2 is an example of an image sample;
[0038] Figure 3 This is the specific flow chart of the defect image coding network (Faster R-CNN);
[0039] Figure 4 An example of the target box output result for the image detection network Faster R-CNN;
[0040] Figure 5 This is the Attention on LSTM module diagram. DETAILED DESCRIPTION
[0041] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.
[0042] like Figure 1 and Figure 2 As shown, the semiconductor defect analysis method for rapid positioning combined with semantic interpretation proposed by the present invention mainly includes the following steps:
[0043] Obtain wafer defect image diagnosis data set, including wafer defect SEM images and diagnosis reports;
[0044] The wafer defect image diagnosis dataset used in this embodiment is produced from a 55nm real wafer production line, including 1475 defects, 4425 high-resolution grayscale images of wafer defects at the same position taken from the left, right, and top, and 7375 original sentences, covering the 7 major types of defects commonly seen in the chip manufacturing process. It is currently the first dataset constructed in the field of wafer defect subtitle processing. The present invention collects scenes on a variety of different background process layers for each defect of the same category, so that the wafer defect image diagnosis dataset has stronger richness and scalability. The wafer defect image diagnosis dataset provides a comprehensive description of five directions for each defect, and the diagnostic report content involved includes the location, shape, size, concavity, degree of deformation, process layer, and further description of the background of the defect itself. The spatial resolution of the defect is distributed in the range of about 0.1 to 1um / pixel.
[0045] Follow these instructions when constructing the image description sentences for the diagnostic report:
[0046] (1) Except for the fifth sentence which is used to define the category of image defects, a sentence contains more than six words.
[0047] (2) Describe all visible defects of the input defect image, one or more.
[0048] (3) Do not use ambiguous word descriptions and grammar.
[0049] (4) The first sentence describes the location, shape, and size of the defect.
[0050] (5) The second sentence describes the convexity and deformation degree of the defect.
[0051] (6) The third sentence describes the process layer where the defect is located.
[0052] (7) The fourth sentence describes the background and composition information of the defect.
[0053] (8) Encourage the use of richer vocabulary and more flexible sentence structures.
[0054] Figure 2This is an example of some image samples. Each image sample contains a SEM image of a wafer defect and a five-sentence diagnostic report. The original file is named SSID.token.txt.
[0055] For example, Figure 2 Take the image in the lower left corner as an example. The five sentences in its diagnosis report are as follows:
[0056] 1) An irregular elliptical defect in the center.
[0057] 2) The defect is protruding and severely deformed.
[0058] 3) This defect occurs during the contact layer tungsten deposition-chemical mechanical polishing process steps.
[0059] 4) The background is darker and consists of many small blocks arranged neatly.
[0060] 5) This is a pit defect.
[0061] In terms of data set processing, the original description of the data set SSID.token.txt ("image ID address#title number+space+description") was traversed five times, and each sentence was converted into a list containing multiple words. The result was saved in a new dictionary. Among them, the key is the image ID in the original dictionary, and the value is the processed word list. The final generated wordmap.json, together with two json format files recording captions and caplens information of the training and verification sets, and hdf5 format files recording images information, were used for training and testing in subsequent models.
[0062] A defect image diagnosis network is constructed, wherein the defect image diagnosis network includes a Faster R-CNN encoder and a decoder, and the decoder adopts an LSTM based on an attention mechanism; a wafer defect image diagnosis data set is used to train the defect image diagnosis network, a wafer defect SEM image is input into the Faster R-CNN encoder, a feature vector of the wafer defect SEM image is obtained, the feature vector and the previous output of the decoder are input into the decoder, and a word vector representation of the word obtained by this decoding is obtained; the word vector representations of all the words generated by the decoder are sequentially spliced to form a sentence of a defect diagnosis task; and sentences of different defect diagnosis tasks are combined to obtain a diagnosis report;
[0063] During the training process of the defect image diagnosis network, the sentence of the defect diagnosis task is used as the input of the LSTM decoder to guide the LSTM decoder to generate the target sentence;
[0064] The trained defect image diagnosis network is used to analyze the SEM image of wafer defects to be processed and generate a diagnosis report.
[0065] Compared with traditional CNN image modeling, the present invention extracts image regions based on a bottom-up mechanism (based on Faster R-CNN), each of which is encoded as an associated feature vector, and the feature weights are determined by a top-down mechanism. Since Faster R-CNN is a two-stage detector, two region selections are required during the processing. The first stage is called the Region Proposal Network (RPN). RPN is a small network that slides on the intermediate feature layer of CNN to predict object scores and bounding box adjustments for each spatial location. These bounding boxes are based on predefined anchor boxes. The best candidate regions are screened by the intersection over union (IoU) threshold using the non-maximum suppression (NMS) technique, which will serve as the input of the second stage. In the second stage, the region of interest (RoI) pooling technique is applied to each candidate region to extract fixed-size feature maps, which are then fed into the last few layers of CNN for further processing. The final output of the model includes a softmax distribution over the class labels and class-specific bounding box refinements for each box proposal.
[0066] The present invention combines Faster R-CNN with ResNet-101. ResNet-101 is a deep residual network that can alleviate the gradient vanishing problem through jump connections and improve the training efficiency of deep networks. By combining Faster R-CNN with ResNet-101, the model can utilize the powerful feature extraction capabilities of ResNet-101 to further improve the accuracy of target detection. For each selected area, its mean-pooled convolutional features are extracted, and the dimension of the feature vector is 2048, providing a rich bottom-up feature representation for the image. Faster R-CNN plays the role of a "hard" attention mechanism in the present invention. It selectively focuses on the key areas in the image and filters out the most representative bounding box features from a large number of possible candidate areas. This mechanism effectively reduces the computational burden while retaining the visual information that is most important for image description. The specific method flow is as follows Figure 3 It shows in detail the complete process from image input to feature extraction, region selection, and feature encoding.
[0067] like Figure 3As shown in the figure, the processing process of the encoder is: the image is input into the encoder, and the encoder part of the network adopts the Faster R-CNN structure. For a given input image, first, Faster R-CNN uses a pre-trained convolutional neural network (ResNet101) as the backbone network to forward propagate the input image and extract the feature map (FeatureMap); wherein the pre-trained ResNet-101 is obtained by pre-training ResNet-101 with the ImageNet dataset. These feature maps capture the multi-scale and multi-level semantic information of the image. Specifically, when an input image is given (for example, a size of 224×224×3), the image is first convolved and pooled using the lower blocks of ResNet until the C4 block. This generates a 14×14×1024 feature map that contains high-level semantic information of the input image. As shown Figure 4 As shown in the figure, on the feature map, the region proposal network (RPN) generates multiple regions of interest (RoIs) by sliding windows. RPN uses the feature map extracted by the encoder to predict the position and target category of each sliding window. This includes using multiple anchor boxes (Anchor Boxes) of different scales and shapes to cover each position of the feature map. RPN predicts two outputs for each anchor box, one is a binary classification problem (foreground or background), and the other is a bounding box regression problem (the offset of the predicted box). By setting thresholds and non-maximum suppression (NMS), high-quality candidate regions are selected.
[0068] For each candidate region generated by RPN, Faster R-CNN converts it into a fixed-size feature using a RoI pooling layer. The RoI pooling layer maps candidate regions of different sizes to feature vectors with the same dimensions, typically a W×H feature map, where W is the width, H is the height, and W and H are fixed values (i.e., preset). Specifically, features are calculated separately for each region, and the C5 block of ResNet-101 is applied to the features after the 14×14RoIPool operation. The output of C5 is then average-pooled by AvgPool into the final vector of each region as a bottom-up feature. Since all final region features come from C5, the detector can be converted back to a ResNet classifier, and the same layer C5 is used as the output feature of the present invention. After RoI pooling, the feature map of each candidate region is converted into a vector of fixed length and then fed into a subsequent decoder for further feature extraction.
[0069] In the present invention, the Faster R-CNN encoder first uses a pre-trained ResNet-101 to extract features from the wafer defect SEM image, obtains a feature map and inputs it into the RPN, obtains multiple candidate regions, and then inputs all candidate regions into the RoI pooling layer to obtain a fixed-length feature vector, which is input into the pre-trained ResNet-101 for feature extraction, and the feature vector after feature extraction is averaged and pooled to obtain the final feature vector and output it to the LSTM decoder. The key to the whole process is RPN and RoI pooling, which enable Faster R-CNN to effectively extract candidate target regions from the original image and convert them into fixed-size features suitable for classification and bounding box prediction.
[0070] It is worth noting that in this process, in order to pre-train the bottom-up attention model, the present invention needs to first initialize Faster R-CNN with the pre-trained ResNet-101, and then train it on the dataset of the present invention.
[0071] Since the present invention adopts the bottom-up feature extraction method of Faster R-CNN, it is necessary to download the previously preprocessed feature data and prepare an additional folder to store the position and size of the target box. The features to be extracted include: bbox is the border information, num_bbox is the number of borders, image_h is the height of the image, image_w is the width of the image, and info is other important information.
[0072] In the decoding stage, referring to the paper "Show, Attend and Tell", the present invention inputs the defect SEM image features and corresponding sentences obtained by the feature extraction network into the LSTM decoder (i.e., the LSTM decoder based on the attention mechanism for different defect diagnosis tasks), and trains the model to predict words one by one for a given image. The network will generate a word at each time step based on the context vector, the previous hidden state, and the previously generated words, thereby generating a complete sentence. The network's next step depends on the sequence of words that have been generated. The initial memory state and hidden state of the LSTM are predicted by the average of the initial vectors provided by two independent MLPs.
[0073] In the sequence generation task using LSTM (Long Short-Term Memory Network) based on attention mechanism as decoder, the attention mechanism plays a vital role. This mechanism enables the decoder to flexibly focus on the most relevant part of the input data in each step of the generation process. Specifically, in the scenario of image description generation, the decoder first receives the image feature representation from the encoder, which captures the visual information of the image. Subsequently, the decoder combines these features with the given language context, such as the user's query or previous description, and the previously generated output to determine the next word in the sequence. In this way, the decoder is able to generate descriptions that are closely related to the image content while maintaining coherence with the previous description.
[0074] The core of the attention mechanism is that it can dynamically assign different weights to the decoder's hidden state, which are adjusted based on the relevance of the current generation task. For example, when generating a description of a specific object, the decoder can use the attention score to make the model pay more attention to the features of the object in the image. As each word is generated, the decoder updates its hidden state and adjusts the attention distribution to reflect the impact of the newly generated word on the context.
[0075] like Figure 5 As shown, the decoder part of the defect image diagnosis network uses the LSTM attention mechanism, and uses the LSTM decoder to gradually decode the feature vector to generate a word vector representation of the word corresponding to the corresponding defect diagnosis task, and generates words according to the word vector representation, and all the words are sequentially spliced to form a sentence of the defect diagnosis task; the sentences of different defect diagnosis tasks are combined to obtain a diagnosis report;
[0076] During the step-by-step decoding process of the feature vector by the LSTM decoder, the LSTM decoder interacts with the feature vector, the previous decoding result of the decoder and the sentence of the defect diagnosis task to obtain the word vector representation of the next word. The whole process is iterative until the generated sequence reaches a predetermined length or encounters a sign of the end of the sequence (such as a specific end symbol). That is, during the step-by-step decoding process of the LSTM decoder, the previous decoding result is used as the input of the next decoding, and the previous decoding result at the initial moment is empty. In this way, through the carefully designed LSTM and attention mechanism, the model can gradually construct an accurate, fluent and rich subtitle sequence, effectively integrating visual information and language information, and achieving high-quality description of image content.
[0077] Specifically, first, the image features extracted by the encoder are used as the initial context for the decoder. These features capture the visual information of the image and provide a basis for generating descriptions. Next, the LSTM decoder generates a context vector at each time step, which is part of the decoder's hidden state and contains information from all previous time steps, including knowledge learned from image features and previously generated language information. At each time step, the decoder calculates attention weights on the image features. These weights are obtained through the interaction between the current hidden state of the decoder and the image features, and are implemented through a learnable attention score function. The image features are weighted using the calculated attention weights to generate a weighted image feature representation that highlights the most relevant parts of the current generation task.
[0078] The weighted image features are combined with the context vector of the decoder to form the input of the decoder. This fusion process can be achieved through simple concatenation, weighted averaging, or other more complex fusion techniques. The LSTM decoder updates its hidden state based on the fused input and generates a word vector representation for the next word. In this process, the hidden state of the decoder continuously accumulates the interaction results of language information and visual information. Finally, the word vector representation generated by the decoder is used to determine the next most likely word through a softmax layer, which is then added to the description sequence being generated. The newly generated word is fed back to the decoder as part of the input for the next step to be generated. In this way, the decoder takes into account the previously generated words when generating each new word, realizing the accumulation and interaction of language information.
[0079] For the training of the defective image diagnosis network, the cross entropy loss function is used to calculate the loss value of the defective image diagnosis network, and the trained defective image diagnosis network is obtained by iterative optimization. The present invention uses the cross entropy loss function to optimize the difference between the generated subtitles and the real subtitles. Specifically, the cross entropy loss function first converts the predicted output of each word (usually a continuous word vector representation) into a probability distribution through a softmax layer. The softmax layer generates a probability value between 0 and 1 for each possible word, and the sum of these probability values is 1. Then, the loss function calculates the gap between this probability distribution and the actual distribution of words in the real subtitles. Each word in the real subtitle sequence is represented by one-hot encoding, that is, the correct word position is 1 and the other positions are 0.
[0080] The smaller the value of the loss function, the higher the consistency between the subtitles predicted by the model and the real subtitles. During the training process, the goal of the present invention is to adjust the parameters of the model through the back propagation algorithm Adam optimizer to minimize the value of this loss function.
[0081] In the present invention, the test for the defective image diagnosis network is to select the best subtitle sequence by maximizing the generation probability and output the generated image subtitles. Specifically, during the test, the present invention uses beam search to maximize the generation probability of the entire subtitle sequence. Consider multiple possible word combinations and select the sequence with the highest probability. With the addition of each new word, the context of the decoder is updated to reflect the impact of the newly generated word on the subtitle sequence. This process continues until the end of the generated sequence is marked or the predetermined maximum length is reached.
[0082] The quality of the generated subtitle sequence is evaluated by BLEU, ROUGE, etc. to guide the further optimization of the model. Finally, the generated subtitle sequence is displayed together with the image, and the image attention activation map corresponding to each word in each image is displayed. This not only enhances the comprehensibility of the image, but also provides an intuitive and practical way for the interpretability of the model.
[0083] In this embodiment, a semiconductor defect analysis system with rapid positioning and combined semantic interpretation is also provided, including:
[0084] A data acquisition module, which is used to acquire a wafer defect image diagnosis data set in a training phase, the wafer defect image diagnosis data set including a wafer defect SEM image and a diagnosis report, wherein the diagnosis report includes a plurality of sentences for different defect diagnosis tasks; and, in a prediction phase, to acquire a wafer defect SEM image to be analyzed;
[0085] A defect image diagnosis network module includes a Faster R-CNN encoder and an LSTM decoder based on an attention mechanism for different defect diagnosis tasks; a wafer defect SEM image is input into the Faster R-CNN encoder to obtain a feature vector of the wafer defect SEM image, the LSTM decoder is used to gradually decode the feature vector to generate a word vector representation of a word corresponding to a corresponding defect diagnosis task, a word is generated according to the word vector representation, and all the words are sequentially concatenated to form a sentence of the defect diagnosis task; sentences of different defect diagnosis tasks are combined to obtain a diagnosis report;
[0086] A network training module is used to train a defect image diagnosis network using a wafer defect image diagnosis dataset. During the training of the defect image diagnosis network, the sentence of the defect diagnosis task needs to be used as the input of the LSTM decoder to guide the LSTM decoder to generate the target sentence.
[0087] The defect analysis module is used to analyze the wafer defect SEM image to be processed using the trained defect image diagnosis network and generate a diagnosis report.
[0088] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.
[0089] The embodiments of the system of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The system embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor of any device with data processing capabilities and run.
[0090] Obviously, the above-described embodiments and drawings are only some examples of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that, although the work done in this development process may be complicated and lengthy, for those of ordinary skill in the art, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application. Without departing from the concept of the present application, several variations and improvements can also be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.
Claims
1. A semiconductor defect analysis method with rapid positioning and joint semantic interpretation, characterized in that: include: Acquire a wafer defect image diagnosis dataset including a wafer defect SEM image and a diagnosis report, wherein the diagnosis report includes a plurality of sentences for different defect diagnosis tasks; A defect image diagnosis network is constructed, wherein the defect image diagnosis network includes a Faster R-CNN encoder and an LSTM decoder based on an attention mechanism for different defect diagnosis tasks; a wafer defect image diagnosis data set is used to train the defect image diagnosis network, a wafer defect SEM image is input into the Faster R-CNN encoder, a feature vector of the wafer defect SEM image is obtained, the feature vector is gradually decoded by the LSTM decoder to generate a word vector representation of a word corresponding to a corresponding defect diagnosis task, a word is generated according to the word vector representation, and all the words are sequentially spliced to form a sentence of the defect diagnosis task; and sentences of different defect diagnosis tasks are combined to obtain a diagnosis report; During the training process of the defect image diagnosis network, the sentence of the defect diagnosis task is used as the input of the LSTM decoder to guide the LSTM decoder to generate the target sentence; The trained defect image diagnosis network is used to analyze the SEM image of wafer defects to be processed and generate a diagnosis report.
2. The semiconductor defect analysis method of rapid positioning combined with semantic interpretation according to claim 1, characterized in that: The defect diagnosis tasks include: Defect location, shape, and size diagnosis tasks; The task of diagnosing the convexity and deformation degree of defects; The task of diagnosing the process layer where the defect is located; The background and composition information diagnosis task of the defect; Defect category diagnosis task; Wherein, the defect diagnosis task includes at least one of the above tasks.
3. The semiconductor defect analysis method of rapid positioning combined with semantic interpretation according to claim 2, characterized in that: The sentences of the defect diagnosis task in the diagnosis report meet the following conditions at the same time: Except for the sentences corresponding to the defect category diagnosis task, a sentence should contain at least six words; Do not use ambiguous words and grammar; The content of the statement contains all the defects.
4. The semiconductor defect analysis method of rapid positioning combined with semantic interpretation according to claim 1, characterized in that: The wafer defect SEM images in the wafer defect image diagnosis data set are obtained by synthesizing wafer defect SEM images taken at different angles.
5. The semiconductor defect analysis method of rapid positioning combined with semantic interpretation according to claim 1, characterized in that: Before training the defect image diagnosis network, the sentences in the diagnosis report of the wafer defect image diagnosis data set are preprocessed, and the preprocessing includes: Convert all uppercase letters in the sentence to lowercase letters, remove all punctuation marks in the sentence and perform word segmentation, convert each sentence into a list containing multiple words, and save the processing results in a dictionary. The key in the dictionary is the ID of the wafer defect SEM image, and the value in the dictionary is a list containing multiple words. The same preprocessing is performed on all sentences in the diagnosis report.
6. The semiconductor defect analysis method of rapid positioning combined with semantic interpretation according to claim 1, characterized in that: The Faster R-CNN encoder first uses a pre-trained ResNet-101 to extract features from the wafer defect SEM image, obtains a feature map and inputs it into the RPN to obtain multiple candidate regions, then inputs all candidate regions into the RoI pooling layer to obtain a feature vector of fixed length, inputs the feature vector of fixed length into the pre-trained ResNet-101 for feature extraction, performs an average pooling operation on the feature vector after feature extraction, obtains the final feature vector and outputs it to the LSTM decoder.
7. The semiconductor defect analysis method of rapid positioning combined with semantic interpretation according to claim 6, characterized in that: The pre-trained ResNet-101 is obtained by pre-training ResNet-101 using the ImageNet data set; and before performing defect image diagnosis network training, the pre-trained ResNet-101 is used to initialize the Faster R-CNN encoder.
8. The semiconductor defect analysis method of rapid positioning combined with semantic interpretation according to claim 1, characterized in that: When the LSTM decoder decodes the feature vector step by step, the decoding result of the previous step is used as the input of the next decoding step, and the decoding result of the previous step at the initial moment is empty.
9. The semiconductor defect analysis method of rapid positioning combined with semantic interpretation according to claim 1, characterized in that: The cross entropy loss function is used to calculate the loss value of the defect image diagnosis network, and the trained defect image diagnosis network is obtained by iterative optimization.
10. A semiconductor defect analysis system with rapid positioning and semantic interpretation, characterized in that: include: A data acquisition module, which is used to acquire a wafer defect image diagnosis data set in a training phase, the wafer defect image diagnosis data set including a wafer defect SEM image and a diagnosis report, wherein the diagnosis report includes a plurality of sentences for different defect diagnosis tasks; and, in a prediction phase, to acquire a wafer defect SEM image to be analyzed; A defect image diagnosis network module includes a Faster R-CNN encoder and an LSTM decoder based on an attention mechanism for different defect diagnosis tasks; a wafer defect SEM image is input into the Faster R-CNN encoder to obtain a feature vector of the wafer defect SEM image, the LSTM decoder is used to gradually decode the feature vector to generate a word vector representation of a word corresponding to a corresponding defect diagnosis task, a word is generated according to the word vector representation, and all the words are sequentially concatenated to form a sentence of the defect diagnosis task; sentences of different defect diagnosis tasks are combined to obtain a diagnosis report; A network training module is used to train a defect image diagnosis network using a wafer defect image diagnosis dataset. During the training of the defect image diagnosis network, the sentence of the defect diagnosis task needs to be used as the input of the LSTM decoder to guide the LSTM decoder to generate the target sentence. The defect analysis module is used to analyze the wafer defect SEM image to be processed using the trained defect image diagnosis network and generate a diagnosis report.
Citation Information
Patent Citations
Wafer surface defect detection method and system based on deep learning and storage medium
CN117710378A
Wafer defect detection method, device and system and computer readable storage medium
CN117853392A
System and method for detecting anomalous events
KR1020220142794A
Cited By
Simulation method for semiconductor device, electronic equipment and storage medium
CN119782826A
Power transmission line defect detection method and system based on deep learning
CN119903433A
