A wafer defect detection method and device based on multi-view feature fusion

Abstract

This specification provides a wafer defect detection method and apparatus based on multi-view feature fusion. The method includes: acquiring a defect view of the wafer to be detected and its corresponding magnified view and top view; inputting the defect view, magnified view, and top view into an image processing network to obtain a first feature map corresponding to the defect view, a second feature map corresponding to the magnified view, and a third feature map corresponding to the top view; fusing the first feature map, the second feature map, and the third feature map to obtain a fused feature map; and determining the target defect type corresponding to the wafer to be detected based on the fused feature map, so as to achieve accurate detection of the defect type of the wafer.

Description

Technical Field

[0001] This disclosure relates to the field of semiconductor technology, and in particular to a wafer defect detection method and apparatus based on multi-view feature fusion. Background Technology

[0002] During the manufacturing process of semiconductor wafers, various defects inevitably occur. Accordingly, defect inspection is performed on the generated wafers during actual wafer production.

[0003] In related technologies, wafer images are typically inspected manually to determine the presence and type of defects. This manual defect inspection method is inevitably affected by subjective factors, and with increased workload and excessive eye strain, it leads to problems such as low inspection accuracy, low efficiency, and high labor costs. Summary of the Invention

[0004] This specification provides one or more embodiments of a wafer defect detection method and apparatus based on multi-view feature fusion to achieve accurate detection of wafer defect types.

[0005] According to a first aspect, a wafer defect detection method based on multi-view feature fusion is provided, characterized in that the method includes:

[0006] Obtain the defect view of the wafer to be inspected, along with its corresponding magnified and top views;

[0007] The defect view, the magnified view, and the top view are respectively input into the image processing network to obtain the first feature map corresponding to the defect view, the second feature map corresponding to the magnified view, and the third feature map corresponding to the top view;

[0008] The first feature map, the second feature map, and the third feature map are fused to obtain a fused feature map;

[0009] Based on the fused feature map, the defect type corresponding to the wafer to be inspected is determined.

[0010] According to a second aspect, a wafer defect detection device based on multi-view feature fusion is provided, characterized in that the device comprises:

[0011] The first acquisition module is configured to acquire a defect view, a magnified view, and a top view of the wafer to be inspected;

[0012] The first input module is configured to input the defect view, the magnified view, and the top view into the image processing network respectively to obtain a first feature map corresponding to the defect view, a second feature map corresponding to the magnified view, and a third feature map corresponding to the top view;

[0013] The first fusion module is configured to fuse the first feature map, the second feature map, and the third feature map to obtain a fused feature map.

[0014] The first determining module is configured to determine the defect type corresponding to the wafer to be inspected based on the fused feature map.

[0015] According to a third aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0016] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in the first aspect.

[0017] According to the wafer defect detection method and apparatus based on multi-view feature fusion provided in the embodiments of this specification, a defect view of the wafer to be detected and its corresponding magnified view and top view are obtained; the defect view, magnified view and top view are respectively input into an image processing network to obtain a first feature map corresponding to the defect view, a second feature map corresponding to the magnified view and a third feature map corresponding to the top view; the first feature map, the second feature map and the third feature map are fused to obtain a fused feature map; based on the fused feature map, the defect type corresponding to the wafer to be detected is determined.

[0018] In the above process, multiple views, namely the defect view and its corresponding magnified and top views, are used to obtain feature maps of each view through an image processing network. Then, the feature maps of each view are fused to obtain a fused feature map. Based on the fused feature map, the defect type of the wafer to be inspected is determined. This can realize the automatic detection and determination of the defect type of the wafer, improve detection efficiency and reduce labor costs. Moreover, the fused feature map of the multiple views contains richer and more comprehensive information about the wafer defects. Using the fused feature map as the basis for determining the defect type of the wafer can better improve the accuracy of the final determined defect type and improve detection precision. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Figure 1 A flowchart illustrating the training process of the image processing network provided in this embodiment.

[0021] Figure 2 A schematic flowchart of a wafer defect detection method based on multi-view feature fusion provided in an embodiment;

[0022] Figure 3A A schematic diagram of the structure of an image processing network provided in an embodiment;

[0023] Figure 3B A schematic diagram of a wafer defect detection process based on multi-view feature fusion provided for an embodiment;

[0024] Figure 3C A schematic diagram of another wafer defect detection process based on multi-view feature fusion provided for an embodiment;

[0025] Figure 3D A schematic diagram of another wafer defect detection process based on multi-view feature fusion provided for an embodiment;

[0026] Figure 4 This is a schematic block diagram of a wafer defect detection device based on multi-view feature fusion provided in an embodiment. Detailed Implementation

[0027] The technical solutions of the embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0028] It should be noted that the wafer defect detection method based on multi-view feature fusion provided in this embodiment draws on the MIL (Multiple Instance Learning) concept. It uses multiple views containing the same defect and their corresponding defect type labels to train an image processing network, resulting in an image processing network with relatively more accurate processing results and more helpful in determining defect type features. Then, the trained image processing network is used to extract features from the multiple views containing wafer defects. The feature maps of each extracted view are then fused, and the fused feature map is used to determine the defect type corresponding to the wafer, thereby achieving automatic detection and identification of defect types and improving the accuracy of defect type determination results.

[0029] The training process of the image processing network will be introduced below.

[0030] Specifically, such as Figure 1 The diagram illustrates a flowchart of the training process of an image processing network in one embodiment of this specification. This training process can be executed by an electronic device, which can be any device, equipment, platform, device cluster, etc., with computing and processing capabilities. The training process may include the following steps S110-S150;

[0031] In step S110, a sample image set and its corresponding defect type label are obtained, wherein the sample image set includes a sample defect view of the corresponding sample wafer and its corresponding magnified sample view and sample top view.

[0032] In one implementation, before training the image processing network to be trained, a training set for training the network can be constructed. This training set includes several sample image sets and corresponding label data for each sample image set. The label data includes defect type labels for each sample image set, i.e., the true types of defects in each view within the sample image set. The defect type labels for each sample image set can be labels manually or assigned by a specific program for the sample image sets.

[0033] A single sample image set includes a defect view (i.e., a defect view) of the corresponding sample wafer, along with its corresponding zoomed-in view (i.e., a zoomed-in view) and top view (i.e., a top view). For clarity, this will be referred to hereafter as the sample image set including the defect view of the corresponding sample wafer and its corresponding zoomed-in view and top view, respectively. Each sample image set can be obtained from any specified data source. The sample wafer can be any wafer with defects.

[0034] The sample defect view of the sample wafer can be an image taken of any defective region of the sample wafer (e.g., the region where defect A is located). The corresponding magnified sample view is a magnified view of defect A within the region where defect A is located, i.e., an image taken of a local area within the region where defect A is located, which includes more detailed information about defect A. In other words, the magnified sample view includes more detailed information about defect A.

[0035] The area included in the top view of the sample corresponding to the sample defect view may be the same as the area included in the magnified view of the sample, but the shooting angle of the top view of the sample corresponding to the sample defect view is different from the shooting angle of the magnified view of the sample corresponding to the sample defect view. The top view of the sample is an image taken from a top-down angle (e.g., an angle perpendicular to the sample wafer).

[0036] After obtaining the training set, the image processing network to be trained can be trained using several sample image sets in the training set and the corresponding defect type labels for each sample image set. It is understandable that the process of training the image processing network to be trained using any sample image set in the training set and its corresponding defect type labels is similar. The following section will use the process of training the image processing network to be trained using any sample image set in the training set and its corresponding defect type labels as an example to introduce this training process.

[0037] The electronic device can obtain any set of sample images and the corresponding defect type labels from the training set. The set of sample images includes a sample defect view of the corresponding sample wafer and its corresponding magnified and top views. Then, in step S120, the sample defect view, magnified view, and top view are input into the image processing network to be trained, respectively, to obtain a fourth feature map corresponding to the sample defect view, a fifth feature map corresponding to the magnified view, and a sixth feature map corresponding to the top view.

[0038] In this step, the electronic device can input the aforementioned sample defect view, sample magnified view, and sample top view into the image processing network to be trained, so that the image processing network to be trained can process the aforementioned sample defect view, sample magnified view, and sample top view respectively to obtain the feature map corresponding to the sample defect view (called the fourth feature map), the feature map corresponding to the sample magnified view (called the fifth feature map), and the feature map corresponding to the sample top view (called the sixth feature map).

[0039] The image processing network to be trained can be any deep learning-based neural network capable of processing images, as described in related technologies. In one embodiment, the image processing network to be trained can be a ResNet-based processing network. In one case, such as... Figure 3A As shown, the image processing network to be trained can be a ResNet18 network, which may include a conv1 layer (convolutional layer), a bn1 layer (batch normalization layer), a ReLU layer (activation layer), a max pooling layer, four sequentially configured layer blocks (each layer block may include multiple convolutional layers), a GAP (Global Average Pooling) layer, a Flatten layer, and a fully connected layer. The conv1, bn1, and ReLU layers extract shallow image features from the input (such as a sample defect view, a magnified view of a sample, a sample top view, and the defect view, magnified view, and top view of the wafer to be inspected mentioned later). The max pooling layer downsamples the input, and the four layer blocks further extract deeper image features from the input. The GAP layer is used to reduce the dimensionality of the input (e.g., reducing an h*w*c input to 1*1*c, where h represents the height of the input, w represents the width of the input, and c represents the number of input channels). Flattening layers are used to "flatten" the input, that is, to reduce a multidimensional input to one dimension (e.g., flattening an input of size 1*1*c into a one-dimensional array of size c). They are commonly used in the transition from convolutional layers to fully connected layers. Fully connected (FC) layers are used to perform further feature extraction processing on the input.

[0040] In one implementation, the electronic device can acquire the output of a designated layer during the image processing network's processing of a sample defect view, a magnified sample view, and a top view of the sample, respectively, and use these as the fourth feature map corresponding to the sample defect view, the fifth feature map corresponding to the magnified sample view, and the sixth feature map corresponding to the top view of the sample, respectively. This designated layer can be, for example, the last layer block, or an FC (fully connected) layer. In one case, to better improve the accuracy of the defect type detection results, this designated layer can be an FC layer.

[0041] After the electronic device obtains the fourth feature map, the fifth feature map, and the sixth feature map, in step S130, the fourth feature map, the fifth feature map, and the sixth feature map are fused to obtain the sample feature map.

[0042] In this step, the electronic device can use any fusion method to fuse the fourth, fifth, and sixth feature maps to obtain a fused sample feature map. In one embodiment, the aforementioned fusion method can be: calculating the average result of the fourth, fifth, and sixth feature maps; and determining the obtained average result as the fused sample feature map.

[0043] In another embodiment, the aforementioned fusion method can be as follows: in the channel dimension, the fourth feature map, the fifth feature map, and the sixth feature map are concatenated to obtain the fused sample feature map. For example, the size of the fourth feature map, the fifth feature map, and the sixth feature map are all h1*w1*c1, and the size of the corresponding fused sample feature map is h1*w1*(3*c1).

[0044] In another embodiment, the aforementioned fusion method can be as follows: The fourth and fifth feature maps are input into a channel attention fusion network to process them, obtaining attention values ​​corresponding to each feature map. The first fusion result is then determined by the sum of the product of the fourth feature map and its attention value, and the product of the fifth feature map and its attention value. Next, the first fusion result and a sixth feature map are input into the channel attention fusion network to process them, obtaining attention values ​​corresponding to each feature map. The second fusion result, i.e., the sample feature map, is then determined by the sum of the product of the first fusion result and its attention value, and the product of the sixth feature map and its attention value. In one implementation, this attention fusion network can be implemented using the channel attention network in SENet. In another implementation, the channel attention network can be a pre-trained network. In yet another implementation, the channel attention network also includes parameters to be adjusted, and the channel attention network is trained simultaneously with the image processing network to be trained.

[0045] After the electronic device fuses to obtain the sample feature map, in step S140, the predicted defect type corresponding to the aforementioned sample image set is determined based on the sample feature map.

[0046] In one implementation, the electronic device can input a sample feature map into a defect detection network to process the feature map, obtaining the network's output. This output includes the probability (hereinafter referred to as the estimated probability) corresponding to each defect type. The estimated probability for each defect type characterizes the probability that the defect detection network predicts the type of defect in the sample defect view, its corresponding magnified view, and its top view (i.e., the defects involved in the sample image set). Next, the electronic device can determine the highest estimated probability from the output and identify the defect type corresponding to the highest estimated probability as the predicted defect type for the aforementioned sample image set, i.e., the predicted defect type to which the defects in the sample defect view, magnified view, and top view belong.

[0047] In one implementation, when the sample feature map is determined based on the average of the fourth, fifth, and sixth feature maps, or when the sample feature map is determined by a channel attention fusion network based on the fourth, fifth, and sixth feature maps, and the fourth, fifth, and sixth feature maps are the outputs of the FC layer shown in 3A, the aforementioned defect detection network may include an activation layer. Accordingly, the electronic device may input the sample feature map into the activation layer to obtain the estimated probability corresponding to each defect type for the sample feature map.

[0048] In another implementation, when the sample feature map is determined by concatenating the fourth, fifth, and sixth feature maps along the channel dimension, and when the fourth, fifth, and sixth feature maps are the outputs of the FC layer shown in 3A, the aforementioned defect detection network of the electronic device may include a designated fully connected layer and an activation layer. Accordingly, the electronic device may input the sample feature map into the designated fully connected layer to process the sample feature map and obtain the corresponding output. Then, the corresponding output is input into the activation layer to obtain the estimated probability corresponding to each defect type for the sample feature map.

[0049] The defect detection network can be a pre-trained detection network; or it can be a detection network trained together with the image processing network to be trained.

[0050] In another embodiment, the image processing network to be trained also outputs a fourth intermediate result for a sample defect view, a fifth intermediate result for a sample magnified view, and a sixth intermediate result for a sample top view; each intermediate result includes the preliminary probability corresponding to each defect type obtained from the preliminary detection.

[0051] Step S140 may include: the electronic device inputs the sample feature map into the defect detection network to obtain the sample detection result, which shows the predicted probability corresponding to each defect type; the electronic device fuses the first detection result, the fourth intermediate result, the fifth intermediate result and the sixth intermediate result to obtain the predicted defect type corresponding to the aforementioned sample image set.

[0052] In one scenario, the process of fusing the sample detection results, the fourth intermediate result, the fifth intermediate result, and the sixth intermediate result can be to calculate the average of the sample detection results, the fourth intermediate result, the fifth intermediate result, and the sixth intermediate result, and determine the defect type corresponding to the average value with the largest value in the average result as the predicted defect type corresponding to the aforementioned sample image set.

[0053] In another case, the process of fusing the sample detection results, the fourth intermediate result, the fifth intermediate result, and the sixth intermediate result can be to determine the defect type corresponding to the highest probability value (there are 4 in total) from the sample detection results, the fourth intermediate result, the fifth intermediate result, and the sixth intermediate result, respectively, and then determine the defect type that appears most frequently from the 4 defect types corresponding to the highest probability values, as the predicted defect type corresponding to the aforementioned sample image set.

[0054] In another scenario, the sample detection results, the fourth intermediate result, the fifth intermediate result, and the sixth intermediate result can be fused based on the aforementioned fusion feature map method. Then, based on the fused result, the predicted defect type corresponding to the aforementioned sample image set can be obtained.

[0055] After determining the predicted defect type corresponding to the aforementioned sample wafer (i.e., sample image set) through the above method, in step S150, the image processing network to be trained is trained based on the defect type label and the predicted defect type to obtain an image processing network that meets the preset convergence condition.

[0056] In this step, the electronic device can determine the prediction loss based on the difference between the defect type label and the predicted defect type according to a preset loss function. Then, with the goal of minimizing the prediction loss, the parameters of the image processing network to be trained are adjusted, i.e., the image processing network to be trained is trained. The preset loss function can be any type of loss function, including but not limited to: cross-entropy loss function, mean squared error loss function, etc.

[0057] In one implementation, the aforementioned process of adjusting the parameters of the image processing network to be trained with the goal of minimizing the prediction loss may further include: adjusting the parameters of the image processing network to be trained and the parameters of the defect detection network with the goal of minimizing the prediction loss.

[0058] In one implementation, where the sample feature map is determined by a channel attention fusion network based on the fourth, fifth, and sixth feature maps, the aforementioned process of adjusting the parameters of the image processing network to be trained with the goal of minimizing the prediction loss may include: adjusting the parameters of the image processing network to be trained and the parameters of the channel attention fusion network with the goal of minimizing the prediction loss.

[0059] Understandably, steps S110 to S150 above constitute one iteration of model training. To train a better image processing network, the above process can be executed iteratively multiple times. That is, after step S150, based on the updated parameters of the image processing network to be trained, step S110 is returned to execute until the image processing network with adjusted parameters reaches the preset convergence condition. The image processing network that has reached the preset convergence condition is determined as the image processing network that has been trained successfully.

[0060] The aforementioned preset convergence conditions may include, for example, the number of iterations reaches a preset threshold, the iteration duration reaches a preset duration, or the prediction loss is less than a set loss threshold.

[0061] To recap the execution process of steps S110 to S150, the above embodiment uses a single sample image set as an example. In another embodiment, steps S110 to S140 can be performed on a batch of samples, i.e., multiple sample image sets, to obtain the predicted defect type for each sample image set. Then, based on the predicted defect type and defect type label for each sample image set, the prediction loss is determined, and the image processing network to be trained is trained with the goal of minimizing the prediction loss. In this embodiment, the prediction loss is determined for a batch of samples, and then the model parameters of the image processing network to be trained are adjusted. This reduces the number of parameter adjustments required for the image processing network to be trained, making the training process easier to implement.

[0062] After obtaining the trained image processing network, the wafer defect detection process is carried out using the image processing network.

[0063] Figure 2 A flowchart of a wafer defect detection method based on multi-view feature fusion according to one embodiment of this specification is shown. This method is executed by an electronic device, which can be any device, equipment, platform, or cluster of devices with computing and processing capabilities. This electronic device can be the same physical device as the aforementioned electronic device that performs the training process of the image processing network, or it can be a different physical device. In the case of a different physical device, the electronic device can pre-acquire the aforementioned trained image processing network and store it locally, or the electronic device can invoke the aforementioned trained image processing network during the wafer defect detection process based on multi-view feature fusion.

[0064] In the wafer defect detection process based on multi-view feature fusion, such as Figure 2 As shown, the method includes the following steps S210-S240:

[0065] In step S210, a defect view of the wafer to be inspected and its corresponding magnified view and top view are obtained.

[0066] The wafer to be inspected can be any wafer requiring defect inspection. The defect view of the wafer can be an image taken of a specific area of ​​the wafer where defect inspection is required (e.g., the area containing defect X). The magnified view corresponding to the defect view is a magnified view of defect X, i.e., an image taken of a local area within the region containing defect X, which includes more detailed information about defect X. The top view corresponding to the defect view can include the same area as the magnified view, but the shooting angle of the top view is different from that of the magnified view. The top view is an image taken from a top-down angle (e.g., perpendicular to the wafer to be inspected).

[0067] Electronic devices can acquire defect views of the wafer under inspection and their corresponding magnified and top views from other devices, or they can acquire defect views of the wafer under inspection and their corresponding magnified and top views based on the set image acquisition device.

[0068] Next, in step S220, the defect view, magnified view, and top view are input into the image processing network to obtain a first feature map corresponding to the defect view, a second feature map corresponding to the magnified view, and a third feature map corresponding to the top view. This image processing network is the one described above... Figure 1 The image processing network trained using the illustrated process is shown. In one embodiment, this image processing network can be a ResNet-based processing network, such as a ResNet18 network, with the following structure: Figure 3A As shown. The first feature map corresponding to the defect view, the second feature map corresponding to the magnified view, and the third feature map corresponding to the top view can be the outputs of the aforementioned specified layer of the image processing network.

[0069] In this step, the electronic device inputs the defect view, magnified view, and top view into the image processing network, respectively. The image processing network processes the defect view, magnified view, and top view to obtain the first feature map corresponding to the defect view, the second feature map corresponding to the magnified view, and the third feature map corresponding to the top view.

[0070] After obtaining the first feature map, the second feature map, and the third feature map, in step S230, the first feature map, the second feature map, and the third feature map are fused to obtain the fused feature map.

[0071] In this step, the electronic device can use any fusion method to fuse the first feature map, the second feature map, and the third feature map to obtain a fused feature map.

[0072] In one embodiment, such as Figure 3BAs shown, step S230 may include the following steps 11-12: In step 11, the average feature map of the first feature map, the second feature map, and the third feature map is calculated. In step 12, the average feature map obtained from the aforementioned calculation is determined as the fused feature map. In this implementation, by calculating the average feature map of the first feature map, the second feature map, and the third feature map, the defect feature information included in each of the three views (defect view, magnified view, and top view) is fused to obtain a fused feature map with richer and more comprehensive defect feature information, providing a basis for improving the accuracy of the defect type determination result.

[0073] In yet another embodiment, such as Figure 3C As shown, step S230 may include the following step 21: In step 21, the first feature map, the second feature map, and the third feature map are stitched together along the channel dimension to obtain a fused feature map. In this implementation, the first feature map, the second feature map, and the third feature map are stitched together along the channel dimension to achieve the fusion of the defect feature information included in each of the three views along the channel dimension, resulting in a fused feature map with richer and more comprehensive defect feature information, providing a foundation for improving the accuracy of defect type determination.

[0074] In one embodiment, step S230 may include the following steps 31-32:

[0075] In step 31, based on the first feature map and the second feature map, an intermediate result is obtained through a channel attention fusion network.

[0076] In step 32, based on the intermediate results and the third feature map, a fused feature map is obtained through a channel attention fusion network.

[0077] In this implementation, such as Figure 3D As shown, the electronic device can input a first feature map and a second feature map into a channel attention fusion network. The channel attention fusion network processes the first and second feature maps to obtain attention values ​​corresponding to each feature map. Then, based on the sum of the product of the first feature map and its attention value, and the product of the second feature map and its attention value, the attention fusion network determines an intermediate result. Next, the intermediate result and a third feature map are input into the channel attention fusion network to process them, obtaining attention values ​​corresponding to the intermediate and third feature maps. Finally, based on the sum of the product of the intermediate result and its attention value, and the product of the third feature map and its attention value, the attention fusion network determines a fused feature map. In one implementation, this attention fusion network can be implemented using the channel attention network in SENet.

[0078] By using a channel attention fusion network, more useful information for defect type determination can be filtered out from each feature map, thus obtaining a fusion feature map that is more helpful in improving the accuracy of defect type determination.

[0079] In another implementation, the electronic device can first obtain an intermediate result based on the second and third feature maps (or the first and third feature maps) through a channel attention fusion network; then, based on the intermediate result and the first (or second) feature map, obtain a fused feature map through a channel attention fusion network, which is also possible.

[0080] After obtaining the fused feature map, in step S240, the target defect type corresponding to the aforementioned wafer to be inspected is determined based on the fused feature map.

[0081] In one embodiment, the electronic device can determine the target defect type corresponding to the wafer to be inspected based on the fused feature map and the aforementioned defect detection network. In one implementation, step S240 may include the following steps 41-42:

[0082] In step 41, the fused feature map is input into the defect detection network to obtain the predicted probabilities corresponding to each defect type. In this step, the electronic device inputs the fused feature map into the defect detection network to process it, obtaining the network's output for the fused feature map. This output includes the predicted probabilities corresponding to each defect type. The estimated probabilities corresponding to each defect type characterize the probability that the defect detection network predicts the type of defect in the defect view, its corresponding magnified view, and the top view, and that the defect is of that specific defect type.

[0083] Next, in step 42, the target defect type is determined based on the predicted probability corresponding to each defect type. The electronic device can determine the predicted probability with the highest value from the predicted probabilities corresponding to each defect type, and determine the defect type corresponding to the predicted probability with the highest value as the target defect type corresponding to the wafer to be inspected, that is, the target defect type to which the defects in the aforementioned defect view, magnified view and top view belong.

[0084] In one implementation, such as Figure 3B and Figure 3C As shown, when the fused feature map is determined based on the average result of the first feature map, the second feature map, and the third feature map, or when the sample feature map is determined by the channel attention fusion network based on the first feature map, the second feature map, and the third feature map, the aforementioned defect detection network may include an activation layer when the specified layer is an FC layer. Accordingly, the electronic device may input the fused feature map into the activation layer to obtain the predicted probability corresponding to each defect type for the fused feature map.

[0085] In another implementation, where the fused feature map is determined by splicing the first, second, and third feature maps along the channel dimension, and the aforementioned designated layer is an FC layer, the aforementioned defect detection network can include a designated fully connected layer and an activation layer. Accordingly, the electronic device can input the fused feature map into the designated fully connected layer to process the fused feature map and obtain the corresponding output. Then, the corresponding output is input into the activation layer to obtain the predicted probability corresponding to each defect type for the fused feature map.

[0086] In yet another embodiment, the image processing network also outputs a first intermediate result for the defect view, a second intermediate result for the magnified view, and a third intermediate result for the top view; each intermediate result includes a preliminary probability corresponding to each defect type obtained from the initial detection; correspondingly, step S240 may include the following steps 51-52:

[0087] In step 51, the fused feature map is input into the defect detection network to obtain the first detection result, which shows the predicted probability corresponding to each defect type. The implementation principle of step 51 is similar to that of step 41 above. The specific implementation of step 51 can be found in the specific implementation of step 41 above, and will not be repeated here.

[0088] In step 52, the first detection result, the first intermediate result, the second intermediate result, and the third intermediate result are fused to obtain the target defect type.

[0089] In one scenario, the process of fusing the first detection result, the first intermediate result, the second intermediate result, and the third intermediate result can be to calculate the average of the first detection result, the first intermediate result, the second intermediate result, and the third intermediate result, and then determine the defect type corresponding to the average value with the largest value in the average result as the target defect type.

[0090] In another scenario, the process of fusing the first detection result, the first intermediate result, the second intermediate result, and the third intermediate result can be as follows: from the first detection result, the first intermediate result, the second intermediate result, and the third intermediate result, determine the defect type corresponding to the highest probability value (there are 4 in total), and from the 4 defect types corresponding to the highest probability values, determine the defect type that appears most frequently as the target defect type.

[0091] In another scenario, the first detection result, the first intermediate result, the second intermediate result, and the third intermediate result can be fused based on the aforementioned fusion feature map method. The target defect type can then be obtained based on the fused result.

[0092] In one implementation, after obtaining the defect type corresponding to the wafer to be inspected, it can be sent to the user side so that the user can view it and perform subsequent operations.

[0093] In this embodiment, multiple views, namely the defect view and its corresponding magnified and top views, along with an image processing network, are used to obtain feature maps for each of the multiple views. These feature maps are then fused to obtain a fused feature map. Based on the fused feature map, the defect type of the wafer to be inspected is determined. This enables automatic detection and determination of the defect type of the wafer, improving detection efficiency and reducing labor costs. Furthermore, the fused feature map of the multiple views contains richer and more comprehensive information about the wafer's defects. Using the fused feature map as the basis for determining the wafer's defect type can better improve the accuracy of the final determined defect type, thereby improving detection precision and robustness.

[0094] The foregoing description describes specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments, and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily need to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0095] Corresponding to the above method embodiments, this specification provides a wafer defect detection device 400 based on multi-view feature fusion, the schematic block diagram of which is shown below. Figure 4 As shown, it includes:

[0096] The first acquisition module 410 is configured to acquire a defect view, a magnified view, and a top view of the wafer to be inspected;

[0097] The first input module 420 is configured to input the defect view, the magnified view and the top view into the image processing network respectively to obtain a first feature map corresponding to the defect view, a second feature map corresponding to the magnified view and a third feature map corresponding to the top view;

[0098] The first fusion module 430 is configured to fuse the first feature map, the second feature map, and the third feature map to obtain a fused feature map.

[0099] The first determining module 440 is configured to determine the target defect type corresponding to the wafer to be inspected based on the fused feature map.

[0100] In one alternative implementation, the image processing network is a ResNet-based processing network.

[0101] In one optional implementation, the first fusion module 430 is specifically configured to calculate the average feature map of the first feature map, the second feature map, and the third feature map; and determine the average feature map as the fused feature map.

[0102] In one optional implementation, the first fusion module 430 is specifically configured to stitch together the first feature map, the second feature map, and the third feature map in the channel dimension to obtain the fused feature map.

[0103] In one optional implementation, the first fusion module 430 is specifically configured to obtain an intermediate feature map based on the first feature map and the second feature map through a channel attention fusion network; and to obtain a fused feature map based on the intermediate result and the third feature map through the channel attention fusion network.

[0104] In one optional implementation, the first determining module 440 is specifically configured to input the fused feature map into the defect detection network to obtain the predicted probability corresponding to each defect type; and determine the target defect type based on the predicted probability corresponding to each defect type.

[0105] In one alternative implementation, the image processing network further outputs a first intermediate result for the defect view, a second intermediate result for the magnified view, and a third intermediate result for the top view; each intermediate result includes a preliminary probability corresponding to each defect type obtained from the initial detection.

[0106] The first determining module 440 is specifically configured to input the fused feature map into the defect detection network to obtain a first detection result, which shows the predicted probability corresponding to each defect type; and to fuse the first detection result, the first intermediate result, the second intermediate result and the third intermediate result to obtain the target defect type.

[0107] In one optional embodiment, it further includes: a second acquisition module (not shown in the figure), configured to acquire a sample image set and its corresponding defect type label, wherein the sample image set includes a sample defect view of the corresponding sample wafer and its corresponding sample magnified view and sample top view;

[0108] The second input module (not shown in the figure) is configured to input the sample defect view, the sample magnified view and the sample top view into the image processing network to be trained, respectively, to obtain the fourth feature map corresponding to the sample defect view, the fifth feature map corresponding to the sample magnified view and the sixth feature map corresponding to the sample top view;

[0109] The second fusion module (not shown in the figure) is configured to fuse the fourth feature map, the fifth feature map, and the sixth feature map to obtain a sample feature map;

[0110] The second determining module (not shown in the figure) is configured to determine the predicted defect type corresponding to the sample image set based on the sample feature map;

[0111] The training module (not shown in the figure) is configured to train the image processing network to be trained based on the defect type label and the predicted defect type, so as to obtain an image processing network that meets the preset convergence condition.

[0112] The above-described apparatus embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The apparatus embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0113] This specification also provides a computer-readable storage medium storing a computer program that, when executed in a computer, causes the computer to perform the wafer defect detection method based on multi-view feature fusion provided in this specification.

[0114] This specification also provides a computing device, including a memory and a processor. The memory stores executable code, and when the processor executes the executable code, it implements the wafer defect detection method based on multi-view feature fusion provided in this specification.

[0115] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for storage media and computing devices are basically similar to the method embodiments, so they are described more simply; relevant parts can be referred to the descriptions of the method embodiments.

[0116] Those skilled in the art will recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0117] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made based on the technical solutions of the present invention should be included within the scope of protection of the present invention.

Claims

1. A wafer defect detection method based on multi-view feature fusion, characterized in that, The method includes: Obtain the defect view of the wafer to be inspected, along with its corresponding magnified and top views; The defect view, the magnified view, and the top view are respectively input into the image processing network to obtain the first feature map corresponding to the defect view, the second feature map corresponding to the magnified view, and the third feature map corresponding to the top view; The first feature map, the second feature map, and the third feature map are fused to obtain a fused feature map. The fusion of the first feature map, the second feature map, and the third feature map to obtain the fused feature map includes: obtaining an intermediate feature map based on the first feature map and the second feature map using a channel attention fusion network; and obtaining the fused feature map based on the intermediate feature map and the third feature map using the channel attention fusion network. Based on the fused feature map, the target defect type corresponding to the wafer to be inspected is determined. The image processing network further outputs a first intermediate result for the defect view, a second intermediate result for the magnified view, and a third intermediate result for the top view. Each intermediate result includes a preliminary probability corresponding to each defect type obtained from preliminary detection. Determining the target defect type corresponding to the wafer to be inspected based on the fused feature map includes: inputting the fused feature map into the defect detection network to obtain a first detection result, which shows the predicted probability corresponding to each defect type; fusing the first detection result, the first intermediate result, the second intermediate result, and the third intermediate result to obtain the target defect type.

2. The method as described in claim 1, characterized in that, The image processing network is a ResNet-based processing network.

3. The method as described in claim 1, wherein, The process of fusing the first feature map, the second feature map, and the third feature map to obtain a fused feature map includes: Calculate the average feature map of the first feature map, the second feature map, and the third feature map; The average feature map is determined as the fused feature map.

4. The method as described in claim 1, characterized in that, The process of fusing the first feature map, the second feature map, and the third feature map to obtain a fused feature map includes: In the channel dimension, the first feature map, the second feature map, and the third feature map are concatenated to obtain the fused feature map.

5. The method as described in claim 1, characterized in that, The step of determining the target defect type corresponding to the wafer to be inspected based on the fused feature map includes: The fused feature map is input into the defect detection network to obtain the predicted probability corresponding to each defect type. The target defect type is determined based on the predicted probability corresponding to each defect type.

6. The method according to any one of claims 1-5, characterized in that, Also includes: Obtain a sample image set and its corresponding defect type labels, wherein the sample image set includes a sample defect view of the corresponding sample wafer and its corresponding magnified sample view and sample top view; The sample defect view, the sample magnified view, and the sample top view are respectively input into the image processing network to be trained to obtain the fourth feature map corresponding to the sample defect view, the fifth feature map corresponding to the sample magnified view, and the sixth feature map corresponding to the sample top view; The fourth feature map, the fifth feature map, and the sixth feature map are fused to obtain a sample feature map; Based on the sample feature map, the predicted defect type corresponding to the sample image set is determined; Based on the defect type label and the predicted defect type, the image processing network to be trained is trained to obtain an image processing network that meets the preset convergence condition.

7. A wafer defect detection device based on multi-view feature fusion, characterized in that, The device includes: The first acquisition module is configured to acquire a defect view, a magnified view, and a top view of the wafer to be inspected; The first input module is configured to input the defect view, the magnified view, and the top view into the image processing network respectively to obtain a first feature map corresponding to the defect view, a second feature map corresponding to the magnified view, and a third feature map corresponding to the top view; The first fusion module is configured to fuse the first feature map, the second feature map, and the third feature map to obtain a fused feature map. Specifically, the first fusion module is configured to obtain an intermediate feature map based on the first feature map and the second feature map through a channel attention fusion network; and obtain the fused feature map based on the intermediate feature map and the third feature map through the channel attention fusion network. The first determining module is configured to determine the target defect type corresponding to the wafer to be inspected based on the fused feature map, wherein the image processing network further outputs a first intermediate result for the defect view, a second intermediate result for the magnified view, and a third intermediate result for the top view; each intermediate result includes a preliminary probability corresponding to each defect type obtained from preliminary detection; the first determining module is specifically configured to input the fused feature map into the defect detection network to obtain a first detection result, which shows the predicted probability corresponding to each defect type; and fuse the first detection result, the first intermediate result, the second intermediate result, and the third intermediate result to obtain the target defect type.

8. A computing device, characterized in that, The method includes a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method according to any one of claims 1-6.

Images