Defect detection method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a defect detection method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: providing a wafer; a sampling image containing the wafer is obtained, the sampling image is obtained when the wafer is irradiated by light, the sampling image comprises a color area and a gray area, the color area is an area where defects are located, the defects of different types have different colors, and the gray area is a non-defect area; according to the color area, the sampling image is extracted, a plurality of sub-images are obtained, and one sub-image comprises the color area of one defect type; inputting the plurality of sub-images into a defect classification model, and determining a defect type corresponding to each sub-image; wherein an input layer of the defect classification model comprises a plurality of input channels, and each input channel is in one-to-one correspondence with the color of one defect type. By adopting the technical scheme, the detection precision of defect types can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a defect detection method, device, electronic equipment, storage medium and program product. Background Art

[0002] Wafers play an important role in semiconductor manufacturing and are widely used in various electronic devices.

[0003] However, during the manufacturing process, various defects inevitably appear on the wafer surface, which can affect the performance and reliability of the chip. If these defects are not detected and repaired in time, they can lead to device performance degradation or even failure. Summary of the Invention

[0004] In view of this, the present invention provides a defect detection method, apparatus, electronic device, storage medium and program product to improve the detection accuracy of defect types.

[0005] The present invention provides a defect detection method, comprising: providing a wafer; obtaining a sampling image containing the wafer, the sampling image being obtained when the wafer is irradiated with light, and the sampling image containing a color area and a grayscale area, the color area being the area where the defect is located, and different types of defects have different colors, and the grayscale area being a non-defect area; extracting and processing the sampling image according to the color area to obtain a plurality of sub-images, and a sub-image containing a color area of ​​a defect type; inputting the plurality of sub-images into a defect classification model to determine the defect type corresponding to each sub-image; wherein the input layer of the defect classification model contains a plurality of input channels, and each input channel corresponds one-to-one to the color of a defect type.

[0006] Optionally, acquiring a sample image containing the wafer includes:

[0007] Coating a precursor solution containing quantum dots on the wafer, wherein the precursor solution further comprises functional groups, wherein the functional groups enable the quantum dots to selectively attach to defective locations of the wafer;

[0008] irradiating the wafer with light that interacts with the quantum dots to activate the quantum dots and cause them to emit light;

[0009] An image of the wafer in the current light-emitting state is acquired to obtain the sample image.

[0010] Optionally, the defect detection method further includes:

[0011] Obtaining key feature parameters of the sample image in the current detection environment and the imaging effect of the sample image;

[0012] The key feature parameters are adjusted according to the imaging effect of the sampled image.

[0013] Optionally, the segmentation processing of the sampled image according to the color area to obtain a plurality of sub-images includes:

[0014] Determining boundary information of each color area according to the distribution positions of the color areas;

[0015] Determine the segmentation pattern to be used based on the boundary information of each color area;

[0016] The sampling image is segmented using the segmentation pattern to obtain a sub-image containing a color area of ​​at least one defect type.

[0017] Optionally, before segmenting the sampled image according to the color area to obtain a plurality of sub-images, the method further includes:

[0018] Determining noise information in the sampled image, wherein the noise information includes: one or more of noise type and noise intensity;

[0019] Based on the noise information, a noise filtering scheme is determined to perform noise filtering on the sampled image.

[0020] Optionally, determining noise information in the sampled image includes:

[0021] Dividing the sampled image into sub-units of preset pixels, with different color areas not overlapping each other;

[0022] A residual neural network is used to obtain the local context feature map of each sub-unit and the descriptor of the preset dimension of the sub-unit;

[0023] Adopting the channel attention mechanism, the local context feature maps and descriptors of each sub-unit are fused to generate a global feature vector;

[0024] A dual-branch network structure is adopted to determine the noise information of the global eigenvector.

[0025] Optionally, the defect classification model further comprises: a backbone feature extraction layer and a plurality of classifiers;

[0026] The number of the classifiers is the same as the number of the input channels, one classifier has multiple classification heads, one classification head is used to output a defect type, and one input channel, the backbone feature extraction layer and one classification head form a classification channel.

[0027] Optionally, inputting the plurality of sub-images into a defect classification model to determine a defect type corresponding to each sub-image includes:

[0028] According to the color area of ​​the defect type contained in the sub-image, the sub-image is input to the backbone feature extraction layer through the input channel of the corresponding color;

[0029] Using the backbone feature extraction layer, performing feature extraction and feature fusion processing on the sub-image to generate a fused feature vector;

[0030] According to the fused feature vector, a defect type corresponding to the sub-image is determined by a classification head corresponding to the input channel.

[0031] Optionally, before inputting the plurality of sub-images into the defect classification model, the method further includes:

[0032] Performing statistical analysis on the color channels of all sub-images to determine parameter characteristics of each color channel, wherein the parameter characteristics include one or more of average brightness, signal strength, and effective pixel ratio;

[0033] When it is determined that the parameter feature does not meet the set requirement, the input channel corresponding to the sub-image is disabled.

[0034] Accordingly, the present invention further provides a defect detection device, comprising:

[0035] an acquisition unit, configured to acquire a sample image of the wafer, wherein the sample image is obtained when the wafer is irradiated with light, and the sample image comprises a color area and a gray area, wherein the color area is an area where defects are located, and different types of defects have different colors, and the gray area is a non-defective area;

[0036] an extraction unit, configured to extract the sampled image according to the color region to obtain a plurality of sub-images, wherein each sub-image includes a color region of one defect type;

[0037] A processing unit is provided with a defect classification model, and the defect classification model is used to determine the defect type corresponding to each sub-image based on the sub-image, wherein the input layer of the defect classification model includes multiple input channels, and each input channel corresponds to the color of a defect type.

[0038] The present invention also provides an electronic device comprising: at least one memory and at least one processor, wherein the memory stores one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the defect detection method as described in any of the aforementioned embodiments.

[0039] The present invention also provides a storage medium, wherein the storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the defect detection method as described in any of the aforementioned embodiments.

[0040] The present invention also provides a computer program product, comprising computer instructions, which are used to implement the defect detection method as described in any of the above embodiments when executed by a processor.

[0041] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0042] In the defect detection method provided by the present invention, on the one hand, the sampling image has both color areas and grayscale areas, and the color areas are the areas where the defects are located, and the grayscale areas are the non-defective areas. Therefore, by utilizing the color difference between the color areas and the grayscale areas, the areas where the defects are located can be accurately distinguished, and by making different types of defects have different colors, effective distinction between different types of defects is achieved. Therefore, when performing the extraction operation, a sub-image can contain a color area of ​​one defect type, which reduces the interference of other factors on the defects; on the other hand, the input layer of the defect classification model includes multiple input channels, and each input channel corresponds to the color of a defect type one by one. Therefore, when the defect type is identified through the defect classification model, the recognition accuracy can be improved.

[0043] Furthermore, by coating a precursor solution containing quantum dots and functional groups on the wafer, the quantum dots can selectively attach to the defect locations of the wafer. When the wafer is irradiated with light that interacts with the quantum dots, the defect locations can emit corresponding light, which can improve the distinction between defects and non-defects and help reduce the difficulty of image segmentation. Moreover, by setting different types of quantum dots in the precursor solution, multiple types of defects can be identified simultaneously in a single detection, thereby improving detection efficiency and accuracy.

[0044] Furthermore, by adjusting key feature parameters based on the imaging results in the current detection environment, this real-time feedback strategy can optimize imaging results and detection efficiency, and improve the stability and reliability of the detection process.

[0045] Furthermore, the distribution of colored regions defines the location of defects, and since colored regions differ significantly from grayscale regions, the boundaries of each colored region can be determined, and thus the shape of the colored region. This allows for the generation of sub-images that more accurately reflect the defect shape when segmenting patterns tailored to this boundary information, significantly improving the accuracy of defect type determination.

[0046] Furthermore, the noise information in the sampled image defines one or more of the noise type and noise intensity. Based on the noise information, a suitable noise filtering scheme can be determined, which can improve the noise removal effect.

[0047] Furthermore, by making the number of classifiers the same as the number of input channels, one classifier can perform classification processing on a sub-image of one color, and further configuring one classifier to have multiple classification heads, one classification head is used to output a defect type, so that accurate processing of different defect types in sub-images of the same color can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a defect detection method according to an embodiment of the present invention;

[0049] Figure 2 A flowchart of a method for obtaining a sample image containing a wafer according to an embodiment of the present invention;

[0050] Figure 3 This is a structural diagram of a sampling image in one embodiment of the present invention;

[0051] Figure 4 for Figure 3 Schematic diagram of the structure of the sub-image obtained by segmentation of the sampled image;

[0052] Figure 5 Schematic diagram of the prediction principle of a defect classification model in one embodiment of the present invention;

[0053] Figure 6 This is a schematic structural diagram of a defect detection device in one embodiment of the present invention;

[0054] Figure 7 The figure is a schematic diagram of an optional hardware structure of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0056] As described in the background art, various defects will inevitably appear on the surface of the wafer, which will affect the performance and reliability of the chip. Therefore, it is necessary to detect the defect information on the wafer.

[0057] In one inspection approach, optical microscopes use visible light to image wafers, enabling rapid and intuitive observation of large-scale defects on the wafer surface. However, the resolution of optical microscopes is limited by the optical diffraction limit, making it difficult to detect small defects at the nanometer level.

[0058] In addition, optical microscopes have limitations and are time-consuming when observing three-dimensional defects under the wafer surface, and cannot meet the rapid detection needs in large-scale production.

[0059] In another detection scheme, a gold nanoparticle solution is pre-coated on the wafer surface, and through chemical modification, the gold nanoparticles are selectively attached to the defective locations of the wafer through chemical bonding or physical adsorption.

[0060] In this way, during imaging detection, the signal in the area coated with gold nanoparticles is enhanced, and the area where the defect is located can be located based on the characteristics of the signal change.

[0061] However, in practical applications, the defect detection method of coating gold nanoparticle solution still has some shortcomings:

[0062] First, single defect detection.

[0063] Gold nanoparticles are typically designed to detect specific types of defects, such as particle contamination or uneven oxide layers. Because the surface modification of gold nanoparticles needs to be optimized for specific defect types, identifying multiple defect types in a single test is difficult.

[0064] Different types of defects may require different types of gold nanoparticle modifications, which increases the complexity and time cost of the detection process.

[0065] Second, the surface modification is complicated.

[0066] The surface modification process of gold nanoparticles is complex and needs to be carried out under specific chemical reaction conditions, such as controlling pH, temperature and reaction time.

[0067] These steps require precise manipulation and strict control to ensure that gold nanoparticles can selectively attach to defect sites. The complex surface modification process not only increases preparation costs but also reduces production efficiency. In large-scale manufacturing, the complexity of surface modification may become a bottleneck, limiting the widespread application of gold nanoparticle technology.

[0068] Third, there is a lack of data collection and intelligent control.

[0069] The signal from gold nanoparticles may change over long periods of time. For example, under prolonged exposure to light or high temperatures, the optical properties of gold nanoparticles may degrade or drift, causing the signal intensity to weaken or change, thus affecting the stability and reliability of the test results.

[0070] In order to solve the above technical problems, the present invention provides a defect detection method. On the one hand, the sampling image has both color areas and grayscale areas, and the color areas are the defect areas, and the grayscale areas are the non-defect areas. Therefore, by utilizing the color difference between the color areas and the grayscale areas, the defect areas can be accurately distinguished, and by making different types of defects have different colors, effective distinction between different types of defects is achieved. Therefore, when performing the extraction process, a sub-image can contain a color area of ​​one defect type, which reduces the interference of other factors on the defects; on the other hand, the input layer of the defect classification model includes multiple input channels, and each input channel corresponds to the color of a defect type one by one. Therefore, when the defect type is identified through the defect classification model, the recognition accuracy can be improved.

[0071] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are exemplarily described below with reference to the accompanying drawings.

[0072] See also Figure 1 A flow chart of a defect detection method according to an embodiment of the present invention is shown in FIG. Figure 1 As shown, you can perform the following steps:

[0073] S10, providing a wafer.

[0074] In some embodiments, according to the process stage or surface structure, the wafer may be a bare wafer or a wafer with at least one pattern layer formed thereon.

[0075] In some embodiments, based on functional division, the wafer may be a wafer used to form logic circuits, memories, sensors and other devices.

[0076] In this embodiment, the wafer may be a CMOS wafer, that is, a semiconductor wafer manufactured using complementary metal oxide semiconductor (CMOS) technology.

[0077] S20, obtaining a sampling image including the wafer, wherein the sampling image is obtained when the wafer is irradiated with light, and the sampling image includes a color area and a grayscale area, wherein the color area is an area where defects are located, and different types of defects have different colors, and the grayscale area is a non-defective area.

[0078] In some embodiments, when the wafer is in a state of being illuminated by light, the surface of the wafer can be illuminated, and then a sampling image at least containing the wafer can be acquired through an image acquisition device.

[0079] Furthermore, in this solution, the captured image can have both color areas and grayscale areas, so that the boundaries of these two types of areas can be more clearly determined to better distinguish between defective areas and non-defective areas.

[0080] As an example, see Figure 3 The structural diagram of a sampling image in one embodiment of the present invention is shown in FIG. Figure 3 As shown, the captured image also includes color areas (e.g., Figure 3 3 colored areas indicated by the three different colors shown), and a grayscale area surrounding the colored areas.

[0081] It should be noted that, first, Figure 3 The colors shown are for illustrative purposes only and are used to indicate the use of different colors to represent different defects. They should not be construed as limitations of the present invention. Second, the colors displayed by the colored areas are determined by the quantum dots used, the defect type, and the light used. Figure 3 It only shows the color area and grayscale area; third, Figure 3 The defect shapes indicated by colors are also illustrative and are used to characterize the outline of the defect.

[0082] In some embodiments, a device with high-resolution imaging capabilities may be used to acquire a sample image containing the wafer.

[0083] Furthermore, the sampling image may include a surface defect image and an internal structure image, thereby detecting defects deeper inside the wafer and the corresponding defect types.

[0084] In some embodiments, different types of defects have different colors, so by collecting the number of colors in the colored area in the image, the number of types of defects existing on the wafer surface under the current inspection environment can be determined.

[0085] It should be noted that the "different types of defects" in this solution refer to defects caused by different materials. For example, defects caused by oxides, defects caused by organic matter, and defects caused by metals. Defects caused by the same material are represented by the same colored area.

[0086] Furthermore, defects caused by oxides may include: oxide blistering, silicon oxide electron hole traps, oxide moisture absorption leading to electrochemical corrosion, and uneven oxide thickness, so color A can be used to represent defects related to oxides; defects caused by organic matter may include: polishing liquid residue, photoresist residue, and stress warping (i.e., organic matter has a high thermal expansion coefficient and is not properly matched with inorganic materials), so color B can be used to represent defects related to organic matter; defects caused by metals may include: metal scratches, metal fractures, metal bridging, open circuits, and metal holes, etc., so color C can be used to represent defects related to organic matter.

[0087] Among them, due to the large differences in material properties of oxides, organic matter and metals, the colors A, B and C are different to highlight different types of defects. Specifically, the difference between different defects lies in their shapes.

[0088] As a non-limiting example, Figure 3 In the figure, green is used to represent oxide blistering, blue is used to represent metal scratches, and red is used to represent organic particles.

[0089] In some embodiments, different color areas can be configured to correspond to one color, thereby achieving clustering processing of defects of the same type, and the differences between different colors reduce the difficulty of sampling image segmentation processing, so that the segmented sub-image can contain complete color areas to improve defect detection accuracy.

[0090] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of a method for obtaining a sample image containing a wafer according to an embodiment of the present invention. Figure 2 , comprising steps S22 to S26:

[0091] S22, coating the wafer with a precursor solution containing quantum dots, wherein the precursor solution further comprises functional groups, and the functional groups can selectively attach to defective positions of the wafer.

[0092] Among them, quantum dots (QDs) are a type of nanomaterial with unique optical properties that can emit strong fluorescence signals when excited by light of a specific wavelength.

[0093] In some embodiments, probes specifically targeting defect locations can be designed based on the physical and chemical properties of quantum dots. Furthermore, by chemically decorating quantum dots, they can selectively attach to defect locations on the wafer through chemical bonding or physical adsorption, thereby enhancing defect visibility.

[0094] Furthermore, when modifying the surface of the quantum dots, specific functional groups are introduced so that the quantum dots can be firmly attached to the defect sites.

[0095] In some embodiments, by introducing functional groups, the quantum dots are bound to the active sites at the defects through covalent bonds or coordination bonds; for example, through weak interactions such as van der Waals forces, electrostatic interactions, or hydrophobic interactions, the quantum dots are adsorbed at the defects.

[0096] In other words, when the possible defects of the wafer are determined, and based on the type of material causing the defects, the quantum dots contained in the precursor solution can be pre-configured so that the quantum dots can selectively attach to the defect locations of the wafer.

[0097] In some embodiments, suitable quantum dot materials, such as CdTe, CdS, etc., are selected to ensure that their optical properties meet detection requirements.

[0098] In some embodiments, the precursor solution may include multiple quantum dot materials. Different quantum dot materials exhibit differences, allowing for the simultaneous visualization of multiple colored regions during a single inspection. A single colored region typically corresponds to a defect caused by a single material, enabling simultaneous detection of multiple defect types, significantly improving inspection efficiency.

[0099] It should be noted that the precursor solution includes a variety of quantum dot materials, and the quantum dot materials repel each other, so that the quantum dots are evenly dispersed in the solution.

[0100] In some embodiments, the preparation process of the precursor solution may include: selecting a suitable solvent (e.g., water, organic solvent, etc.) and a stabilizer, then adding the quantum dot material to the solvent and mixing; then, surface modifying the quantum dots and introducing different functional groups so that they can selectively attach to specific defect locations, for example: using chemical reagents such as thiol compounds, amino groups, and carboxyl groups to ensure the functional modification of the quantum dot surface.

[0101] Accordingly, the coating process may include selecting a coating method, such as spray coating, spin coating, or dip coating, to ensure uniformity of the coating process.

[0102] It should be noted that the concentration of the precursor solution and coating parameters (such as speed, pressure, time, etc.) are adjusted according to the selected coating method.

[0103] The quantum dot solution is evenly coated on the wafer surface to ensure uniform distribution of quantum dots.

[0104] Among them, automatic coating equipment is used to ensure the consistency and repeatability of the coating, so that the quantum dots can be selectively attached to the defect location through chemical bonding or physical adsorption to form a clear mark.

[0105] Next, the reaction conditions (such as temperature, time, and pH) are controlled to ensure the selective attachment of quantum dots, thereby ensuring that all defect locations are marked by quantum dots, facilitating subsequent imaging and detection.

[0106] In some embodiments, the coating effect can also be checked microscopically to ensure the accuracy and reliability of the marking.

[0107] In other words, this approach leverages the unique physical and chemical properties of quantum dots to design probes specifically targeting defect locations. Furthermore, quantum dots can selectively attach to defect locations on the wafer through chemical bonding or physical adsorption, enhancing defect visibility during imaging. This approach offers at least the following advantages:

[0108] High sensitivity and selectivity, that is, quantum dots can be designed to be highly selective for specific types of defects, which can significantly improve detection sensitivity; simultaneous detection of multiple defects, that is, through different types of quantum dots, multiple types of defects can be identified in a single detection, improving detection efficiency.

[0109] S24, using light that interacts with the quantum dots to irradiate the wafer, activating the quantum dots so that the quantum dots emit light.

[0110] As mentioned earlier, quantum dots emit a strong fluorescence signal when excited by light of a specific wavelength. Therefore, when determining which quantum dots to use for this inspection, light of a specific wavelength can be selected to illuminate the wafer. When the light strikes the quantum dots, they become activated, revealing the defect area.

[0111] It should be noted that when multiple quantum dots are present at the same time, light with multiple specific wavelengths is used for irradiation.

[0112] For example, during detection, multiple excitation light sources with different wavelengths may be used to illuminate the wafer, causing the quantum dots attached to the defects to emit light.

[0113] In some embodiments, the wafer surface can be illuminated by a tunable light source (eg, a laser and a tunable filter adapted to the laser).

[0114] S26, acquiring an image of the wafer in the current light-emitting state to obtain the sample image.

[0115] In some embodiments, when light is used to illuminate an image, due to the presence of quantum dots, a portion of the wafer is in a luminous state, and a sample image can be acquired at this time.

[0116] For example, see Figure 3By sampling the wafer coated with the precursor solution, part of the wafer can emit light of the corresponding color, so that the defective area in the sampled image is clearly distinguished from the non-defective area. This can improve the distinction between defects and non-defects and help reduce the difficulty of image segmentation.

[0117] And by setting different types of quantum dots in the precursor solution, multiple types of defects can be identified simultaneously in a single detection, improving detection efficiency and accuracy.

[0118] During the actual sampling process, the inventors discovered that when performing detection in different detection scenarios, the imaging effects are different, which in turn affects the consistency of the detection effects.

[0119] Based on this, the defect detection method may further include: obtaining key feature parameters of the sample image in the current detection environment and the imaging effect of the sample image; and adjusting the key feature parameters according to the imaging effect of the sample image.

[0120] In other words, if the wafer remains unchanged, the key characteristic parameters will determine the imaging effect of the sampled image. Different key characteristic parameters may correspond to different imaging effects, which will affect the accuracy of defect identification. Therefore, the key characteristic parameters can be adjusted to ensure that the imaging effect meets the inspection requirements.

[0121] In some embodiments, key characteristic parameters may include light source intensity, exposure time, imaging resolution, etc., while imaging effect may be reflected by image clarity, contrast, signal-to-noise ratio, etc. In this way, it is possible to determine whether at least one of the image clarity, contrast, or signal-to-noise ratio meets the detection requirements by using one of the image clarity, contrast, or signal-to-noise ratio, thereby adjusting at least one of the light source intensity, exposure time, and imaging resolution so that the image clarity meets the preset clarity, the contrast meets the preset contrast, and the signal-to-noise ratio does not exceed the set signal-to-noise ratio, thereby improving the imaging effect.

[0122] In short, by obtaining the quality evaluation parameters of the sampled image under different key feature parameters, and based on the quality evaluation parameters, at least one is selected from multiple different key feature parameters as the key feature parameter under the current detection environment.

[0123] Through this real-time feedback strategy, the imaging effect and detection efficiency can be optimized, and the stability and reliability of the detection process can be improved.

[0124] S30 , performing extraction processing on the sampled image according to the color region to obtain a plurality of sub-images, wherein each sub-image includes a color region of one defect type.

[0125] In some embodiments, the color area is generally adjacent to the grayscale area, and the color difference between the two is large, so the boundary contour of the color area can be more clearly defined, so that extraction processing can be performed according to the location of the color area to obtain multiple sub-images.

[0126] As a non-limiting example, “extraction processing” refers to segmenting an image into multiple sub-images; wherein the number of sub-images is determined based on the number of defects.

[0127] For example, combined with Figure 3 and Figure 4 , Figure 4 for Figure 3 Schematic diagram of the structure of the sub-image obtained by segmenting the sampled image. There are multiple color regions, and each color region has its own corresponding color, so three pictures can be obtained, Fig1, Fig2 and Fig3.

[0128] In some embodiments, for defects caused by the same type of material, the colors of the corresponding color regions are consistent when the quantum dots are coated, for example Figure 3 The color corresponding to the multiple oxide bubbles shown is green, and the color corresponding to the multiple metal scratches is blue. Therefore, when dividing, clustering processing can be performed, and these areas can be segmented together, and then segmented here according to their respective contours; or, they can be obtained separately according to the displayed contours.

[0129] In addition, the segmentation process can be performed because different defects exhibit different morphological features. Therefore, based on color distribution and morphological similarity, when performing the segmentation process, the sub-image can contain the defect.

[0130] In some embodiments, step S30 may specifically include: determining the boundary information of each color area based on the distribution position of the color area; determining the segmentation pattern to be used based on the boundary information of each color area; and using the segmentation pattern to segment the sampled image to obtain a sub-image of the color area containing at least one defect type.

[0131] Specifically, during image acquisition, coordinates can be established for each pixel in the sampled image based on the image acquisition device's own coordinate system information. Based on the color distribution, the distribution of the colored area can be determined, and accordingly, the boundary contours of the colored area can be determined. This allows the selection of a segmentation pattern that matches the colored area and the execution of the segmentation operation, resulting in a sub-image that more accurately reflects the defect's appearance, significantly improving the accuracy of defect type determination.

[0132] In some embodiments, an adapted segmentation pattern may be selected based on the similarity between the outline of the sub-pattern and the outline of the segmentation pattern.

[0133] In some embodiments, the process of acquiring the sample image is inevitably affected by noise, which is reflected in the sample image and affects the cutting accuracy.

[0134] Based on this, in this solution, before the sampled image is segmented according to the color area to obtain multiple sub-images, a noise filtering process may be performed.

[0135] For example, noise information in the sampled image is determined, wherein the noise information includes one or more of noise type and noise intensity; and based on the noise information, a noise filtering scheme is determined to perform noise filtering processing on the sampled image.

[0136] In other words, the noise filtering method to be used is determined according to the noise type, and the specific parameters of the noise filtering process are determined according to the noise intensity.

[0137] For example, for salt and pepper noise, median filtering is used; for Gaussian noise, Gaussian filtering is used; for mixed noise, a combination of the two filtering methods is used.

[0138] In some embodiments, determining noise information in the sampled image includes:

[0139] A1) dividing the sampled image into sub-units of preset pixels, with different color areas not overlapping each other.

[0140] In some embodiments, the sampled image may be divided with a fixed pixel size to obtain a plurality of sub-units, wherein each sub-unit is similar to a checkerboard, and the plurality of sub-units form a checkerboard network.

[0141] For example, a sampled image with high resolution pixels can be divided into multiple sub-units according to the size of 16×16 pixels.

[0142] In some embodiments, defects can be revealed by coating quantum dots, so that different colored areas are isolated from each other when divided.

[0143] A2) Using a residual neural network, obtain the local context feature map of each sub-unit and the descriptor of the preset dimension of the sub-unit.

[0144] In some embodiments, the middle layer of the residual neural network can be used to obtain a 256-dimensional context feature map of each sub-unit, thereby obtaining an abstract representation of the corresponding area.

[0145] Furthermore, the 64-dimensional texture features of the subunit are extracted by SIFT. Specifically, after SIFT generates a high-dimensional descriptor through key point detection, it performs dimensionality reduction processing to obtain a geometric invariant feature of a preset dimension.

[0146] In other words, the local context feature map provides semantic information, and the descriptor of preset dimensions supplements the geometric details.

[0147] A3) Adopt the channel attention mechanism to fuse the local context feature maps and descriptors of each sub-unit to generate a global feature vector.

[0148] In some embodiments, spatial consistency mapping of local context feature maps and descriptors is achieved through coordinate alignment, and feature splicing or weighted fusion strategies are adopted to combine the semantic representation ability of residual neural networks with the geometric robustness of SIFT. This can not only utilize the global context perception advantages of deep learning, but also retain the accurate depiction of local details, significantly enhancing the feature expression capabilities in complex scenarios.

[0149] A4) using a dual-branch network structure to determine the noise information of the global eigenvector.

[0150] In some embodiments, a classification perception module based on a convolutional neural network is used as a dual-branch network structure, wherein one branch network structure is used to predict the noise type (e.g., 5-category classification) and the other branch network structure is used for the noise intensity (e.g., 0-1 normalized value).

[0151] Among them, the classification perception module based on convolutional neural network can be implemented using convolutional layers and fully connected layers.

[0152] By executing steps A1) to A4), the noise information can be determined, and then the noise reduction algorithm library can be dynamically called according to the noise information, for example, DnCNN is used for Gaussian noise, and StripeNet is used for stripe noise.

[0153] In some embodiments, after performing noise filtering, a multi-scale cosine weighted fusion strategy may be used to eliminate splicing traces.

[0154] For example, the fusion process may be performed by combining the block results of the original scale (the weight may be 0.6), the 1 / 2 scale (the weight is 0.3), and the 1 / 4 scale (the weight is 0.1).

[0155] Therefore, by adopting the above solution, multiple sub-images can be obtained, and each sub-image has been subjected to noise filtering, so that the color area can be defined more clearly, thereby improving the accuracy of the defect type.

[0156] S40: Input the plurality of sub-images into a defect classification model to determine the defect type corresponding to each sub-image.

[0157] The input layer of the defect classification model includes multiple input channels, and each input channel corresponds to the color of a defect type.

[0158] As mentioned above, for defects caused by one material, the color represented by the colored area is the same. In this way, the type of color is consistent with the number of materials. Therefore, multiple input channels are set in the input layer of the defect classification model, so that one input channel can correspond one-to-one to the sub-image corresponding to one material.

[0159] In this way, when the corresponding input channel receives the sub-images, fusion and classification processing can be performed to determine the type of defects in each sub-image.

[0160] In some embodiments, see Figure 5 The schematic diagram of the prediction principle of a defect classification model in one embodiment of the present invention is shown in FIG. Figure 5 As shown, the defect classification model may include: an input layer including multiple input channels (as a non-limiting example, Figure 5 3 input channels are shown), each input channel corresponds to the color of a defect type (as a non-limiting example, along the current viewing angle, from top to bottom, input channel 1 is used to input sub-images related to oxide defects, input channel 2 is used to input sub-images related to organic defects, and input channel 3 is used to input sub-images related to metal defects).

[0161] The backbone feature extraction layer is the core component of the defect classification model and is used to generate high-dimensional features.

[0162] For example, see Figure 3 If the sub-image in the input channel is related to oxide defects, then after passing through the backbone feature extraction layer, the oxide high-dimensional features are output; if the sub-image in the input channel is related to organic defects, then after passing through the backbone feature extraction layer, the organic high-dimensional features are output; if the sub-image in the input channel is related to metal defects, then after passing through the backbone feature extraction layer, the metal high-dimensional features are output.

[0163] In some embodiments, the backbone feature extraction layer can be implemented by CNN (convolutional neural network).

[0164] Multiple classifiers are used to output defect types based on high-dimensional features.

[0165] In some embodiments, the number of classifiers is the same as the number of input channels.

[0166] For example, Figure 5Three classifiers are shown, among which classifier 1 is used to output oxide defects based on the high-dimensional features of oxides; classifier 2 is used to output organic defects based on the high-dimensional features of organics; and classifier 3 is used to output metal defects based on the high-dimensional features of metals.

[0167] In some embodiments, a classifier has multiple classification heads (wherein the number of classification heads is determined based on the type of defects caused by a single material), one classification head is used to output a defect type, and one of the input channels, the backbone feature extraction layer and one of the classification heads form a classification channel.

[0168] For example, see Figure 5 Classifier 1 has 5 classification heads to detect 5 defects corresponding to oxides; Classifier 2 has 8 classification heads to detect 8 defects corresponding to organic matter; Classifier 3 has 6 classification heads to detect 6 defects corresponding to metals.

[0169] In other words, this scheme uses different input channels to receive sub-images of different colors, and reduces overhead by sharing the backbone feature extraction layer. Afterwards, the defect type is predicted through the classifier corresponding to the input channel.

[0170] In some embodiments, the plurality of sub-images are input into a defect classification model to determine the defect type corresponding to each sub-image, including:

[0171] B1) Inputting the sub-image into the backbone feature extraction layer through the input channel of the corresponding color according to the color area of ​​the defect type contained in the sub-image.

[0172] As can be seen from the preceding, defects caused by a single material type appear in a consistent color. Since each input channel corresponds to a single color, pre-classification can be performed based on the color of the color region, allowing sub-images with the same color to be output to the same input channel.

[0173] B2) Using the backbone feature extraction layer, perform feature extraction and feature fusion processing on the sub-image to generate a fused feature vector.

[0174] In some embodiments, B2) may include:

[0175] Extracting backbone features of the sub-image at different scales to form a multi-scale feature map, and extracting a texture feature vector, a shape feature vector, and a position coding matrix of the sub-image, wherein the position coding matrix is ​​generated based on the size of the sub-image.

[0176] In some embodiments, the sub-image may be converted to Lab space, and the mean, variance, and skewness of the a / b channels may be calculated to generate a three-dimensional vector F_color as a multi-scale feature map.

[0177] In some embodiments, a shallow convolutional layer is used to extract contrast, energy, and correlation indicators based on the gray-level co-occurrence matrix to generate a three-dimensional texture feature vector F_texture.

[0178] In some embodiments, the defect area is extracted, and the aspect ratio and Fourier descriptor are analyzed and calculated to generate a 3D shape feature vector F_shape.

[0179] In some embodiments, if the size of the input sub-image is HxW, a row coding matrix Row_Mat and a column coding matrix Col_Mat may be generated.

[0180] Based on the channel attention mechanism, the multi-scale feature map and the texture feature map are spliced ​​to generate a first joint vector, and the weights of each feature dimension in the first joint vector are determined by performing average pooling, full connection and mapping processing, and a first enhanced vector is generated based on the first joint vector and its corresponding weights.

[0181] In some embodiments, a 6-dimensional first joint vector can be formed by concatenating and fusing the 3-dimensional vector F_color and the 3-dimensional texture feature vector F_texture.

[0182] Afterwards, through global average pooling, fully connected layers, and Sigmoid, the weights of each feature dimension in the 6-dimensional first joint vector can be generated.

[0183] Furthermore, through weighted processing, a first enhanced vector containing 6-dimensional enhanced features can be formed.

[0184] In some embodiments, only the weight corresponding to the 3-dimensional texture feature vector F_texture may be calculated.

[0185] The shape feature map and the position encoding matrix are spliced ​​to generate a second joint vector, and the weights of each feature dimension in the second joint vector are determined by performing convolution and mapping processing, and a second enhancement vector is generated based on the second joint vector and its corresponding weights.

[0186] In some embodiments, a 5-dimensional second joint vector can be formed by concatenating and fusing the shape feature vector F_shape and the position encoding matrix.

[0187] Afterwards, through convolution and Sigmoid, the weights of each feature dimension in the 5-dimensional second joint vector can be generated.

[0188] The first enhanced vector and the second enhanced vector are fused to generate the fused feature vector.

[0189] B3) Determining the defect type corresponding to the sub-image through a classification head corresponding to the input channel according to the fused feature vector.

[0190] In some embodiments, by fusing the first enhancement vector and the second enhancement vector, a 5-dimensional F_fused can be generated, and then a classification head (eg, a fully connected layer) is used to output a corresponding label to characterize the defect type of the sub-image.

[0191] It's important to note that the "defect classification model" involved in this solution can be trained based on the sub-images described in this solution. During training, the input sub-images are labeled to indicate the defect type. This allows a loss function to be established based on the defect type predicted by the classification head and the corresponding defect type. This allows the defect classification model to be adjusted until each classification head outputs the correct defect type, terminating training or reaching a set number of training rounds.

[0192] In some embodiments, defects on a wafer are random, and the input layer of the defect classification model is configured to include input channels corresponding to all defects. Therefore, before inputting the multiple sub-images into the defect classification model, the defect detection method may further include:

[0193] Perform statistical analysis on the color channels of all sub-images to determine parameter characteristics of each color channel, wherein the parameter characteristics include one or more of average brightness, signal strength, and effective pixel ratio; when it is determined that the parameter characteristics do not meet the set requirements, disable the input channel corresponding to the sub-image.

[0194] For example, when the parameter feature is average brightness, if the average brightness is lower than the set brightness, it means that the current sub-image cannot meet the detection requirements. Therefore, the input channel corresponding to the color displayed by the sub-image can be disabled to avoid meaningless processing of these information-free channels.

[0195] In other words, before making a prediction, a color recognition operation is performed. If there is no color or the color cannot meet the detection requirements, the corresponding channel is skipped to reduce overhead.

[0196] In some embodiments, based on the identified defects, a detailed inspection report may be generated, including information such as the type, location, and quantity of the defects.

[0197] Furthermore, data analysis software can be used to generate visual reports showing defect distribution maps and statistical analysis results, providing a basis for quality control and process improvement.

[0198] It is understandable that the above-mentioned embodiments provide multiple implementation plans, and the various implementation plans can be combined and cross-referenced with each other without conflict, thereby extending multiple possible implementation plans, which can all be considered as embodiment plans disclosed and open in the embodiments of this application.

[0199] The present invention also provides a defect detection device corresponding to the above-mentioned defect detection method, which is described in detail below through specific embodiments with reference to the accompanying drawings.

[0200] See also Figure 6 The structural diagram of a defect detection device in one embodiment of the present invention is shown in FIG. Figure 6 As shown, the defect detection device M may include:

[0201] An acquisition unit M1 is configured to acquire a sample image of a wafer, wherein the sample image is obtained when the wafer is irradiated with light and comprises a color region and a gray region. The color region is where defects are located, and different types of defects have different colors. The gray region is a non-defective region.

[0202] An extraction unit M2 is configured to extract the sampled image according to the color region to obtain a plurality of sub-images, wherein each sub-image includes a color region of one defect type;

[0203] A processing unit M3 is provided with a defect classification model, and the defect classification model is used to determine the defect type corresponding to each sub-image based on the sub-image, wherein the input layer of the defect classification model includes multiple input channels, and each input channel corresponds to the color of a defect type.

[0204] The specific working principles and processes of the acquisition unit M1 , the extraction unit M2 and the processing unit M3 can be found in the above examples.

[0205] It is understandable that the division of the above units is only a division of logical functions, and in actual implementation, they can be fully or partially integrated into one physical entity, or physically separated. In addition, the above modules can be implemented in the form of processor calling software.

[0206] Correspondingly, the present invention also provides an electronic device, which can implement the defect detection method provided by the present invention by loading the above-mentioned defect detection method in the form of a program.

[0207] See also Figure 7 , shows a schematic diagram of an optional hardware structure of an electronic device provided by an embodiment of the present invention.

[0208] The device of the present invention includes: at least one processor 01 , at least one communication interface 02 , at least one memory 03 and at least one communication bus 04 .

[0209] In some embodiments, the number of each of the processor 01 , the communication interface 02 , the memory 03 and the communication bus 04 is at least one, and the processor 01 , the communication interface 02 and the memory 03 communicate with each other via the communication bus 04 .

[0210] The communication interface 02 may be an interface of a communication module for network communication, such as an interface of a GSM module.

[0211] The processor 01 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the defect detection method of this embodiment.

[0212] The memory 03 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0213] The memory 03 stores one or more computer instructions, and the one or more computer instructions are executed by the processor 01 to implement the defect detection method provided in the aforementioned embodiment.

[0214] It should be noted that the above-mentioned electronic device may also include other devices (not shown) that may not be necessary for understanding the disclosure of the embodiments of the present invention; since these other devices may not be necessary for understanding the disclosure of the embodiments of the present invention, the present invention will not introduce them one by one.

[0215] Accordingly, the present invention further provides a computer program product, comprising a computer program / instructions, which are used to implement the defect detection method of the present invention when executed by a processor.

[0216] The present invention also provides a storage medium, wherein the storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the defect detection method provided in the above embodiment.

[0217] The embodiments of the present invention described above are combinations of elements and features of the present invention. Unless otherwise mentioned, elements or features may be considered as optional. Each element or feature may be put into practice without being combined with other elements or features. In addition, embodiments of the present invention may be constructed by combining some elements and / or features. The order of operations described in the embodiments of the present invention may be rearranged. Some configurations of any one embodiment may be included in another embodiment, and may be replaced by the corresponding configuration of another embodiment. It is obvious to those skilled in the art that claims that do not have a clear reference relationship to each other in the appended claims may be combined into embodiments of the present invention, or may be used as new claims in amendments after submitting this application.

[0218] The embodiments of the present invention may be implemented by various means such as hardware, firmware, software, or a combination thereof. In a hardware configuration, the method according to the exemplary embodiment of the present invention may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0219] In a firmware or software configuration, embodiments of the present invention may be implemented in the form of modules, procedures, functions, and the like. Software code may be stored in a memory unit and executed by a processor. The memory unit is located inside or outside the processor and may send data to and receive data from the processor via various known means. The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0220] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A defect detection method, characterized in that: include: Provide wafers; Acquire a sample image containing the wafer, wherein the sample image is obtained when the wafer is irradiated with light, and the sample image includes a color area and a grayscale area, wherein the color area is an area where a defect is located, and different types of defects have different colors, and the grayscale area is a non-defective area; According to the color area, the sample image is extracted to obtain a plurality of sub-images, wherein each sub-image includes a color area of ​​one defect type; Inputting the plurality of sub-images into a defect classification model to determine the defect type corresponding to each sub-image; The input layer of the defect classification model includes multiple input channels, and each input channel corresponds to the color of a defect type.

2. The defect detection method according to claim 1, characterized in that: The acquiring of a sample image containing the wafer includes: Coating a precursor solution containing quantum dots on the wafer, wherein the precursor solution further comprises functional groups, wherein the functional groups enable the quantum dots to selectively attach to defective locations of the wafer; irradiating the wafer with light that interacts with the quantum dots to activate the quantum dots and cause them to emit light; An image of the wafer in the current light-emitting state is acquired to obtain the sample image.

3. The defect detection method according to claim 2, characterized in that: Also includes: Obtaining key feature parameters of the sample image in the current detection environment and the imaging effect of the sample image; The key feature parameters are adjusted according to the imaging effect of the sampled image.

4. The defect detection method according to claim 1, wherein: The sampling image is segmented according to the color area to obtain a plurality of sub-images, including: Determining boundary information of each color area according to the distribution positions of the color areas; Determine the segmentation pattern to be used based on the boundary information of each color area; The sampling image is segmented using the segmentation pattern to obtain a sub-image containing a color area of ​​at least one defect type.

5. The defect detection method according to claim 1 or 4, characterized in that: Before segmenting the sampled image according to the color region to obtain a plurality of sub-images, the method further includes: Determining noise information in the sampled image, wherein the noise information includes: one or more of noise type and noise intensity; Based on the noise information, a noise filtering scheme is determined to perform noise filtering on the sampled image.

6. The defect detection method according to claim 5, characterized in that: The determining of noise information in the sampled image includes: Dividing the sampled image into sub-units of preset pixels, with different color areas not overlapping each other; A residual neural network is used to obtain the local context feature map of each sub-unit and the descriptor of the preset dimension of the sub-unit; Adopting the channel attention mechanism, the local context feature maps and descriptors of each sub-unit are fused to generate a global feature vector; A dual-branch network structure is adopted to determine the noise information of the global eigenvector.

7. The defect detection method according to claim 1, characterized in that: The defect classification model further includes: a backbone feature extraction layer and a plurality of classifiers; The number of the classifiers is the same as the number of the input channels, one classifier has multiple classification heads, one classification head is used to output a defect type, and one input channel, the backbone feature extraction layer and one classification head form a classification channel.

8. The defect detection method according to claim 7, characterized in that: Inputting the plurality of sub-images into a defect classification model to determine a defect type corresponding to each sub-image includes: According to the color area of ​​the defect type contained in the sub-image, the sub-image is input to the backbone feature extraction layer through the input channel of the corresponding color; Using the backbone feature extraction layer, perform feature extraction and feature fusion processing on the sub-image to generate a fused feature vector; According to the fused feature vector, a defect type corresponding to the sub-image is determined by a classification head corresponding to the input channel.

9. The defect detection method according to claim 7 or 8, characterized in that: Before inputting the plurality of sub-images into the defect classification model, the method further includes: Performing statistical analysis on the color channels of all sub-images to determine parameter characteristics of each color channel, wherein the parameter characteristics include one or more of average brightness, signal strength, and effective pixel ratio; When it is determined that the parameter feature does not meet the set requirement, the input channel corresponding to the sub-image is disabled.

10. A defect detection device, characterized in that: include: an acquisition unit, configured to acquire a sample image of the wafer, wherein the sample image is obtained when the wafer is irradiated with light, and the sample image comprises a color area and a gray area, wherein the color area is an area where defects are located, and different types of defects have different colors, and the gray area is a non-defective area; an extraction unit, configured to extract the sampled image according to the color region to obtain a plurality of sub-images, wherein each sub-image includes a color region of one defect type; A processing unit is provided with a defect classification model, and the defect classification model is used to determine the defect type corresponding to each sub-image based on the sub-image, wherein the input layer of the defect classification model includes multiple input channels, and each input channel corresponds to the color of a defect type.

11. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory is suitable for storing one or more computer instructions, and when the processor runs the computer instructions, the method executes the steps of the defect detection method according to any one of claims 1 to 9.

12. A computer storage medium, characterized in that The storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the steps of the defect detection method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The method comprises computer instructions, which are used to implement the steps of the defect detection method according to any one of claims 1 to 9 when executed by a processor.

Images