Defect detection method and device and electronic equipment

Abstract

The invention provides a defect detection method and device and electronic equipment, and belongs to the technical field of automatic optical detection.The defect detection method comprises the steps that a bright field image and a dark field image of a to-be-detected area of a wafer are obtained; feature extraction is carried out on the bright field image and the dark field image, and multi-feature spatial data is constructed; performing context analysis on the multi-feature space data, and determining context attribute information corresponding to each local area in the to-be-detected area of the wafer; according to the context attribute information, self-adaptive fusion weights of all the local areas are generated, and the self-adaptive fusion weights comprise weight components corresponding to the bright field image and the dark field image respectively; and according to the adaptive fusion weight, carrying out fusion processing on the bright field image and the dark field image to generate a defect enhanced image. Complementary information in bright and dark field multi-mode image data can be deeply mined, rough background noise can be inhibited, and feature expression of weak defects can be remarkably enhanced.

Description

Technical Field

[0001] This disclosure relates to the field of automated optical inspection technology, and in particular to a defect detection method, apparatus and electronic equipment. Background Technology

[0002] In integrated circuit manufacturing, as device feature sizes continue to shrink, the requirements for wafer surface quality inspection are becoming increasingly stringent. Automated Optical Inspection (AOI) is currently the primary means of controlling wafer surface defects. Depending on the imaging optical path, existing optical inspection systems are typically equipped with two imaging channels: bright-field (BF) and dark-field (DF). Bright-field imaging mainly utilizes specular reflection light and is suitable for detecting defects with large-area contamination or smooth morphology; dark-field imaging, on the other hand, receives surface scattered light and is more sensitive to defects with high scattering characteristics, such as small particles and scratches.

[0003] However, existing detection technologies have significant shortcomings for wafers with rough surfaces: the micro-textures of rough surfaces generate high background noise, resulting in insufficient contrast of small defects in a single image. Even with some fixed fusion strategies, they cannot adapt to different defect types and roughness variations.

[0004] Therefore, for wafers with rough surfaces, there is an urgent need for a method that can improve the accuracy of inspection. Summary of the Invention

[0005] This disclosure provides a defect detection method, apparatus, and electronic device that can deeply mine complementary information in bright and dark field multimodal image data, and can significantly enhance the feature expression of weak defects while suppressing coarse background noise.

[0006] The technical solution disclosed herein is implemented as follows: In a first aspect, this disclosure provides a defect detection method, which includes: acquiring a bright-field image and a dark-field image of a wafer region to be tested; extracting features from the bright-field image and the dark-field image respectively to construct multi-feature space data; performing context analysis on the multi-feature space data to determine the context attribute information corresponding to each local region in the wafer region to be tested; generating adaptive fusion weights for each local region based on the context attribute information, wherein the adaptive fusion weights include weight components corresponding to the bright-field image and the dark-field image respectively; and performing fusion processing on the bright-field image and the dark-field image based on the adaptive fusion weights to generate a defect-enhanced image.

[0007] Secondly, this disclosure provides a defect detection device, comprising: an acquisition section, a feature extraction and construction section, a determination section, and a generation section; the acquisition section is configured to acquire a bright-field image and a dark-field image of a wafer region to be tested; the feature extraction and construction section is configured to extract features from the bright-field image and the dark-field image respectively to construct multi-feature space data; the determination section is configured to perform context analysis on the multi-feature space data to determine context attribute information corresponding to each local region in the wafer region to be tested; the generation section is configured to generate adaptive fusion weights for each local region based on the context attribute information, wherein the adaptive fusion weights include weight components corresponding to the bright-field image and the dark-field image respectively; and, based on the adaptive fusion weights, perform fusion processing on the bright-field image and the dark-field image to generate a defect enhancement image.

[0008] Thirdly, this disclosure provides an electronic device including an optical acquisition unit configured to acquire bright-field and dark-field images of a wafer to be inspected; it also includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the wafer inspection method as described in the first aspect.

[0009] Fourthly, this disclosure provides a computer-readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the wafer inspection method as described in the first aspect.

[0010] Fifthly, this disclosure provides a computer program product, wherein the computer program product includes a computer program or instructions, which, when run on a processor, cause the processor to execute the computer program or instructions to implement the steps of the wafer inspection method as described in the first aspect.

[0011] In a sixth aspect, this disclosure provides a chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the wafer inspection method as described in the first aspect.

[0012] This disclosure provides a wafer inspection method. For rough wafer surfaces, in the rough background region occupying most of the wafer image area, the adaptive fusion weights favor the suppression of stable bright or dark fields, thus making the background cleaner and more uniform. In defect regions, the adaptive fusion weights allow high-intensity dark field scattering signals to pass through, resulting in stronger signals corresponding to defects compared to the original bright field image. This makes the final defect-enhanced image not only visually clearer but also easier to separate defects from the background, thereby improving the detection accuracy of rough-surface wafers and reducing the false alarm rate. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the hardware structure of an electronic device provided in this disclosure.

[0014] Figure 2 This is a flowchart illustrating a defect detection method provided in this disclosure.

[0015] Figure 3 This is a schematic diagram of the process for determining multi-feature spatial data provided in this disclosure.

[0016] Figure 4 A flowchart illustrating the context analysis provided in this disclosure.

[0017] Figure 5 This is a schematic diagram of the adaptive fusion weight determination process provided in this disclosure.

[0018] Figure 6 This is a schematic diagram of the fusion processing provided in this disclosure.

[0019] Figure 7 This is a flowchart illustrating another defect detection method provided in this disclosure.

[0020] Figure 8 This is a diagram of a defect detection system architecture provided in this disclosure.

[0021] Figure 9 This is a structural block diagram of a defect detection device provided in this disclosure. Detailed Implementation

[0022] The technical solutions in the embodiments of this disclosure will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure are within the scope of protection of this disclosure.

[0023] The terms “first,” “second,” etc., used in this disclosure are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that this disclosure can be implemented in orders other than those illustrated or described herein, and the objects distinguished by “first,” “second,” etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0024] Based on different optical imaging principles, AOI technology is mainly divided into two modes: BF imaging and DF imaging.

[0025] Browsing (BF) imaging typically uses vertical or coaxial illumination. The optical sensor receives light reflected directly from the wafer surface into the lens. BF imaging is sensitive to detecting changes in surface reflectivity, large areas of contamination, or defects with smooth morphology. The image background is usually bright, and defects appear as areas of grayscale anomalies. Diffusion imaging (DF) typically uses oblique, annular, or low-angle incident illumination. The optical sensor mainly receives light scattered or diffracted from the wafer surface into the lens, while directly reflected light avoids the imaging aperture. DF imaging is extremely sensitive to detecting defects with high scattering characteristics, such as abrupt edges, tiny particles, and fine scratches. Especially on an ideally smooth mirror-like wafer, the dark background is almost completely black, and scattering defects appear as high-contrast bright spots.

[0026] However, in actual semiconductor manufacturing processes, the back side of a wafer, the surface after etching, or the unpolished raw wafer are not perfectly smooth surfaces; they have a certain degree of roughness. This causes a sharp decrease in the detection accuracy of single BF or DF imaging for wafer defects. The reason is as follows: Due to the microscopic texture and irregular grain structure of rough surfaces, strong background scattering signals, often referred to as haze noise, are generated in optical imaging, especially in dark-field mode. This background noise exhibits high-frequency, random speckle characteristics, and its signal intensity is often on the same order of magnitude as that of tiny defects, such as submicron particles or shallow scratches. Traditional gray-scale threshold-based segmentation algorithms struggle to effectively distinguish between real defect signals and background texture noise while maintaining high sensitivity, leading to extremely high false alarm rates or sacrificing detection sensitivity to suppress noise, resulting in missed detections.

[0027] Furthermore, existing multimodal fusion detection techniques typically employ fixed image processing logic. For example, they may use a uniform weighting coefficient to overlay bright and dark field data on the entire wafer image, or combine detection results based on simple logical operations, such as OR operations. However, wafer surface roughness often exhibits spatial non-uniformity, with significant differences in background noise levels across different regions. Fixed fusion strategies fail to detect variations in texture characteristics in localized areas, leading to the introduction of excessive dark field noise in areas with high roughness, while failing to fully utilize the high sensitivity of dark fields in relatively flat areas. This lack of local adaptability limits the device's ability to detect wafers with complex topography.

[0028] In view of the above problems, this disclosure aims to provide a defect detection method that can deeply mine complementary information in bright and dark field multimodal image data, so as to significantly enhance the feature expression of weak defects while suppressing coarse background noise.

[0029] first, Figure 1This is a schematic diagram of the hardware structure of an electronic device corresponding to a defect detection method provided in this embodiment. This electronic device is intended to provide the hardware foundation for subsequent defect detection methods, particularly for achieving high-precision acquisition of bright and dark field images.

[0030] like Figure 1 As shown, the electronic device mainly includes: an optical acquisition unit 100, a processor 200, and a memory 300.

[0031] The optical acquisition unit 100 is used to acquire images of the same wafer's test area under different illumination conditions. Specifically, the optical acquisition unit 100 includes: a bright-field illumination module, a dark-field illumination module, and an imaging sensor.

[0032] The bright-field illumination module 101 is configured to provide perpendicularly or coaxially incident light, typically employing a high-brightness light-emitting diode (LED) or laser diode light source. The wavelength can be selected based on the wafer material characteristics, such as blue light or ultraviolet light, to improve the resolution of fine surface features. The light passes through a semi-transparent mirror and is perpendicularly incident onto the surface of the wafer to be measured through the objective lens. Bright-field imaging mainly relies on specular reflection from the surface, appearing as high brightness for flat areas and as dark spots for defects that absorb or scatter light (such as deep pits or particles of specific materials).

[0033] Dark-field illumination modules are configured to provide oblique or low-angle incident light, such as using a ring light source or a side point light source. The incident angle (the angle between the light ray and the wafer surface) is typically set between 5 and 30 degrees. This low-angle illumination causes the light to undergo primarily specular reflection on flat, smooth surfaces and move away from the aperture of the imaging objective, resulting in a dark background. Only when there are bumps, particles, or edge defects on the surface will the light be scattered and enter the objective. Dark-field imaging is extremely sensitive to scattering defects.

[0034] Imaging sensors are used to acquire images of the wafer's test area under different lighting conditions. These sensors can be Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) sensors. To acquire both bright-field and dark-field images, a dual-optical-path configuration can be used. A beam splitter divides the reflected light path into two paths, each acquired by a synchronously triggered camera, corresponding to the bright-field and dark-field channels. Alternatively, a time-division multiplexing configuration can be employed, using a single high-frame-rate camera in conjunction with a high-speed strobe controller. The bright-field light source is activated in odd-numbered frames, and the dark-field light source is activated in even-numbered frames, enabling rapid alternating acquisition.

[0035] The processor 200 can be a general-purpose central processing unit (CPU) or a heterogeneous computing platform that includes a graphics processing unit (GPU), a neural network processing unit (NPU), or a field-programmable gate array (FPGA). The processor 200 is responsible for controlling the triggering of the optical acquisition unit 100, receiving image data, and executing the deep learning algorithms in subsequent embodiments.

[0036] The memory 300 includes high-speed random access memory (RAM) and non-volatile memory, such as a hard disk. The memory 300 stores a computer program that, when executed by the processor 200, implements the defect detection method described in this disclosure.

[0037] The defect detection method provided in this disclosure will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0038] like Figure 2 As shown, this disclosure provides a defect detection method, which may include the following steps S201 to S205.

[0039] In step S201, bright-field and dark-field images of the area to be tested on the wafer are acquired.

[0040] The wafer inspection area refers to a specific surface region of a silicon substrate or other compound semiconductor substrate that requires optical inspection during the semiconductor manufacturing process. A key characteristic of this area is that its surface morphology is not an ideal mirror finish, but may contain microscopic morphologies, textures, and unexpected anomalies left over from the manufacturing process. Examples include silicon wafer surfaces with a roughness between 1 nm and 10 nm after coarse polishing, the back side of the wafer after back-side thinning processes exhibiting a textured surface covered with grinding wheel patterns, or areas with deposited polycrystalline silicon layers exhibiting a granular crystalline texture.

[0041] Bright-field imaging refers to a two-dimensional digital image formed by illuminating the test area of ​​a wafer with a bright-field illumination module and receiving the light, primarily composed of light reflected from the surface and returning to the lens, by an imaging sensor. In bright-field imaging, flat and highly reflective surface areas will appear as high grayscale values ​​on the sensor, i.e., bright; while areas with pits, absorbing impurities, or drastic topographic changes that cause light to deviate from the optical axis will appear as low grayscale values, i.e., dark. For example, in an 8-bit bright-field image, the background grayscale value of a normal silicon surface might be around 200, while a deep pit defect would appear as a black dot with a grayscale value of 50 in a bright-field image.

[0042] Dark-field imaging refers to a two-dimensional digital image formed by illumination through a dark-field illumination module and reception by an imaging sensor of scattered or diffracted light caused by surface microstructures, particles, or defects. In dark-field imaging, an ideal smooth surface produces no scattering, and the image background grayscale is close to 0, i.e., completely black. However, on rough surfaces, micro-textures produce background scattering. For example, in a dark-field image of a wafer with a rough surface, the background is not completely black, but exhibits hazy noise with a grayscale average of 30-50. The scratches on the rough surface appear as discontinuous high-frequency bright lines in the dark field.

[0043] Bright-field and dark-field images of the wafer under test are acquired simultaneously within an extremely short time window (usually on the microsecond level) or by using physical beam splitting methods to control the optical sensor to perform exposure operations in both bright-field and dark-field modes on the wafer under test area of ​​the same field of view at the same time.

[0044] Due to mechanical vibrations or minor differences in the optical path, physically acquired bright-field and dark-field images may have pixel discrepancies. For example, a particle defect in the bright-field image might have coordinates (100.5, 100.5), while in the dark-field image, due to mechanical vibration, the particle might be located at (101.2, 100.8). Direct fusion would result in ghosting or false edges in the image. After feature extraction, this misalignment would be amplified by the convolutional layers, preventing subsequent neural networks from correctly learning feature correspondences. Therefore, to ensure effective fusion, sub-pixel-level spatial registration is performed on the directly acquired initial bright-field and initial dark-field images to obtain bright-field and dark-field images aligned in the pixel coordinate system.

[0045] Subpixel-level spatial configuration refers to the process of geometrically transforming two images to ensure that the physical spatial error of corresponding pixels is less than one pixel unit. Specifically, the offsets δx and δy between the two images can be calculated using phase correlation or gradient-based optical flow methods, such as δx = 0.7 pixels and δy = 0.3 pixels. Then, bicubic interpolation is used to resample the initial dark-field image, shifting it by -δx and -δy. This ensures that each subsequent feature describes the comprehensive optical properties of the same tiny physical point on the wafer, eliminating false alarms of edge artifacts caused by registration errors.

[0046] In step S202, features are extracted from the bright field image and the dark field image respectively to construct multi-feature spatial data.

[0047] Feature extraction refers to the process of transforming the original image pixel matrix into a high-dimensional numerical representation that can characterize the local structure, texture, frequency, or semantic information of an image using specific computational operators or function transformations. The original pixel values ​​of an image only represent photon flux and lack neighborhood correlations; feature extraction is used to uncover the underlying features associated with each pixel. For example, shallow feature extraction uses the Sobel operator to extract edge features from bright-field images and Gabor filters to extract texture direction features from dark-field images. Deep feature extraction utilizes the convolutional layers of convolutional neural networks to extract abstract semantic features such as ring structures and linear continuity.

[0048] Multi-feature spatial data refers to a unified data structure formed by combining, stacking, or splicing feature data from bright-field and dark-field images in a specific dimension.

[0049] Specifically, the bright-field image is denoted as I. BF The dark field image is denoted as I. DF The first feature extractor, such as a combination of a series of convolutional layers, activation functions, and pooling layers, is used to extract I... BF Perform multi-scale scanning to generate a bright-field feature map set F BF F BF It may contain low-frequency illumination distribution under bright field, mid-frequency texture direction, and high-frequency edge breakage information. A second feature extractor is used to extract I... DF Processing is required because I DF Due to the different characteristics (sparse bright spots), the parameters or structure of the second feature extractor are specifically designed (or trained) to respond to point scattering sources, ultimately generating a dark field feature map set F. DF F DF The key encoding focuses on the statistical distribution of scattering point intensity, density, local contrast, and background noise. A stitching operation is performed along the channel dimension, mathematically expressed as M. Space =Concat(F BF F DF ), thus obtaining the fusion of F BF and F DF Multi-feature spatial data.

[0050] In the original pixel space, pseudo-defects and real defects generated by a rough background often have the same grayscale value, and cannot be distinguished by grayscale threshold alone. That is, they are linearly inseparable in 1D pixel space. Through feature extraction, the 1D grayscale is expanded into an N-dimensional feature vector. Although the rough protrusion has a high grayscale value in the dark field, it may appear as part of a continuous texture in the bright field feature (feature A), and as diffuse in the dark field feature (feature B). Real particles have a high grayscale value in the dark field, but may appear as isolated points in the bright field feature (feature A'), and as an extremely steep gradient in the dark field feature (feature B'). In the N-dimensional multi-feature space, the point cloud clusters representing the rough background and the point cloud clusters representing real defects become distinguishable due to the different combinations of their multi-dimensional features.

[0051] In step S203, context analysis is performed on the multi-feature spatial data to determine the context attribute information corresponding to each local region in the wafer test area.

[0052] Contextual analysis refers to the process of examining the correlation between a pixel or feature point and its surrounding neighborhood, the global environment, and feature points with topological connections when processing image or feature data, rather than considering individual pixels or feature points in isolation. For example, neighborhood analysis assesses the variance of features within a 5×5 area around a pixel; a large and chaotic variance indicates that the pixel is in a rough background. Topological analysis uses graph theory to determine whether a series of highlighted feature points are aligned in a straight line; if so, it may indicate a scratch; if randomly scattered, it may indicate noise.

[0053] Contextual attribute information refers to the descriptive labels, state parameters, or classification results assigned to each local region (which can be pixel-level, superpixel-level, or grid-level) in the wafer's test area after contextual analysis. It is a semantic summary of the local physical state. For example, contextual attribute information includes: surface roughness level, and / or, defect type; the surface roughness level is used to quantify the complexity of the background texture of the region, such as smooth, rough, extremely rough, etc.; the defect type refers to the preliminary identification of the possible defect categories that may exist in the region, such as scattering defects, linear defects, defect-free, etc.

[0054] For example, the contextual attribute information of wafer center region A is {roughness level: smooth, defect type: none}. The contextual attribute information of wafer edge region B is {roughness level: rough, defect type: scratch}.

[0055] Specifically, the multi-feature space data is divided into several local regions, such as based on sliding windows or superpixels. Algorithms such as cluster analysis, conditional random fields (CRF), or graph neural networks are used to mine the relationships between features. For example, the consistency of bright-field texture features within a local region is calculated. If the bright-field texture is highly consistent and periodic, while the dark-field exhibits high-frequency clutter, context analysis will determine that this belongs to the inherent texture of a rough surface. If a bright feature appears in the dark field, and this feature has a significant distance anomaly from the surrounding background features in the multi-feature space, context analysis will mark it as a potential defect. Based on the mined relationships, contextual attribute information is output for each local region.

[0056] In step S204, adaptive fusion weights for each local region are generated based on contextual attribute information.

[0057] The adaptive fusion weights include weight components corresponding to the bright field image and the dark field image, respectively.

[0058] Adaptive fusion weights refer to a set of numerical parameters dynamically calculated for each local region in the image based on its contextual attribute information. These parameters control the contribution or retention ratio of bright-field and dark-field data in the final synthesized image. For example, at coordinates (100, 100), with a smooth background as the contextual attribute information, a bright-field weight W is generated. BF =0.5, Dark Field W DF =0.5, meaning that equalization and fusion are performed in this local area. At coordinates (200, 200), the context attribute information is a high-roughness background, and a bright-field weight W is generated. BF ==0.9, Dark Field W DF =0.1 to suppress the dark field; at coordinates (300, 300), the context attribute information is a high scattering defect, generating a bright field weight W. BF ==0.2, Dark Field W DF =0.8, to enhance the dark field.

[0059] Based on contextual attribute information, adaptive fusion weights for each local region are generated. In some specific implementations, this can be a noise suppression mechanism. When the contextual attribute information indicates that the surface roughness level of a local region is greater than a preset threshold, it is inferred that the dark field image of that region contains a large amount of background scattering noise. To prevent this noise from contaminating the final result, the weight component W corresponding to that local region in the dark field image is reduced. DF At the same time, increase the weight component W of the bright field image. BF Because bright-field images are relatively insensitive to roughness, they can provide a smoother background reference.

[0060] It can also be a defect enhancement mechanism. When contextual attribute information indicates the presence of a defect type with high scattering characteristics in a local region, such as microparticles or micro-scratches, the bright spots in the dark-field image are inferred to be the signals corresponding to the defects. Therefore, the weight component W corresponding to this local region in the dark-field image is increased. DF The high sensitivity of dark fields is used to highlight defects.

[0061] The resulting adaptive fusion weights are biased towards bright fields in rough backgrounds and towards dark fields in areas with small defects.

[0062] Compared to using only bright field, which is prone to missing tiny scattering defects, using only dark field results in numerous false alarms due to rough surfaces. Alternatively, fixed-ratio fusion introduces half the noise into rough areas, leading to false alarms, while in defect areas, half the signal is lost, resulting in decreased contrast. This embodiment seeks the optimal signal-to-noise ratio combination at each pixel. In non-defect rough areas, it reduces W... DF This blocks the channel for dark field noise to enter the final image. In defect areas, by increasing W... DF This maximizes the throughput of the defect-corresponding signal. The processing adopts different fusion strategies for the background and defects, thereby resolving the contradiction between severe background noise interference and insufficient defect contrast.

[0063] In step S205, the bright field image and the dark field image are fused according to the adaptive fusion weights to generate a defect-enhanced image.

[0064] Fusion processing refers to the process of using adaptive fusion weights to perform weighted combination, nonlinear transformation, or reconstruction of bright-field and dark-field images (or their feature maps) to output a new defect-enhanced image. Fusion can be performed at the pixel level or at the feature level, i.e., fusing features first and then reconstructing the image. This disclosure does not limit this. For example, if a pixel has a bright-field grayscale of 100 and a dark-field grayscale of 50 (noise), and the adaptive fusion weights determine it as a rough background W... BF =0.9, W DF =0.1, the fusion result is 100×0.9+50×0.1=95, noise is suppressed; a certain pixel has a bright field grayscale of 100 and a dark field grayscale of 200 (defect). The adaptive fusion weight is determined as a rough background W. BF ==0.1, W DF =0.9, the fusion result is 100×0.1+200×0.9=190. Defects are highlighted.

[0065] The defect-enhanced image refers to the final image output after the above fusion steps. Compared to a single bright-field or dark-field image, the background texture, especially noise caused by roughness, is significantly smoothed or suppressed, while the contrast of the real defect area is significantly improved.

[0066] In this embodiment, for rough wafer surfaces, in the rough background region occupying most of the wafer image area, the adaptive fusion weights favor the suppression of stable bright or dark fields, making the background cleaner and more uniform. In defect regions, the adaptive fusion weights allow high-intensity dark-field scattering signals to pass through, resulting in stronger signals corresponding to defects compared to the original bright-field image. This makes the final defect-enhanced image not only visually clearer, but also easier to separate defects from the background, thereby improving the detection accuracy for rough-surface wafers and reducing the false alarm rate.

[0067] In some embodiments, such as Figure 3 As shown, in this defect detection method, the aforementioned multi-feature spatial data can be realized through the following steps S301 to S303.

[0068] In step S301, the first convolutional neural network is used to extract multi-scale features from the bright field image to obtain a bright field feature map set.

[0069] In step S302, the second convolutional neural network is used to extract multi-scale features from the dark field image to obtain a dark field feature map set.

[0070] In step S303, the bright field feature map set and the dark field feature map set are stitched together along the channel dimension to obtain multi-feature space data.

[0071] The first and second convolutional neural networks (CNNs) are two logically independent deep learning models that may have the same or different structures. The first CNN is used to process the bright-field branch of images, while the second CNN is used to process the dark-field branch. However, in the early stages of training or on computationally limited edge devices, the first few layers of the two networks can share weights to extract general low-level edge features, and only branch in deeper layers to extract modality-specific semantic features.

[0072] This embodiment can employ a two-stream twin or pseudo-twin network architecture, which includes two parallel backbone networks, serving as a bright-field feature extractor and a dark-field feature extractor, respectively.

[0073] The backbone network can be ResNet-50 (Residual Network-50) as the basic architecture for the first and second convolutional neural networks. Defect detection on rough surfaces requires extremely deep networks to extract high-level semantic features that can distinguish background texture from real defects. ResNet-50 effectively solves the gradient vanishing problem in deep networks through residual skip connections, ensuring the effectiveness of feature extraction. ResNet-50 is merely an illustrative example and is not intended to limit this disclosure; in practice, architectures such as ResNet-101, EfficientNet, or VGG can also be used.

[0074] The first and second convolutional neural networks have completely identical internal structures, but their weights are independent. Both contain the following five stages to generate feature maps of different scales.

[0075] The first stage, shallow feature extraction, consists of a 7×7 convolutional layer (Stride=2), a normalization layer, an activation function, and a 3×3 max-pooling layer. This is used for initial dimensionality reduction and noise reduction of the original pixels. The large 7×7 convolutional kernel can capture the basic edges of the wafer surface, the direction of the lattice texture, and the outlines of larger dust particles. This stage preserves high spatial resolution, which is crucial for locating minute defects (such as micron-level scratches), but semantic information is weak and easily affected by rough backgrounds.

[0076] The second stage involves residual learning and texture encoding, such as a bottleneck residual block consisting of three stacked blocks. Each bottleneck residual block contains a convolutional sequence used to extract low-level texture features. For rough wafers, the network in this stage begins to learn normal surface roughness patterns, such as the periodicity of grinding marks.

[0077] In the third stage, mid-layer geometric features, such as those comprising four bottleneck residual blocks, are downsampled using a 3×3 convolutional layer with a stride of 2 in the first block. This is used to extract mid-layer geometric features. From this stage onward, the network begins to identify anomalies with specific shapes, such as circular pits or linear scratches. At this point, the first convolutional neural network focuses on the geometric shape of reflectivity variations, while the second convolutional neural network focuses on the distribution shape of scattered light spots.

[0078] In the fourth stage, high-level semantic features, such as those containing six bottleneck residual blocks, are downsampled. This is used to extract high-level semantic features. The receptive field is significantly expanded in this stage, enabling it to cover a larger surface area. The network can then distinguish between large areas of rough background and localized anomalous regions.

[0079] The fifth stage involves abstracting topological features, such as those containing three bottleneck residual blocks, and performing downsampling. This is used to extract the highest-level abstract features, possessing the strongest noise resistance and the ability to identify complex defect logic, such as discontinuous scratch sequences.

[0080] After the above five stages, the feature pyramid formed by the outputs of each stage of the first convolutional neural network is the bright-field feature map set, and the feature pyramid formed by the outputs of each stage of the second convolutional neural network is the dark-field feature map set. The bright-field or dark-field feature map sets retain the shallow and deep features of the corresponding bright-field or dark-field images. They can accurately locate tiny particles using shallow features and identify large-area macroscopic defects using deep features. Simultaneously, they utilize deep semantic information to suppress false positives from high-frequency noise on rough surfaces, achieving comprehensive coverage of defects across all scales.

[0081] Channel stitching is used to stitch together the bright-field feature map set and the dark-field feature map set to obtain multi-feature space data. This embodiment uses channel stitching instead of element-by-element addition because element-by-element addition is a lossy compression operation, and in wafer inspection, logical judgments are often non-linear. For example: Case A: Metal particles: DF is extremely bright (high scattering), BF is dark (blocking light). Case B: Watermarks: DF is dark (no scattering), BF has abnormal grayscale (thin-film interference). If addition is used, Case A and Case B may yield similar summed values, leading to classification confusion. Channel stitching, however, preserves the complete original information distribution, supporting subsequent complex non-linear logical judgments.

[0082] In this disclosure, the background frequency (BF) reflects reflectivity, and the scattering frequency (DF) reflects scattering frequency. For rough surfaces, background noise manifests as high-frequency speckle in the DF and as low-frequency light intensity unevenness in the BF. If early pixel fusion is used, simple linear superposition may lead to a deterioration of the signal-to-noise ratio due to the difference in noise distribution between the BF and DF. For example, a bright background in the BF may obscure weak defect bright spots in the DF, or strong texture scattering noise in the DF may interfere with the morphological features in the BF.

[0083] In this embodiment, the first convolutional neural network specifically learns convolutional kernels for reflective features, such as learning to identify large areas of light intensity attenuation as morphological defects; the second convolutional neural network specifically learns convolutional kernels for scattering features, such as learning to identify isolated bright pixels as particles. These specialized sub-networks in deep learning can learn more discriminative feature representations. This achieves decoupling and purification of modal features, avoids mutual interference of heterogeneous optical information, and maximizes the preservation of the sensitivity of bright and dark fields to different types of defects.

[0084] In some embodiments, such as Figure 4 As shown, in this defect detection method, the above-mentioned contextual analysis of multi-feature spatial data can be achieved through the following steps S401 and S402.

[0085] In step S401, a graph neural network is used to model the topological relationships of feature points in multi-feature spatial data and generate node embedding vectors.

[0086] Topological modeling refers to the process of establishing nodes and edges in non-Euclidean space based on the inherent relationships between data points. This process transforms rasterized image data into graph-structured data.

[0087] Traditional convolutional neural networks assume that data is distributed on a regular grid, and the convolutional kernel can only aggregate fixed 3×3 or 5×5 neighborhoods. Such physical neighborhoods often cut off targets with the same semantics but physical discontinuity, such as discontinuous scratches.

[0088] Topological relationship modeling allows direct edges to be established between pixels with similar features, regardless of their physical distance on the wafer. In this way, discontinuous defect points that were originally scattered in a rough background can be connected into a connected subgraph through feature similarity. In contrast, although there are many randomly distributed background noise points, they often lack a consistent connection pattern.

[0089] For example, a tiny scratch on a wafer surface, due to roughness interference, appears as a series of discontinuous bright spots A, B, C, and D in a dark-field image. In the physical mesh, A and B may be separated by several noisy pixels, causing ordinary algorithms to consider them as independent noise points. By modeling topological relationships, since A, B, C, and D have similar linear direction characteristics and scattering intensities, the algorithm establishes strong connecting edges between AB and BC. Ultimately, these four points form a chain on the topological graph, thus being identified as a unified structure.

[0090] After passing through message passing and aggregation operations in a graph neural network, the node embedding vectors become new, high-level feature representations for each graph node. The node embedding vectors highly condense the node's local features and their contextual information within the overall graph topology.

[0091] The input to a graph neural network is the original feature vector, and the output is the embedding vector. For example, if the original feature vector represents a bright spot, the corresponding embedding vector represents the same bright spot, and the bright spot is surrounded by similar linearly distributed bright spots, it is considered a scratch; or, if it is a bright spot, but the surrounding neighboring points are disordered, it is considered random noise.

[0092] The specific process of topological relationship modeling is as follows: First, the multi-feature spatial data is divided into N local regions, such as by using a regular grid or a simple linear iterative clustering algorithm; for each local region i, the statistical aggregation value of all pixel feature vectors within it is calculated, such as mean or maximum pooling, to obtain the initial feature vector of local region i, i.e., a node. Each node vi includes the initial feature vector and the physical center coordinates.

[0093] For any node vi, its potential neighbors are restricted to local windows where the physical distance R is within the specified range. This is to avoid establishing irrelevant connections between points at the top left and bottom right corners of the wafer. Although their features may be similar, in semiconductor manufacturing, correlations across extremely large distances are generally meaningless. Within the determined candidate set of potential neighbors, the distance between nodes vi and vj in the feature space is calculated, using methods such as Euclidean or cosine distance. The K nodes with the smallest distance (or highest similarity) are selected as the formal neighbors of vi, and directed edges are established.

[0094] In a flat background region, node vi connects to nodes in all directions around it, forming an isotropic connection pattern. In a linear defect region (such as a scratch), the characteristics of node vi are similar to those of nodes along the scratch direction, but differ greatly from those of background nodes perpendicular to the scratch direction. Therefore, it establishes connections along the scratch direction, forming an anisotropic linear topology.

[0095] After topological relationship modeling is completed, node embedding vectors are generated. The core operation of a graph neural network is layer-by-layer feature aggregation. Assuming the network depth is L layers, and the current layer is the l-th layer (l=0, 1, 2… L-1), the initial feature vectors of the nodes in the current layer are first linearly mapped using a learnable weight matrix to increase the feature dimensionality and enhance expressive power. The attention coefficient e between node vi and its neighbor vj is then calculated. ij The attention coefficient represents the importance of node vj to vi. Finally, the attention coefficient is normalized using the Softmax function to obtain α. ij If both vi and vj are defect points, then the feature similarity is high. ij Approaching 1, information flows smoothly; if vi is the defect and vj is the background, the difference is large, ɑ ij Approaching 0, background noise is blocked. Based on the attention coefficient, the neighbor features are weighted and summed to obtain aggregated neighborhood information. This aggregated neighborhood information is then fused with the node's own information, such as through residual connections or concatenation, and activated using a non-linear activation function, such as ReLU, to obtain the features for the next layer. After L layers of iteration, each node aggregates topological information within its L-hop range, and the final output feature is the node embedding vector.

[0096] In step S402, cluster analysis is performed on the node embedding vectors to separate the defect feature cluster from the rough background feature cluster.

[0097] Clustering analysis is an unsupervised learning method used to divide all nodes in a graph into several disjoint subsets, or clusters, based on the similarity between their embedding vectors. Because the wafer background is statistically homogeneous, most nodes representing the background will cluster very tightly in the embedding space, forming a high-density core cluster. Defects are typically heterogeneous and sparse, appearing as outliers or several small, high-density clusters in the embedding space. Through clustering, it is not necessary to know the appearance of defects beforehand (supervised learning); it is only necessary to know that defects are different from the background to separate them.

[0098] The specific process of cluster analysis is as follows: First, set the neighborhood radius and the minimum number of points. These two parameters are determined based on the statistical distribution of the embedding space. For example, calculate the average distance between all pairs of nodes, and set the neighborhood radius as the product of the average distance and a coefficient. Traverse all node embedding vectors. If a node's set neighborhood radius contains more than the minimum number of points, it is called a core node. Starting from the core nodes, recursively find all density-reachable nodes to form a cluster.

[0099] After clustering, several clusters and a set of unclassified noise points are obtained. In the wafer inspection scenario, based on the prior knowledge that the background accounts for the vast majority, the following decision logic can be formulated: the main cluster with the largest number of nodes and the highest distribution density is identified as the coarse background feature cluster, and all local regions belonging to this cluster are labeled as background in their context attributes. Other small clusters besides the background clusters and outlier noise points are identified as defect feature clusters. Local regions belonging to these clusters are labeled as potential defects in their context attributes.

[0100] In this embodiment, multi-feature spatial data is mapped to a topological graph structure. Due to the significant difference between defect features and background features, defect nodes are mainly connected to similar defect nodes, or have extremely low connection weights with background nodes. For structural defects (scratches), clustering of similar nodes strengthens the signal features, making them further away from the background center in the embedding space. For isolated defects, due to the lack of smoothing from effective neighbors, their outlier characteristics are fully preserved or even amplified through nonlinear transformations, increasing the distance from the background clusters. The message passing mechanism of the graph neural network effectively smooths the homogeneous background noise while enhancing the structural features of heterogeneous defects. Subsequent clustering analysis achieves high-confidence signal separation in the embedding space.

[0101] In some embodiments, such as Figure 5 As shown, in this defect detection method, the above-mentioned generation of adaptive fusion weights for each local region based on contextual attribute information can be achieved through the following steps S501 and S503.

[0102] In step S501, an attention mechanism network is used to perform self-attention calculation on the feature data corresponding to the bright field image and the dark field image respectively, to obtain enhanced feature data containing modal internal context dependencies.

[0103] In step S502, an attention mechanism network is used to perform cross-attention calculation between contextual attribute information and enhanced feature data to obtain a pixel-level weight map.

[0104] Modal context dependency refers to the correlation between pixels at different spatial locations within a single imaging modality (bright field only or dark field only). For example, on a rough surface, background textures often exhibit repetitive statistical patterns, while defects disrupt these patterns.

[0105] Attention mechanism networks can simulate the human visual attention mechanism, automatically focusing on important features and suppressing irrelevant features in neural network modules.

[0106] The attention mechanism network comprises three functional modules: an intramodal self-attention module, which processes bright-field and dark-field features respectively, capturing their respective global dependencies; a context-guided cross-attention module, which injects contextual attribute information into the feature map for feature recalibration; and a weight generation decoder, which maps the enhanced features into a single-channel pixel-level weight map.

[0107] With dark field feature atlas F DF For example, the self-attention mechanism is used to calculate F. DF The correlation between each location and all other locations is calculated. First, a 1×1 convolution is used to map the feature map into three spaces: query (Q), key (K), and value (V). Q and K are then multiplied by a matrix to calculate the similarity score Sij between pixel i and pixel j. For rough backgrounds, Sij is usually higher due to texture similarity, while defective pixels have lower similarity to background pixels due to their unique shape. The similarity scores are normalized to obtain an attention map. Finally, V is weighted and summed using the attention map A to obtain enhanced feature data containing modality-internal contextual dependencies.

[0108] Cross-attention computation refers to the process of using contextual attribute information to guide or adjust the processing of enhanced feature data. Specifically, the contextual attribute information is mapped through a multilayer perceptron to generate channel attention vectors and spatial attention vectors. These two vectors are then used to weight the enhanced feature data of the dark field and the enhanced feature data of the bright field after self-attention enhancement. A series of convolutional layers and upsampling layers are then used to compress the feature map into a single channel. Finally, the output layer uses the sigmoid function to constrain the values ​​to the (0, 1) interval, resulting in the final adaptive fusion weight map.

[0109] In this embodiment, through self-attention computation, background pixels aggregate information from all other background pixels in the entire image, causing their features to tend towards smoothing and averaging. Defect pixels, due to their low similarity to the background, primarily aggregate their own information or information about similar defects, thus preserving and highlighting their features. Furthermore, since different types of defects and backgrounds exhibit strong prior patterns in bright and dark fields—for example, high roughness results in high noise in dark fields—the cross-attention mechanism parameterizes these prior patterns and dynamically selects the most reliable feature channels. For instance, in rough regions, the reliability of dark field channels is forcibly reduced by mathematical multiplication, thereby achieving physically prior-driven feature selection. This avoids the introduction of noise caused by blind fusion and significantly reduces the false alarm rate.

[0110] In some embodiments, such as Figure 6 As shown, in this defect detection method, the above-mentioned fusion processing of bright field image and dark field image can be achieved through the following steps S601 and S602.

[0111] In step S601, the bright field image, the dark field image, and the adaptive fusion weights are input into the generator of the generative adversarial network.

[0112] In step S602, a defect enhancement image is generated using a generator.

[0113] The generator is trained to suppress background noise textures corresponding to rough surfaces.

[0114] Generative Adversarial Networks (GANs) are deep learning architectures that combine a generator and a discriminator. They are trained through a game-like process to generate high-quality images for image fusion. The generator receives bright-field images, dark-field images, and an adaptive weight map, and outputs a defect-enhanced image. Its goal is to create a perfectly flawed image to deceive the discriminator. The discriminator receives the defect-enhanced image and determines whether it is a genuine, clear image or a forged image generated by the generator.

[0115] The generator is an encoder-decoder structure with skip connections. The input to the generator is a bright-field image I. BF Dark field image I DF And adaptive fusion weight map. The encoder progressively extracts multi-scale features, reducing spatial resolution. The bottleneck layer is used for the transformation and fusion of deep features. The decoder progressively restores the image size. Features from the encoder layer are directly skipped and concatenated to the decoder layer at the same level. For tiny defects (such as scratches a few pixels wide), deep networks are prone to losing their spatial location information. Skip connections allow high-frequency edge information to flow directly to the output, ensuring that defect edges in the fused image are clear and sharp.

[0116] Instead of outputting a single true / false score for the entire image, the discriminator outputs an N×N matrix, where each element represents the true / false status of a local patch of a specific size in the original image. The discriminator receives the input image, a real, noise-free reference image, and the generated defect-enhanced image. After convolution, it outputs a feature map, where each feature point corresponds to the receptive field of a pixel the size of a local region in the original image. The discriminator then performs a true / false judgment for each local region separately. Because the discriminator focuses on local realism, it is extremely sensitive to high-frequency texture details in the image. If the image generated by the generator retains coarse scattering noise in the background area—noise that does not conform to a smooth background distribution—the discriminator will classify it as fake, thus forcing the generator to not only fuse brightness but also reconstruct texture.

[0117] In some embodiments, such as Figure 7 As shown, the defect detection method includes the following steps S701 to S706.

[0118] In step S701, bright-field and dark-field images of the area to be tested on the wafer are acquired.

[0119] In step S702, features are extracted from the bright field image and the dark field image respectively to construct multi-feature spatial data.

[0120] In step S703, context analysis is performed on the multi-feature spatial data to determine the context attribute information corresponding to each local region in the wafer test area.

[0121] In step S704, adaptive fusion weights for each local region are generated based on contextual attribute information.

[0122] In step S705, the bright field image and the dark field image are fused according to the adaptive fusion weights to generate a defect enhancement image.

[0123] In step S706, the defect enhancement image is input into the instance segmentation network to output the bounding box, category label, and pixel-level mask of the defect in the wafer test area.

[0124] Instance segmentation networks are deep learning-based computer vision models that integrate object detection and semantic segmentation. This particular instance segmentation network is configured not only to identify the category and location of objects in an image, but also to distinguish different individual instances of the same category and generate accurate pixel-level masks for each instance.

[0125] In traditional machine vision or simple deep learning applications, semantic segmentation is commonly used. Semantic segmentation classifies each pixel in an image as either background or a defect, but it cannot distinguish between two adjacent defects. For example, if two tiny particles are stuck together, semantic segmentation will label them as a red, grainy area, and engineers will not know the exact particle count.

[0126] The instance segmentation network introduces the concept of instances. Even if two particles are physically in contact or overlapping, the network can identify them as particle 1 and particle 2 through feature space analysis and delineate their outlines. This enables the system to output an accurate defect density distribution map. The specific instance segmentation network chosen in this disclosure is not limited; for example, Mask R-CNN could be used.

[0127] For example, suppose there is a dense set of scratch defects in the area to be tested on the wafer. The semantic segmentation output is a binary image where the scratched areas are white and the background is black. All scratches are connected, making it impossible to count the number of scratches. The instance segmentation network outputs: Instance 1: Scratch, confidence 0.98, length 15μm, curved shape. Instance 2: Scratch, confidence 0.95, length 5μm, straight shape. Instance 3: Scratch, confidence 0.92, overlaps with Instance 1, but is identified as a separate object.

[0128] A bounding box is the smallest axis-aligned rectangle in an image coordinate system that completely contains all pixels of a target defect. In data structures, it is represented by a set of four values, commonly in the format (xmin, ymin, xmax, ymax) or (xcenter, ycenter, width, height). The bounding box provides rough location information for the defect and serves as the basic index for all subsequent processing. In a wafer inspection machine, the microscope moves the stage based on the center coordinates of the bounding box to center the defect in the field of view. The aspect ratio of the bounding box is also a preliminary basis for determining whether the defect is linear or circular.

[0129] For example, in a partial wafer image with a resolution of 2048×2048, a pit defect located in the upper right corner is detected. The bounding box data output by the network may be (1800, 1850, 200, 250), indicating that the defect is located within a rectangular area with x-coordinates from 1800 to 1850 and y-coordinates from 200 to 250, with a width of 50 pixels and a height of 50 pixels.

[0130] Category labels are identifiers that indicate the physical properties of defects, calculated by the instance segmentation network through classification branches based on extracted features. They typically correspond to index values ​​in a predefined defect classification system. In semiconductor manufacturing, different defect types correspond to different process root causes. For example, particles may originate from insufficient cavity cleanliness, scratches may stem from problems with the polishing pads used in chemical mechanical polishing, and collapses may originate from lattice defects. Category labels are crucial for converting image signals into process diagnostic information. The network outputs the probability of each category through a Softmax function, and the label is selected based on the probability of the category with the highest probability.

[0131] For example, the preset category mapping table is as follows: ID 1, particle; ID 2, scratch; ID 3, dent or crystal defect; ID 4, residue; ID 5, graphic error. If the probability vector output for a certain candidate region is [0.01, 0.02, 0.95, 0.01, 0.01], then the category label of the defect is determined to be ID 3, dent or crystal defect.

[0132] A pixel-level mask refers to a binary matrix or bitmap that corresponds to the size of the bounding box of the target defect (or is the same size as the original image). In this matrix, pixels belonging to the defect body are marked with a specific value (such as 1), while pixels belonging to the background or not belonging to this instance are marked with another value (such as 0).

[0133] Because the bounding box contains a large amount of background information that is not related to the defect, the mask precisely extracts the shape of the defect. It preserves the topological structure of the defect, such as whether it has holes, jagged edges, or consists of multiple discontinuous parts.

[0134] For example, for an L-shaped metal residue defect, the bounding box is a rectangle that includes not only the L itself but also the blank background area in the middle of the L-shaped bend. The pixel-level mask is a matrix at the original image resolution, where only the pixels forming the L have a value of 1, and the rest are 0. By counting the number of 1s in the mask, the coverage area of ​​the residue can be accurately calculated.

[0135] This embodiment of the disclosure achieves accurate separation and independent counting of dense defects through an instance segmentation network, improving the accuracy of defect density data. Simultaneously, it outputs the bounding boxes, category labels, and pixel-level masks of defects in the wafer's test area, which not only indicate the defect type but also characterize the defect's shape information and precise area.

[0136] like Figure 8 The diagram shown is an architecture diagram of the defect detection system based on bright and dark field optical co-imaging provided in this disclosure, including: a sub-pixel spatial registration module 80, a multi-feature space construction module 81, a context analysis module 82, an attention mechanism network 83, a generative adversarial network 84, and an instance segmentation network 85.

[0137] The sub-pixel spatial registration module 80 takes the initial bright-field image and the initial dark-field image as input data, and outputs the bright-field image I. BF and dark field image I DF .

[0138] The multi-feature space construction module 81 includes: a first convolutional neural network 811, a second convolutional neural network 812, and a channel dimension splicing layer 813. BF Input to the first convolutional neural network 811, output bright-field feature map set F BF I DF Input the second convolutional neural network 811, output the bright field feature map set F DF F BF and F DF Input channel dimension concatenation layer 813, output multi-feature space data M Space .

[0139] Context analysis module 82 includes: graph neural network 821 and clustering analysis layer 822. Space Input the graph neural network 821, output the node embedding vector P, input the clustering analysis layer 822, output the context attribute information Info.

[0140] The attention mechanism network 83 includes: a self-attention layer 831 and a cross-attention layer 832, F BF and F DF Input from attention layer 831, output enhanced feature data D info Info and D info Input a cross-attention layer of 832, output a pixel-level weight map W. map .

[0141] W map I BF and I DF Input: Generative Adversarial Network 84; Output: Defect Enhancement Image I Enhanced .

[0142] I Enhanced Input an instance segmentation network 85 and output bounding boxes, category labels, and pixel-level masks.

[0143] Figure 9 This is a structural block diagram of a defect detection device disclosed herein, such as... Figure 9 As shown, it includes: acquisition part 901, feature extraction and construction part 902, determination part 903, and generation part 904; The acquisition section 901 is configured to acquire bright-field and dark-field images of the wafer test area; the feature extraction and construction section 902 is configured to extract features from the bright-field and dark-field images respectively to construct multi-feature space data; the determination section 903 is configured to perform context analysis on the multi-feature space data to determine the context attribute information corresponding to each local region in the wafer test area; the generation section 904 is configured to generate adaptive fusion weights for each local region based on the context attribute information, wherein the adaptive fusion weights include weight components corresponding to the bright-field and dark-field images respectively; and, based on the adaptive fusion weights, perform fusion processing on the bright-field and dark-field images to generate a defect enhancement image.

[0144] In some embodiments, the contextual attribute information includes at least one of the following: the surface roughness level of the local region, and the defect type of the local region.

[0145] In some embodiments, the generation part 904 is configured to reduce the weight component corresponding to the dark field image in the adaptive fusion weight when the context attribute information indicates that the surface roughness level of the local region is greater than a preset threshold; and to increase the weight component corresponding to the dark field image in the adaptive fusion weight when the context attribute information indicates that there is a defect type with high scattering characteristics in the local region.

[0146] In some embodiments, the feature extraction and construction section 902 is configured to perform multi-scale feature extraction on the bright field image using a first convolutional neural network to obtain a bright field feature map set; perform multi-scale feature extraction on the dark field image using a second convolutional neural network to obtain a dark field feature map set; and concatenate the bright field feature map set and the dark field feature map set in the channel dimension to obtain multi-feature space data.

[0147] In some embodiments, the determining portion 903 is configured to use a graph neural network to model the topological relationships of feature points in multi-feature spatial data to generate node embedding vectors; and to perform cluster analysis on the node embedding vectors to separate defect feature clusters from rough background feature clusters.

[0148] In some embodiments, the generation part 904 is configured to perform self-attention computation on the feature data corresponding to the bright field image and the dark field image respectively using an attention mechanism network to obtain enhanced feature data containing modality internal context dependencies; and to perform cross-attention computation between context attribute information and enhanced feature data using an attention mechanism network to obtain a pixel-level weight map.

[0149] In some embodiments, the generation section 904 is configured to input a bright-field image, a dark-field image, and adaptive fusion weights into a generator of a generative adversarial network; and to generate a defect-enhanced image using the generator, wherein the generator is trained to suppress background noise textures corresponding to rough surfaces.

[0150] In some embodiments, the acquisition section 901 is configured to simultaneously acquire an initial bright-field image and an initial dark-field image; and to perform sub-pixel-level spatial registration on the initial bright-field image and the initial dark-field image to obtain a bright-field image and a dark-field image aligned in the pixel coordinate system.

[0151] In some embodiments, the defect detection apparatus further includes an input / output section configured to input a defect enhancement image into an instance segmentation network to output a bounding box, category label, and pixel-level mask of a defect in the wafer region to be tested.

[0152] In this embodiment, each part can implement the defect detection method provided in the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0153] This disclosure also provides a computer-readable storage medium storing at least one instruction that is executed by a processor to implement the defect detection method described in the above embodiments.

[0154] This disclosure also provides a computer program product including computer instructions stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the defect detection method described in the above embodiments.

[0155] This disclosure also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described defect detection method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0156] It should be understood that the chip mentioned in the embodiments of this disclosure may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0157] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, servers, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0161] Those skilled in the art will recognize that the functions described in this disclosure in one or more of the examples above can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0162] It should be noted that the technical solutions described in this disclosure can be combined arbitrarily as long as they do not conflict.

[0163] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A defect detection method, characterized in that, The defect detection method includes: Acquire bright-field and dark-field images of the test area on the wafer; Feature extraction is performed on the bright field image and the dark field image respectively to construct multi-feature space data; Context analysis is performed on the multi-feature spatial data to determine the context attribute information corresponding to each local region in the wafer test area; Based on the context attribute information, adaptive fusion weights for each local region are generated, wherein the adaptive fusion weights include weight components corresponding to the bright field image and the dark field image, respectively. Based on the adaptive fusion weights, the bright field image and the dark field image are fused to generate a defect-enhanced image.

2. The defect detection method according to claim 1, characterized in that, The contextual attribute information includes at least one of the following: the surface roughness level of the local region, and the defect type of the local region.

3. The defect detection method according to claim 2, characterized in that, The step of generating adaptive fusion weights for each local region based on the context attribute information includes: When the context attribute information indicates that the surface roughness level of the local region is greater than a preset threshold, the weight component corresponding to the dark field image in the adaptive fusion weight is reduced; When the contextual attribute information indicates that there is a defect type with high scattering characteristics in the local region, the weight component corresponding to the dark field image in the adaptive fusion weight is increased.

4. The defect detection method according to claim 1, characterized in that, The step of extracting features from the bright-field image and the dark-field image respectively to construct multi-feature space data includes: The first convolutional neural network is used to extract multi-scale features from the bright field image to obtain a bright field feature map set; The dark field image is subjected to multi-scale feature extraction using a second convolutional neural network to obtain a dark field feature map set; The bright-field feature map set and the dark-field feature map set are stitched together along the channel dimension to obtain the multi-feature space data.

5. The defect detection method according to claim 1, characterized in that, The contextual analysis of the multi-feature spatial data includes: A graph neural network is used to model the topological relationships of feature points in the multi-feature spatial data to generate node embedding vectors. Cluster analysis is performed on the node embedding vectors to separate the defect feature cluster from the rough background feature cluster.

6. The defect detection method according to claim 1, characterized in that, The step of generating adaptive fusion weights for each local region based on the context attribute information includes: Self-attention computation is performed on the feature data corresponding to the bright field image and the dark field image respectively using an attention mechanism network to obtain enhanced feature data containing modal internal context dependencies; The attention mechanism network is used to perform cross-attention calculation between the contextual attribute information and the enhanced feature data to obtain a pixel-level weight map.

7. The defect detection method according to claim 1, characterized in that, The fusion process of the bright field image and the dark field image includes: The bright field image, the dark field image, and the adaptive fusion weights are input into the generator of the generative adversarial network; The defect enhancement image is generated using the generator, wherein the generator is trained to suppress background noise textures corresponding to rough surfaces.

8. The defect detection method according to claim 1, characterized in that, The acquisition of bright-field and dark-field images of the area to be tested on the wafer includes: Simultaneously acquire initial bright-field and initial dark-field images; Subpixel-level spatial registration is performed on the initial bright-field image and the initial dark-field image to obtain the bright-field image and the dark-field image aligned in the pixel coordinate system.

9. The defect detection method according to claim 1, characterized in that, The defect detection method further includes: The defect enhancement image is input into an instance segmentation network to output the bounding box, category label, and pixel-level mask of the defect in the wafer test area.

10. A defect detection device, characterized in that, The defect detection device includes: an acquisition part, a feature extraction and construction part, a determination part, and a generation part; The acquisition section is configured to acquire bright-field and dark-field images of the area to be tested on the wafer; The feature extraction and construction section is configured to extract features from the bright field image and the dark field image respectively, and construct multi-feature space data. The determining part is configured to perform context analysis on the multi-feature spatial data to determine the context attribute information corresponding to each local region in the wafer test area; The generation part is configured to generate adaptive fusion weights for each local region based on the context attribute information, wherein the adaptive fusion weights include weight components corresponding to the bright field image and the dark field image, respectively; and, Based on the adaptive fusion weights, the bright field image and the dark field image are fused to generate a defect-enhanced image.

11. An electronic device, characterized in that, The electronic device includes: The optical acquisition unit is configured to acquire bright-field and dark-field images of the area to be tested on the wafer; Memory, which stores computer programs; A processor, connected to the optical acquisition unit and the memory, is configured to execute a computer program stored in the memory to implement the defect detection method as described in any one of claims 1 to 9.

Images