Chip defect detection method based on target detection

Abstract

According to the chip defect detection method based on target detection, features are enhanced through spatial pyramid pooling, details of the features are supplemented through a PAN path aggregation network, semantic information is added, and the information of the features is enriched. According to the technical scheme, the precision, the efficiency and the robustness are remarkably improved. The invention aims to solve the four core problems of chip defect detection in the prior art: a sub-pixel level defect missing detection problem caused by a fixed anchor frame; the dichotomy design cannot meet the requirements of fine-grained defect type identification; the problems of calculation redundancy and insufficient real-time performance caused by a two-stage detection architecture are solved; and the problem of misjudgment of a fixed NMS threshold value in a high-density defect scene is solved.

Description

Technical Field

[0001] This invention relates to the field of chip defect detection technology, and in particular to a chip defect detection method based on target detection. Background Technology

[0002] Chip defect detection is a core step in the entire semiconductor manufacturing process, directly determining wafer yield and manufacturing costs. As process nodes advance to 7nm and below, the size of chip surface defects has shrunk to the sub-micron level (e.g., the diameter of residual metal defects in 5nm processes can be as low as 0.8μm). Traditional detection technologies face the dual challenges of accuracy and efficiency. Therefore, developing high-precision, real-time defect detection technologies has become a key requirement for intelligent semiconductor manufacturing.

[0003] The current mainstream approach is based on an improved Faster R-CNN architecture, and its technical process is as follows:

[0004] 1. Image Acquisition: Acquire images of the chip surface using a 20-megapixel industrial camera (1920×1080 resolution) at a frame rate of approximately 15 FPS;

[0005] 2. Feature extraction: ResNet-34 is used as the backbone network to generate 4 levels of feature maps (resolution from 120×120 to 7.5×7.5);

[0006] 3. Candidate Region Generation: The Region Proposal Network (RPN) generates 9 fixed-size anchor frames (32×32, 64×64, 128×128, etc., with aspect ratios of 1:1, 1:2, and 2:1);

[0007] 4. Feature alignment and classification: The candidate region features are unified to a 7×7 size through ROIPooling, and the binary classification of "defect / non-defect" and bounding box regression are completed through two levels of fully connected layers;

[0008] 5. Post-processing: Non-maximum suppression (NMS) with a fixed threshold (0.7) is used to remove overlapping detection boxes, and the final result is output.

[0009] Four core defects of existing technology

[0010] 1. Poor compatibility of fixed anchor frames leads to serious missed detection of subpixel defects.

[0011] The minimum anchor frame of the traditional solution is 32×32 pixels, while nanoscale scratches in the 7nm process (such as 1.2μm long and 0.3μm wide) occupy only 5×1 pixels in the image, much smaller than the anchor frame size. Actual tests show that this solution has a recall rate of only 42.3% for defects <10×10 pixels, causing such defects to be discovered only in the packaging stage, resulting in huge losses.

[0012] 2. Limited classification capabilities, unable to meet the needs of fine-grained detection.

[0013] The existing binary classification design can only determine "whether it is a defect" and cannot distinguish between eight typical defects such as short circuits, metal residues, and pinholes. If further subdivision of the type is required, three additional fully connected layers need to be connected in series, which increases the number of model parameters by 60% and the inference latency from 83ms to 135ms, making it unsuitable for high-speed production lines (which require latency <50ms).

[0014] 3. Two-phase architecture is inefficient and lacks real-time performance.

[0015] The two-stage process of Faster R-CNN, "candidate box generation-classification regression", has computational redundancy: at VGA resolution (640×480), the processing speed of a single image is only 8.3 FPS, while a high-speed detection production line for 12-inch wafers requires a processing capacity of at least 30 FPS. The existing solution causes the production line to slow down by 37%.

[0016] 4. Fixed NMS threshold results in poor robustness in dense defect scenarios.

[0017] When defect density > 5 defects / mm 2 In cases such as multi-bit defects in memory chips, a fixed NMS threshold (typically 0.7) can lead to the false deletion of overlapping detection boxes. Tests conducted by a DRAM chip manufacturer showed that in such scenarios, the recall rate plummeted from 89% to 52%, the false positive rate increased by 41%, and the cost of manual re-inspection increased by 30%. Summary of the Invention

[0018] This invention provides a chip defect detection method based on target detection. Spatial pyramid pooling enhances features, and a PAN path aggregation network supplements feature details and adds semantic information, enriching the feature information. The technical solution of this invention achieves significant improvements in accuracy, efficiency, and robustness. The specific technical solution is as follows:

[0019] In a first aspect, the present invention provides a chip defect detection method based on target detection, the method comprising:

[0020] The surface image of the chip product is captured by an industrial camera, and the acquired image is preprocessed to obtain a standardized image;

[0021] Standardized images are input into the backbone network for image feature extraction to obtain chip defect feature images;

[0022] The chip defect feature image is input into the spatial pyramid pooling module for feature enhancement processing to obtain the enhanced feature image;

[0023] The enhanced feature image is input into the PAN path aggregation network. The PAN path aggregation network performs detail repair on the enhanced feature image and associates it with corresponding semantic information, so that the enhanced feature image contains both detailed information and semantic information, resulting in high-quality features.

[0024] Dynamic anchor boxes are generated for high-quality features. Chip defects in the high-quality features are matched with corresponding anchor boxes of the corresponding level by the feature map resolution. The chip defect features with selected anchor boxes are obtained, and chip defects can be detected.

[0025] Furthermore, after generating dynamic anchor boxes for high-quality features, the process also includes:

[0026] The decoupled detection head enhances defect features through the CBAM attention module, and then outputs the probability of 8 types of defects through the classification branch and the bounding box parameters through the regression branch. This solves the optimization conflict problem caused by traditional shared parameters, realizes fine-grained defect classification and accurate localization, and outputs the preliminary classification and localization results of each potential defect.

[0027] Furthermore, after outputting the preliminary results of the classification and location of each potential defect, the following is also included:

[0028] The dynamic NMS optimizer first counts the number of defects and calculates the density using a 3×3 window, then dynamically adjusts the NMS threshold based on the density, and finally removes overlapping detection boxes through multi-level NMS to improve the recall rate in dense defect scenes and output the final detection results.

[0029] Furthermore, after outputting the final detection result, the following is also included:

[0030] During the hardware deployment phase, TensorRT 8.6 was used to quantize the model to INT8 precision, improving inference speed by 2.3 times. The model was deployed on NVIDIA Jetson AGX Orin and power consumption was optimized to 18.8W. Batch size was set to 8 to achieve batch inference, increasing wafer inspection throughput from 12 wafers / hour to 30 wafers / hour, meeting the edge computing needs of high-speed production lines.

[0031] Furthermore, the step of acquiring surface images of the chip product using an industrial camera and preprocessing the acquired images to obtain standardized images includes:

[0032] The surface image of the chip product is acquired by using an industrial camera;

[0033] The acquired image is divided into 8x8 pixel blocks, and the cumulative distribution function of the gray-level histogram of each block is determined.

[0034] By cropping the extreme gray values ​​at both ends of the cumulative distribution function by 1% and stretching them to the 0-255 range, the image contrast is improved by 40%.

[0035] Then, the gamma value is dynamically adjusted based on the average gray value of the acquired images to solve the problem of uneven illumination;

[0036] Finally, bilinear interpolation was used to uniformly scale the acquired images to 1080×1080 pixels, and a standardized image was output.

[0037] Secondly, the present invention also provides a chip defect detection system based on target detection, the system comprising: a preprocessing unit, a feature extraction unit, a feature enhancement unit, a semantic supplementation unit, and an anchor box selection unit;

[0038] The preprocessing unit is used to acquire surface images of chip products using an industrial camera, and to preprocess the acquired images to obtain standardized images.

[0039] The feature extraction unit is used to extract image features from the standardized image input backbone network to obtain chip defect feature images.

[0040] The feature enhancement unit is used to perform feature enhancement processing on the input spatial pyramid pooling module of the chip defect feature image to obtain the enhanced feature image.

[0041] The supplementary semantic unit is used to input the enhanced feature image into the PAN path aggregation network. The PAN path aggregation network performs detail repair on the enhanced feature image and associates it with corresponding semantic information, so that the enhanced feature image contains both detail information and semantic information, resulting in high-quality features.

[0042] The anchor frame selection unit is used to dynamically generate anchor frames for high-quality features, determine the corresponding level of the matching anchor frame for the chip defects in the high-quality features through the feature map resolution, obtain the chip defect features with selected anchor frames, and realize the detection of chip defects.

[0043] In another embodiment of the present invention, the system further includes: a classification module; the classification module is specifically used for:

[0044] The decoupled detection head enhances defect features through the CBAM attention module, and then outputs the probability of 8 types of defects through the classification branch and the bounding box parameters through the regression branch. This solves the optimization conflict problem caused by traditional shared parameters, realizes fine-grained defect classification and accurate localization, and outputs the preliminary classification and localization results of each potential defect.

[0045] In another embodiment of the present invention, a result output module is provided; the result output module is specifically used for:

[0046] The dynamic NMS optimizer first counts the number of defects and calculates the density using a 3×3 window, then dynamically adjusts the NMS threshold based on the density, and finally removes overlapping detection boxes through multi-level NMS to improve the recall rate in dense defect scenes and output the final detection results.

[0047] In another embodiment of the present invention, the system further includes: a power consumption optimization module; the power consumption optimization module is specifically used for:

[0048] During the hardware deployment phase, TensorRT 8.6 was used to quantize the model to INT8 precision, improving inference speed by 2.3 times. The model was deployed on NVIDIA Jetson AGX Orin and power consumption was optimized to 18.8W. Batch size was set to 8 to achieve batch inference, increasing wafer inspection throughput from 12 wafers / hour to 30 wafers / hour, meeting the edge computing needs of high-speed production lines.

[0049] In another embodiment of the present invention, the preprocessing unit is specifically used for:

[0050] The surface image of the chip product is acquired by using an industrial camera;

[0051] The acquired image is divided into 8x8 pixel blocks, and the cumulative distribution function of the gray-level histogram of each block is determined.

[0052] By cropping the extreme gray values ​​at both ends of the cumulative distribution function by 1% and stretching them to the 0-255 range, the image contrast is improved by 40%.

[0053] Then, the gamma value is dynamically adjusted based on the average gray value of the acquired images to solve the problem of uneven illumination;

[0054] Finally, bilinear interpolation was used to uniformly scale the acquired images to 1080×1080 pixels, and a standardized image was output.

[0055] The beneficial effects of this invention are as follows:

[0056] The proposed technical solution achieves significant improvements in accuracy, efficiency, and robustness. In terms of accuracy, the mAP@0.5:0.95 in 12-inch wafer inspection reaches 0.78, a 50% improvement over traditional Faster R-CNN. The fine-grained recognition accuracy for eight typical defect types reaches 96.7%, solving the problem of traditional binary classification's inability to subdivide defect types. Simultaneously, the recall rate for submicron-level defects <10×10 pixels increases from 42.3% to 89.1%, avoiding significant losses in defect detection during the packaging stage. In terms of efficiency, the 1080p resolution image processing speed is increased to 45FPS, far exceeding the minimum requirement of 30FPS for high-speed production lines. Wafer inspection throughput increases from 12 wafers / hour to 30 wafers / hour, matching the pace of high-speed 12-inch wafer production lines. Furthermore, when deployed as an edge device, power consumption is only 18.8W (a 46.3% reduction), meeting the requirements for real-time edge inspection. In terms of robustness, at a defect density of 10 defects / mm²... 2 In dense scenarios, the dynamic NMS optimizer improves recall by 29% and keeps false positive rate below 3.2%, reducing manual re-inspection costs by 30% compared to traditional fixed NMS solutions, and is well-suited to the defect detection needs of the entire semiconductor manufacturing process. Attached Figure Description

[0057] Figure 1 This is the system architecture diagram of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments in this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this application without creative effort are within the scope of protection of this document. Specific Implementation Example 1:

[0060] This application aims to solve four core problems in chip defect detection in the prior art: the problem of missing subpixel-level defects caused by fixed anchor frames; the inability of binary classification design to meet the needs of fine-grained defect type identification; the problem of computational redundancy and insufficient real-time performance caused by two-stage detection architecture; and the problem of misjudgment in high-density defect scenarios with fixed NMS threshold.

[0061] (I) Multi-scale feature pyramid optimization: enhancing the representation of minute defect features

[0062] (1) Technical Principles and Network Architecture

[0063] To address the challenge of feature extraction for sub-pixel defects, this solution constructs a multi-scale detection architecture combining a backbone network and feature fusion.

[0064] Backbone network: CSPDarknet53 is used instead of ResNet-34. Its Cross-Stage Local Connection (CSP) structure divides the feature map into two parts for parallel processing, reducing computation by 30% while preserving more detailed features. The network outputs four layers of feature maps with resolutions of 160×160 (shallow, rich in detail), 80×80, 40×40, and 20×20 (deep, strong semantic information).

[0065] SPP Spatial Pyramid Pooling: An SPP module is embedded at the output of the backbone network. Through parallel pooling kernels of 1×1, 5×5, 9×9, and 13×13, feature maps of arbitrary size are transformed into fixed dimensions, enhancing the adaptability to defects of different sizes (especially tiny defects <10×10 pixels).

[0066] PAN Path Aggregation Network: It adopts a two-way fusion strategy of bottom-up and top-down - shallow features (high resolution) and deep features (high semantics) are fused through lateral connections, so that the 160×160 feature map contains both "defect edge details" and "defect type semantics", improving the feature expression capability by 58%.

[0067] (2) Technical Implementation Details

[0068] Feature hierarchy design:

[0069] P1 layer (160×160): Responsible for detecting defects (such as nano scratches) in pixels ranging from 4×4 to 10×10.

[0070] P2 layer (80×80): Responsible for detecting defects (such as pinholes) in pixels ranging from 10×10 to 32×32.

[0071] P3 layer (40×40): Responsible for detecting defects (such as short circuits) in pixels ranging from 32×32 to 64×64.

[0072] P4 layer (20×20): Responsible for detecting defects >64×64 pixels (such as large-area contamination).

[0073] Fusion strategy: After the features of each layer are reduced in dimensionality by 1×1 convolution, they are fused with the features of adjacent layers by weighted summation (the weights are dynamically adjusted by the attention mechanism) to ensure that features with minor defects are not overwhelmed by deep semantic information.

[0074] (II) Dynamic Anchor Frame Generation Mechanism: Adaptive Matching of Defect Size Distribution

[0075] (1) Technical principles and process design

[0076] To address the limitations of manually setting anchor frames, this solution proposes a data-driven dynamic anchor frame generation strategy:

[0077] Offline clustering stage: The K-means++ algorithm is used to cluster the historical defect dataset (containing 100,000+ labeled boxes) by width and height, generating 5 anchor boxes (minimum 4×4 pixels, maximum 128×128 pixels) to adapt to the chip defect distribution. The clustering loss uses the intersection-over-union ratio (IoU) instead of Euclidean distance to avoid the dominance of large anchor boxes in the clustering results.

[0078] Online adaptive phase: During inference, the anchor frame automatically scales with the resolution of the input image (e.g., if the input image is enlarged from 640×640 to 1080×1080, the anchor frame size is enlarged proportionally by 1.68 times) to ensure that the size ratio of the anchor frame to the defect is consistent under different resolutions.

[0079] Anchor frame filtering mechanism: Filter the appropriate anchor frames according to the feature layer resolution (e.g., only use 4×4 and 8×8 pixel anchor frames for a 160×160 feature layer), reducing invalid anchor frame calculations by 70%.

[0080] (2) Clustering parameters and anchor frame size design

[0081] Cluster center (pixel) Adaptation Defect Type Percentage (historical data) Traditional solution corresponding to anchor frame 4×4 Nano scratches, micro pinholes 23％ None (minimum 32×32) 16×16 Small-scale pollution 31％ 32×32 (Size runs large) 32×32 Short circuit, open circuit 27％ 32×32 (adaptable) 64×64 Metal residue 12％ 64×64 (adapted) 128×128 Large area defects 7％ 128×128 (adapted)

[0082] (III) Decoupling the attention detection head: Improving classification and localization accuracy

[0083] (1) Technical principles of decoupling design

[0084] Traditional methods share network parameters for classification and regression, leading to conflicting optimization objectives for the two tasks (classification focuses on semantic features, while regression focuses on location features). This solution employs a decoupled detection head:

[0085] Classification Branch: An 8-class defect classification network is constructed based on EfficientNet-B3. Fine-grained features are extracted through depthwise separable convolution of the MBConv module. The output layer uses the Softmax activation function to achieve accurate differentiation of 8 types of defects such as short circuits and metal residues.

[0086] Regression branch: A 5-layer convolutional network is used to predict the center point coordinates and width and height offsets of the bounding boxes. The loss function uses GIoU (Generalized IoU) instead of the traditional L1 loss to solve the optimization problem when the bounding boxes do not overlap (GIoU can still provide gradients when IoU=0).

[0087] CBAM Attention Fusion: Channel Attention (CAM) and Spatial Attention (SAM) modules are embedded at the input of the classification and regression branches. CAM enhances the defect feature channels (such as the bright channels of metal residues) through the squeeze-excitation mechanism, and SAM highlights the defect edge region through 7×7 convolution, improving the feature response value by 3.2 times.

[0088] (2) Network structure and parameter design

[0089] Classification branch: EfficientNet-B3 has a depth coefficient of 1.2, a width coefficient of 1.1, an input feature map dimension of 256, and an output of 8-dimensional classification probability.

[0090] Regression branch: 5 layers of 3×3 convolutions (number of channels 256→128→64→32→4), outputting 4-dimensional bounding box parameters (dx,dy,dw,dh).

[0091] Attention module: CAM uses a 2-layer fully connected network (256→64→256), and SAM uses 3×3 convolution + sigmoid activation.

[0092] (iv) Dynamic NMS Optimizer: Adapting to Dense Defect Scenarios

[0093] (1) Dynamic threshold adjustment mechanism

[0094] To address the limitations of a fixed NMS threshold, this solution adjusts the threshold in real time based on defect density.

[0095] Density calculation: Slide a 3×3 window across the feature map, count the number of predicted bounding boxes within each window, and convert this to defect density (number of defects / mm²). 2 ).

[0096] Threshold mapping: when density ≤ 5 particles / mm 2 When the density is >5 / mm, the NMS threshold is set to 0.7 (to retain more candidate boxes); 2 At this point, the threshold linearly decreases to 0.3-0.5 (reducing accidental deletion of overlapping boxes). Threshold calculation formula:

[0097] NMS_threshold=max(0.3,0.7-0.08×(density-5))

[0098] Multi-level NMS: First, perform NMS on defects of the same category, and then perform cross-category NMS on defects of different categories to avoid accidental deletion of overlapping boxes of different types of defects (such as adjacent boxes of "short circuit" and "metal residue"). Specific Implementation Example 2:

[0100] (I) Overall System Architecture

[0101] This solution constructs an end-to-end chip defect detection system comprising "input-preprocessing-feature extraction-feature fusion-anchor box matching-detection prediction-postprocessing-output," with a clear overall process and coordinated module operation. First, the input layer acquires raw images of the chip surface using an industrial camera, providing initial detection data for the system. The raw images then enter the preprocessing module, undergoing adaptive histogram cropping, gamma correction, and resolution normalization to output a high-quality standardized image, laying the foundation for subsequent feature extraction. The preprocessed image is then input into the CSPDarknet53 backbone network, which processes the image in parallel through a cross-stage local connection structure, outputting a 4-level raw feature map containing different details and semantic information. Next, the deepest feature map c4 is processed by the SPP spatial pyramid pooling module, transforming it into a fixed-dimensional feature map s4 through parallel pooling with multi-size pooling kernels, enhancing adaptability to defects of different sizes. Subsequently, the PAN path aggregation network employs a "bottom-up + residual connection" strategy to generate a 4-level fused feature map, achieving complementarity between deep and shallow features. The dynamic anchor box generator, based on five anchor box categories obtained from offline clustering, selects and adapts anchor boxes according to the resolution of the fused feature map and adaptively scales them with the image resolution, matching accurate anchor boxes for each level of feature map. Then, four decoupled detection heads, corresponding to feature maps p1 to p4 respectively, enhance defect features through the CBAM attention module. The classification branch outputs the probability of eight types of defects, and the regression branch outputs bounding box parameters, completing the initial prediction of defect classification and localization. Finally, the dynamic NMS optimizer in the post-processing layer first calculates the defect density and dynamically adjusts the NMS threshold, then removes overlapping detection boxes through multi-level NMS, and finally, the output layer outputs the detection results containing defect category, location, and confidence level.

[0102] 1. Preprocessing module

[0103] The preprocessing module, as the core of the system's data optimization, primarily addresses issues such as low contrast, uneven lighting, and inconsistent resolution in images acquired by industrial cameras. It first divides the image into 8×8 pixel blocks, calculates the cumulative distribution function of the grayscale histogram for each block, and crops the extreme grayscale values ​​at both ends, stretching them to the 0-255 range to improve image contrast by 40%. Next, it dynamically adjusts the gamma value based on the image's average grayscale value (γ = 0.5-0.8 for grayscale <128, γ = 1.2-1.5 for grayscale ≥128) to resolve the uneven lighting issue. Finally, it uses bilinear interpolation to uniformly scale the image to 1080×1080 pixels, outputting a standardized image to provide high-quality input for subsequent modules.

[0104] 2. Extracting image features using the CSPDarknet53 backbone network

[0105] The CSPDarknet53 backbone network undertakes the important task of feature extraction. Its cross-stage local connection structure divides the feature map of each stage into two parts. One part is processed by convolution and residual blocks, and the other part is directly passed. The two parts are concatenated and output, which reduces the amount of computation by 30% while retaining key features. Finally, four levels of original feature maps are generated, where c1 focuses on shallow details and c4 focuses on deep semantics, providing basic feature support for subsequent fusion modules.

[0106] 3. SPP Spatial Pyramid Pooling Module: Implementing Feature Enhancement

[0107] To address the issue of variable sizes in deep feature maps, the SPP spatial pyramid pooling module employs parallel pooling with multiple pooling kernels of 1×1, 5×5, 9×9, and 13×13 sizes on the c4 feature map. The result is then concatenated with the original feature map to transform it into a fixed-dimensional s4 feature map. This effectively enhances the adaptability to defects of different sizes (especially tiny defects <10×10 pixels) and provides deep features with a unified dimension for feature fusion.

[0108] 4. PAN Path Aggregation Network: Achieving Fusion of Details and Semantics

[0109] The PAN path aggregation network achieves deep and shallow feature fusion through "bottom-up + residual connection". First, s4 and c4 are convolved and then residually connected to obtain p4. Then, p4 is upsampled and fused with c3 to obtain p3, p3 is upsampled and fused with c2 to obtain p2, and p2 is upsampled and fused with c1 to obtain p1. This ensures that each level of fused feature map contains both defect details and semantic information, solving the problem of "missing details" or "insufficient semantics" in a single feature layer. The output p1 to p4 provide high-quality feature input for the detection head.

[0110] 5. Dynamic Anchor Frame Generator

[0111] The dynamic anchor box generator first clusters 100,000+ labeled defect boxes using the K-means++ algorithm, obtaining 5 classes of anchor boxes with the intersection-union ratio as the loss. During inference, the anchor boxes are adaptively scaled according to the image resolution, and the appropriate anchor boxes are selected according to the resolution of the fused feature map (e.g., p1 is adapted to 4×4 and 8×8 pixel anchor boxes), reducing 70% of invalid computation and matching accurate anchor boxes for each level of feature map, thereby improving the detection rate of small defects.

[0112] 6. Decoupling detection head

[0113] The decoupled detection head enhances defect features through the CBAM attention module (channel attention highlights key channels, and spatial attention highlights edge regions). Then, it outputs the probability of 8 types of defects through the classification branch (EfficientNet-B3) and the bounding box parameters through the regression branch (5-layer convolution) (using GIoU loss to optimize localization). This solves the optimization conflict problem caused by traditional shared parameters, realizes fine-grained defect classification and accurate localization, and outputs the preliminary classification and localization results of each potential defect.

[0114] 7. Dynamic NMS Optimizer

[0115] The dynamic NMS optimizer first counts the number of defects and calculates the density using a 3×3 window, and then dynamically adjusts the NMS threshold based on the density (density ≤ 5 defects / mm). 2 The threshold is set at 0.7, and linearly decreases to 0.3-0.5 when the threshold is >5. Finally, overlapping detection boxes are removed by multi-level NMS (first deduplication within the same category, then deduplication across categories) to improve the recall rate of dense defect scenes and output the final detection results.

[0116] 8. Hardware Deployment and Optimization Strategies

[0117] During the hardware deployment phase, TensorRT 8.6 was used to quantize the model to INT8 precision, improving inference speed by 2.3 times. The model was deployed on an NVIDIA Jetson AGX Orin (64GB of video memory) and power consumption was optimized to 18.8W (a 46.3% reduction). Batch size was set to 8 to achieve batch inference, increasing wafer inspection throughput from 12 wafers / hour to 30 wafers / hour, meeting the edge computing needs of high-speed production lines.

[0118] The proposed technical solution achieves significant improvements in accuracy, efficiency, and robustness. In terms of accuracy, the mAP@0.5:0.95 in 12-inch wafer inspection reaches 0.78, a 50% improvement over traditional Faster R-CNN. The fine-grained recognition accuracy for eight typical defect types reaches 96.7%, solving the problem of traditional binary classification's inability to subdivide defect types. Simultaneously, the recall rate for submicron-level defects <10×10 pixels increases from 42.3% to 89.1%, avoiding significant losses in defect detection during the packaging stage. In terms of efficiency, the 1080p resolution image processing speed is increased to 45FPS, far exceeding the minimum requirement of 30FPS for high-speed production lines. Wafer inspection throughput increases from 12 wafers / hour to 30 wafers / hour, matching the pace of high-speed 12-inch wafer production lines. Furthermore, when deployed as an edge device, power consumption is only 18.8W (a 46.3% reduction), meeting the requirements for real-time edge inspection. In terms of robustness, at a defect density of 10 defects / mm²... 2In dense scenarios, the dynamic NMS optimizer improves recall by 29% and keeps false positive rate below 3.2%, reducing manual re-inspection costs by 30% compared to traditional fixed NMS solutions, and is well-suited to the defect detection needs of the entire semiconductor manufacturing process.

[0119] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some embodiments, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth by the appended claims.

Claims

1. A chip defect detection method based on target detection, characterized in that, The method includes: The surface image of the chip product is captured by an industrial camera, and the acquired image is preprocessed to obtain a standardized image; Standardized images are input into the backbone network for image feature extraction to obtain chip defect feature images; The chip defect feature image is input into the spatial pyramid pooling module for feature enhancement processing to obtain the enhanced feature image; The enhanced feature image is input into the PAN path aggregation network. The PAN path aggregation network performs detail repair on the enhanced feature image and associates it with corresponding semantic information, so that the enhanced feature image contains both detailed information and semantic information, resulting in high-quality features. Dynamic anchor boxes are generated for high-quality features. Chip defects in the high-quality features are matched with corresponding anchor boxes of the corresponding level by the feature map resolution. The chip defect features with selected anchor boxes are obtained, and chip defects can be detected.

2. The chip defect detection method based on target detection as described in claim 1, characterized in that, The process of generating dynamic anchor boxes for high-quality features also includes: The decoupled detection head enhances defect features through the CBAM attention module, and then outputs the probability of 8 types of defects through the classification branch and the bounding box parameters through the regression branch. This solves the optimization conflict problem caused by traditional shared parameters, realizes fine-grained defect classification and accurate localization, and outputs the preliminary classification and localization results of each potential defect.

3. The chip defect detection method based on target detection as described in claim 2, characterized in that, Following the output of the preliminary results of the classification and location of each potential defect, the following is also included: The dynamic NMS optimizer first counts the number of defects and calculates the density using a 3×3 window, then dynamically adjusts the NMS threshold based on the density, and finally removes overlapping detection boxes through multi-level NMS to improve the recall rate in dense defect scenes and output the final detection results.

4. The chip defect detection method based on target detection as described in claim 3, characterized in that, After outputting the final detection result, the following is also included: During the hardware deployment phase, TensorRT 8.6 was used to quantize the model to INT8 precision, improving inference speed by 2.3 times. The model was deployed on NVIDIA Jetson AGX Orin and power consumption was optimized to 18.8W. Batch size was set to 8 to achieve batch inference, increasing wafer inspection throughput from 12 wafers / hour to 30 wafers / hour, meeting the edge computing needs of high-speed production lines.

5. The chip defect detection method based on target detection as described in claim 1, characterized in that, The process of acquiring surface images of chip products using an industrial camera and preprocessing the acquired images to obtain standardized images includes: The surface image of the chip product is acquired by using an industrial camera; The acquired image is divided into 8x8 pixel blocks, and the cumulative distribution function of the gray-level histogram of each block is determined. By cropping the extreme gray values ​​at both ends of the cumulative distribution function by 1% and stretching them to the 0-255 range, the image contrast is improved by 40%. Then, the gamma value is dynamically adjusted based on the average gray value of the acquired images to solve the problem of uneven illumination; Finally, bilinear interpolation was used to uniformly scale the acquired images to 1080×1080 pixels, and a standardized image was output.

6. A chip defect detection system based on target detection, characterized in that, The system includes: a preprocessing unit, a feature extraction unit, a feature enhancement unit, a semantic supplementation unit, and an anchor box selection unit; The preprocessing unit is used to acquire surface images of chip products using an industrial camera, and to preprocess the acquired images to obtain standardized images. The feature extraction unit is used to extract image features from the standardized image input backbone network to obtain chip defect feature images. The feature enhancement unit is used to perform feature enhancement processing on the input spatial pyramid pooling module of the chip defect feature image to obtain the enhanced feature image. The supplementary semantic unit is used to input the enhanced feature image into the PAN path aggregation network. The PAN path aggregation network performs detail repair on the enhanced feature image and associates it with corresponding semantic information, so that the enhanced feature image contains both detail information and semantic information, resulting in high-quality features. The anchor frame selection unit is used to dynamically generate anchor frames for high-quality features, determine the corresponding level of the matching anchor frame for the chip defects in the high-quality features through the feature map resolution, obtain the chip defect features with selected anchor frames, and realize the detection of chip defects.

7. The chip defect detection system based on target detection as described in claim 6, characterized in that, The system further includes a classification module; the classification module is specifically used for: The decoupled detection head enhances defect features through the CBAM attention module, and then outputs the probability of 8 types of defects through the classification branch and the bounding box parameters through the regression branch. This solves the optimization conflict problem caused by traditional shared parameters, realizes fine-grained defect classification and accurate localization, and outputs the preliminary classification and localization results of each potential defect.

8. The chip defect detection system based on target detection as described in claim 7, characterized in that, The system further includes: a result output module; the result output module is specifically used for: The dynamic NMS optimizer first counts the number of defects and calculates the density using a 3×3 window, then dynamically adjusts the NMS threshold based on the density, and finally removes overlapping detection boxes through multi-level NMS to improve the recall rate in dense defect scenes and output the final detection results.

9. The chip defect detection system based on target detection as described in claim 8, characterized in that, The system further includes a power optimization module; the power optimization module is specifically used for: During the hardware deployment phase, TensorRT 8.6 was used to quantize the model to INT8 precision, improving inference speed by 2.3 times. The model was deployed on NVIDIA Jetson AGX Orin and power consumption was optimized to 18.8W. Batch size was set to 8 to achieve batch inference, increasing wafer inspection throughput from 12 wafers / hour to 30 wafers / hour, meeting the edge computing needs of high-speed production lines.

10. A chip defect detection system based on target detection as described in claim 6, characterized in that, The preprocessing unit is specifically used for: The surface image of the chip product is acquired by using an industrial camera; The acquired image is divided into 8x8 pixel blocks, and the cumulative distribution function of the gray-level histogram of each block is determined. By cropping the extreme gray values ​​at both ends of the cumulative distribution function by 1% and stretching them to the 0-255 range, the image contrast is improved by 40%. Then, the gamma value is dynamically adjusted based on the average gray value of the acquired images to solve the problem of uneven illumination; Finally, bilinear interpolation was used to uniformly scale the acquired images to 1080×1080 pixels, and a standardized image was output.

Images