VCSEL chip multi-modal defect pixel-level segmentation model and method based on global-local prior aggregation and dense link context fusion

Abstract

The present application relates to the technical fields of computer vision, deep learning and semiconductor manufacturing quality detection, in particular to a VCSEL chip multi-modal defect pixel-level segmentation model and method based on global-local prior aggregation and dense link context fusion, which solves the problems that the existing deep learning method is difficult to cope with the multi-scale irregular edge changes of defects in VCSEL chip multi-modal defect detection, and is difficult to represent the local details of defects in the high-level semantic space. The model comprises a basic convolution block, an encoder, a bottleneck layer, a decoder and a prediction head; the global-local prior aggregation module is introduced in the encoder to solve the feature difference and scale mismatch problems between surface defects and dark line defects; and through the dense link context fusion module, the local details of weak dark line defects are effectively retained in the high-level semantic space, so that the synchronous and high-precision automatic pixel-level segmentation of VCSEL surface defects and internal dark line defects is realized.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, deep learning, and semiconductor manufacturing quality inspection technology, specifically a pixel-level segmentation model and method for multimodal defects in VCSEL (Vertical Cavity Surface Emitting Laser) chips based on global-local prior aggregation and dense connection context fusion. Background Technology

[0002] Oxide-confined vertical-cavity surface-emitting lasers (VCSELs) are widely used in high-speed data communication, 3D sensing, and lidar. However, achieving high yield in mass production is a challenging task. The main defects in VCSELs include surface defects (such as scratches, cracks, and epitaxial defects) and internal dark-line defects (DLDs). Currently, industry mainly relies on manual visual inspection, a time-consuming and error-prone process. Under multimodal imaging (optical microscopy and electroluminescence), surface defects exhibit diverse morphologies and random spatial distribution; while internal dark-line defects are typically confined to a local emission aperture and are characterized by blurred boundaries, low contrast, and weak features (e.g., Figure 1 As shown in the figure. Although existing deep learning methods (such as FCN, U-Net, CNN-Transformer, etc.) have been applied to wafer or chip defect detection, they often struggle to handle multi-scale irregular edge variations when dealing with these multimodal defects (i.e., the aforementioned surface defects and internal dark line defects), and they also struggle to represent and preserve subtle local details of defects in a high-level semantic space. Summary of the Invention

[0003] This invention addresses the technical challenges of existing deep learning methods for detecting multimodal defects in VCSEL chips. These challenges include the difficulty in handling irregular edge variations across multiple scales of defects and the difficulty in representing and preserving subtle local details of defects in a high-level semantic space. The invention provides a pixel-level segmentation model and method for multimodal defects in VCSEL chips based on global-local prior aggregation and densely linked context fusion.

[0004] This invention is achieved using the following technical solution: a pixel-level segmentation model for multimodal defects in VCSEL chips based on global-local prior aggregation and densely linked context fusion, comprising: The basic convolutional block is used to receive images from the VCSEL chip and preprocess the images; The encoder, connected to the basic convolutional block and receiving the preprocessed VCSEL chip image, includes multiple sequentially connected global-local prior aggregation modules. Each global-local prior aggregation module employs three parallel encoding strategies for multimodal defects on the preprocessed VCSEL chip image, including surface defects and internal dark line defects. The three parallel encoding strategies include a text prompt path, an irregular path, and a circular path: the text prompt path introduces text prompts for multimodal defects, maps and encodes the text prompts into global semantic prior features of the multimodal defects; the irregular path can adapt to the boundaries of multimodal defects of different shapes and extract the deformation boundary features of multimodal defects; the circular path uses a circular convolutional kernel with a circular mask to extract the fixed-position prior features of dark line defects. After the prior perceptual features extracted by the above three paths are aligned to a unified dimension, residual fusion is performed with the model baseline features and added to output the fused features. The bottleneck layer, connected to the encoder, includes a densely linked context fusion module. The densely linked context fusion module includes a depthwise separable convolution for spatial coding, and cascades multiple dilated convolutions with different dilation rates. The outputs of the depthwise separable convolution and the dilated convolution are densely concatenated. The decoder, connected to the bottleneck layer, includes multiple deep shuffle upsampling modules, which are skip-connected to multiple global-local prior aggregation modules to fuse model baseline features; The prediction head, connected to the decoder, is used to generate the final pixel-level defect prediction results.

[0005] Furthermore, the depthwise separable convolution is divided into two, located at the beginning and end of the multiple dilated convolutions respectively; the dense splicing means that the output of the depthwise separable convolution at the beginning and each dilated convolution is connected to the next level dilated convolution and the depthwise separable convolution at the end.

[0006] Furthermore, the VCSEL chip image received by the basic convolutional block is a multimodal defect image obtained by fusing the optical microscope image and the electroluminescence image of the same VCSEL chip; the preprocessing is to adjust the resolution of the multimodal defect image to the model input size.

[0007] This invention also discloses a pixel-level segmentation method for multimodal defects in VCSEL chips based on global-local prior aggregation and densely linked context fusion, comprising the following steps: Step S1: Data acquisition and input preprocessing; acquire optical microscope images and electroluminescence images of the same VCSEL chip, construct multimodal defect images of the VCSEL chip based on the above images using the basic convolutional block, and adjust the resolution of the multimodal defect images to the model input size; Step S2: Feature downsampling operation; The multimodal defect image with adjusted resolution is fed to the encoder, and the resolution of the multimodal defect image is reduced by using a pixel-based shuffling operation; Step S3: Feature Encoding and Global-Local Prior Aggregation; In the encoding stage, the global-local prior aggregation module generates global semantic prior features of multimodal defects through text prompt paths, extracts deformation boundary features of multimodal defects through irregular paths, and extracts fixed position prior features of dark line defects through circular paths; Finally, after aligning these three sets of prior perceptual features to a unified dimension, residual fusion is performed with the model baseline features and added, and the fused features are output. Step S4: Dense fusion of multi-scale context features; The encoder inputs the fused features into the densely connected context fusion module of the bottleneck layer, which then passes through depthwise separable convolutional layers and multiple dilated convolutional layers with different dilation rates. The context feature matrices output by each layer are densely concatenated along the channel dimension. Step S5: Feature Decoding and Skip Connections; The result after dense stitching is input into the decoder, and the spatial resolution of the multimodal defect image is gradually restored using the deep shuffle upsampling module; At the same time, skip connections are used to integrate the model baseline features extracted by the encoder into the high-resolution features to restore the continuity of the multimodal defect features; Step S6: Defect segmentation prediction output; The model's prediction head generates the final pixel-level prediction result, which outputs a segmentation mask that accurately distinguishes the background, surface defects, dark line defects, and luminous aperture.

[0008] The beneficial effects of this invention are as follows: This invention solves the problem of feature differences and scale mismatch between surface defects and dark line defects by introducing a global-local prior aggregation module; and effectively preserves the local details of weak dark line defects in the high-level semantic space through a densely linked context fusion module, thereby achieving simultaneous, high-precision automated pixel-level segmentation of VCSEL surface defects and internal dark line defects. This results in significant progress in improving the yield of automated quality inspection of VCSEL chips and reducing the false negative rate, and has extremely high industrial application value. Attached Figure Description

[0009] Figure 1 The images show electroluminescence (EL) images (characterizing internal dark line defects) and optical microscope images (characterizing surface defects) of the same VCSEL chip; (a) is a classification diagram of internal defects in infrared light; (b) is a classification diagram of surface defects in visible light. Figure 2 This is an architectural diagram of the model described in this invention; Figure 3The figures show a comparison between the detection results obtained by the method described in this invention and the results obtained by existing advanced methods; (a) is a comparison of the detection results of internal dark line defects; (b) is a comparison of the detection results of surface defects; ① - true positive, ② - false positive, ③ - false negative. Detailed Implementation

[0010] The present invention will be further described below with reference to the accompanying drawings.

[0011] like Figure 2 The diagram shows the architecture of the VCSEL chip multimodal defect pixel-level segmentation model based on global-local prior aggregation and densely linked context fusion, as described in this invention, including: The base convolutional block (BaseConv) is used to receive images from the VCSEL chip and preprocess them.

[0012] The encoder, connected to the basic convolutional blocks and receiving the preprocessed VCSEL chip image, includes multiple sequentially connected Global-Local Prior Aggregation Blocks (GLPA). Each GLPA employs three parallel encoding strategies for multimodal defects on the preprocessed VCSEL chip image, including surface defects and internal dark line defects. The three parallel encoding strategies are: a Text Prompt Path, an Irregular Path, and a Circular Path. The Text Prompt Path introduces text prompts for the multimodal defects, mapping and encoding these prompts into global semantic prior features of the multimodal defects to stabilize the semantic direction of multimodal defect segmentation. The Irregular Path combines multi-scale dilated convolutions... Convolutions and Deformable Convolutions can adapt to the boundaries of multimodal defects of different shapes and extract the deformation boundary features of multimodal defects. The circular path targets the approximately circular features of the luminous aperture, and uses a circular convolution kernel with a circular mask to restrict the effective weights within the circular support area, so as to promote the smooth transition of features from global surface defects to local weak dark line defect features, thereby extracting the fixed-position prior features of dark line defects. After the prior perceived features extracted by the above three paths are aligned to a unified dimension, residual fusion is performed with the model baseline features and added to output the fused features.

[0013] The bottleneck layer, connected to the encoder, includes a densely linked context fusion module (Dense Atrous Context Fusion, abbreviated as DACF). The Dense Atrous Context Fusion module includes depthwise separable convolutions (DWConv 3×3) for spatial encoding, and cascades multiple dilated convolutions with different dilation rates. The outputs of the depthwise separable convolutions and dilated convolutions are densely concatenated to expand the receptive field, specifically for capturing and preserving subtle boundary details of faint dark line defects in the high-level semantic space.

[0014] The decoder, connected to the bottleneck layer, includes multiple deep shuffle upsampling blocks (DSUBs). These multiple deep shuffle upsampling blocks are skipped to multiple global-local prior aggregation blocks to fuse the model baseline features. The prediction head, connected to the decoder, is used to generate the final pixel-level defect prediction results.

[0015] The depthwise separable convolution is divided into two, located at the beginning and end of the multiple dilated convolutions respectively; the dense connection means that the output of the depthwise separable convolution at the beginning and each dilated convolution is connected to the next level dilated convolution and the depthwise separable convolution at the end.

[0016] The VCSEL chip image received by the basic convolutional block is a multimodal defect image obtained by fusing the optical microscope image and the electroluminescence image of the same VCSEL chip; the preprocessing is to adjust the resolution of the multimodal defect image to the model input size.

[0017] As is well known to those skilled in the art, before formal detection, the model needs to be tested on a test set until it meets the relevant requirements before it can be used to detect VCSEL chips. To address the severe class imbalance and boundary ambiguity challenges in multimodal defect datasets, the model is optimized during training using a hybrid loss function combining Focal Loss and Dice Loss.

[0018] In specific implementation, the VCSEL chip multimodal defect pixel-level segmentation method based on global-local prior aggregation and dense connection context fusion described in this invention includes the following steps: Step S1: Data Acquisition and Input Preprocessing; Acquire optical microscope images (characterizing surface defects) and electroluminescence (EL) images (characterizing internal dark line defects) of the same VCSEL chip. The basic convolutional block constructs a multimodal defect image of the VCSEL chip based on the above images, and adjusts the resolution of the multimodal defect image to the model input size of 256×256, such as... Figure 1 As shown; Step S2: Feature downsampling operation; The multimodal defect image with adjusted resolution is fed to the encoder, and the resolution of the multimodal defect image is reduced by using a pixel shuffle operation to avoid the loss of weak feature information caused by traditional downsampling methods; Step S3: Feature Encoding and Global-Local Prior Aggregation; In the encoding stage, the feature map and text prompts for different semantic categories (background, surface defects, dark line defects, aperture) are input into the global-local prior aggregation module. The global-local prior aggregation module generates global semantic prior features for multimodal defects through text prompt paths, extracts deformation boundary features for multimodal defects through irregular paths, and extracts fixed-position prior features for dark line defects through circular paths; finally, these three sets of prior perceptual features are aligned to a unified dimension, and residual fusion is performed with the model baseline features to add them together, and the fused features are output. Step S4: Dense fusion of multi-scale contextual features; The encoder inputs the fused features into the densely connected contextual fusion module of the bottleneck layer, which then passes through depthwise separable convolutional layers and multiple dilated convolutional layers with different dilation rates. The contextual feature matrices output by each layer are densely connected along the channel dimension to enhance long-distance dependency modeling. Step S5: Feature Decoding and Skip Connections; The result after dense stitching is input into the decoder, and the spatial resolution of the multimodal defect image is gradually restored using the deep shuffle upsampling module; At the same time, skip connections are used to integrate the model baseline features extracted by the encoder into the high-resolution features to restore the continuity of the dark line features. Step S6: Defect segmentation prediction output; The final pixel-level prediction result is generated through the model's prediction head, that is, the segmentation mask that accurately distinguishes the background, surface defects, dark line defects and luminous aperture.

[0019] The present invention will be further described below with reference to the embodiments.

[0020] To objectively evaluate the effectiveness of the proposed VCSEL chip multimodal defect pixel-level segmentation model and method based on global-local prior aggregation and dense link context fusion (hereinafter referred to as UniDefect model and method), this embodiment conducted rigorous comparative experiments using an industrial-grade VCSEL chip multimodal dataset.

[0021] To verify the superiority of this invention, the UniDefect method of this invention was compared with existing mainstream defect segmentation models and deep learning networks. Specific results are shown in Table 1 and... Figure 3The experimental results show that: The overall performance of this invention is at a leading level: on real industrial datasets, the UniDefect method described in this invention ultimately achieves excellent results. mIOU and Dice The fraction overcomes the bottleneck of multimodal defect detection, namely, that existing mainstream methods often have difficulty simultaneously taking into account surface defects of different shapes and internal dark line defects (DLDs) with extremely weak features when processing VCSEL chips.

[0022] By comparing with the actual annotations. Figure 3 The diagram visually reflects the correctness of the judgment: the area indicated by label ① represents a true positive (TP), which is the area where the prediction results of each model or deep learning network completely match the actual defect, indicating a valid detection; the area indicated by label ② represents a false positive (FP), which is a misdetected area, indicating that each model or deep learning network over-responds to features and incorrectly identifies a normal area as a defect; the area indicated by label ③ represents a false negative (FN), which is a missed detection area, indicating that each model or deep learning network's predicted feature response is insufficient and fails to successfully detect the actual defect present therein. Figure 3 The term "the proposed method" refers to the UniDefect method designed in this invention.

[0023] Table 1. Comparison of metrics between UniDefect method and existing mainstream methods

[0024] In Table 1, "Proposed Method" represents the UniDefect method designed in this invention, wherein... Dice Used to assess the similarity between predicted and actual values. mIOU It is used to assess and measure the overall overlap between predicted and actual results.

[0025] Objectively speaking, mIOU The significant lead in metrics, as shown in Table 1, demonstrates that the "Global-Local Prior Aggregation (GLPA)" technique employed in this invention can fit the true physical boundaries of defects with extremely high accuracy. Whether it is an irregularly shaped surface defect or an approximately circular aperture structure, the spatial overlap between the predicted mask and the actual annotation in this invention reaches the high precision requirements of industrial-grade standards.

[0026] Up to 96.69% DiceThe scores strongly demonstrate the invention's superior robustness in dealing with the problem of "extreme sample imbalance." In the background of the entire chip image, the proportion of tiny defective pixels is extremely low, and existing technologies are prone to feature loss or assimilation into the background; however, the overwhelming advantage of this invention in this metric proves its extremely high sensitivity to weak targets.

[0027] In summary, the above indicative evaluations demonstrate that this invention overcomes the technical limitations of existing machine vision and deep learning methods in handling "multimodal, multi-scale, and weak edge" defects. The comprehensive superiority of various objective evaluation data fully confirms that this invention not only possesses outstanding substantive technical features but also achieves significant progress in improving the yield of automated quality inspection of VCSEL chips and reducing the false negative rate, thus possessing extremely high industrial application value.

Claims

1. A pixel-level segmentation model for multimodal defects in VCSEL chips based on global-local prior aggregation and densely linked context fusion, characterized in that, include: The basic convolutional block is used to receive images from the VCSEL chip and preprocess the images; The encoder, connected to the basic convolutional block and receiving the preprocessed VCSEL chip image, includes multiple sequentially connected global-local prior aggregation modules. Each global-local prior aggregation module employs three parallel encoding strategies for multimodal defects on the preprocessed VCSEL chip image, including surface defects and internal dark line defects. The three parallel encoding strategies include a text prompt path, an irregular path, and a circular path: the text prompt path introduces text prompts for multimodal defects, maps and encodes the text prompts into global semantic prior features of the multimodal defects; the irregular path can adapt to the boundaries of multimodal defects of different shapes and extract the deformation boundary features of multimodal defects; the circular path uses a circular convolutional kernel with a circular mask to extract the fixed-position prior features of dark line defects. After the prior perceptual features extracted by the above three paths are aligned to a unified dimension, residual fusion is performed with the model baseline features and added to output the fused features. The bottleneck layer, connected to the encoder, includes a densely linked context fusion module. The densely linked context fusion module includes a lightweight depthwise separable convolution for spatial coding, and cascades multiple dilated convolutions with different dilation rates. The outputs of the lightweight depthwise separable convolution and the dilated convolution are densely concatenated. The decoder, connected to the bottleneck layer, includes multiple deep shuffle upsampling modules, which are skip-connected to multiple global-local prior aggregation modules to fuse model baseline features; The prediction head, connected to the decoder, is used to generate the final pixel-level defect prediction results.

2. The VCSEL chip multimodal defect pixel-level segmentation model based on global-local prior aggregation and densely linked context fusion as described in claim 1, characterized in that, The lightweight depthwise separable convolution consists of two parts, located at the beginning and end of the multiple dilated convolutions respectively; the dense connection means that the output of the lightweight depthwise separable convolution at the beginning and each dilated convolution is connected to the next level dilated convolution and the lightweight depthwise separable convolution at the end.

3. The VCSEL chip multimodal defect pixel-level segmentation model based on global-local prior aggregation and densely linked context fusion as described in claim 2, characterized in that, The VCSEL chip image received by the basic convolutional block is a multimodal defect image obtained by fusing the optical microscope image and the electroluminescence image of the same VCSEL chip; the preprocessing is to adjust the resolution of the multimodal defect image to the model input size.

4. A pixel-level segmentation method for multimodal defects in VCSEL chips based on global-local prior aggregation and dense linking context fusion, implemented using the pixel-level segmentation model for multimodal defects in VCSEL chips based on global-local prior aggregation and dense linking context fusion as described in claim 3, characterized in that... Includes the following steps: Step S1: Data acquisition and input preprocessing; acquire optical microscope images and electroluminescence images of the same VCSEL chip, construct multimodal defect images of the VCSEL chip based on the above images using the basic convolutional block, and adjust the resolution of the multimodal defect images to the model input size; Step S2 Feature downsampling operation; The adjusted resolution multimodal defect image is fed to the encoder, and the resolution of the multimodal defect image is reduced by using a pixel-based shuffling operation. Step S3: Feature Encoding and Global-Local Prior Aggregation; In the encoding stage, the global-local prior aggregation module generates global semantic prior features of multimodal defects through text prompt paths, extracts deformation boundary features of multimodal defects through irregular paths, and extracts fixed position prior features of dark line defects through circular paths; Finally, after aligning these three sets of prior perceptual features to a unified dimension, residual fusion is performed with the model baseline features and added, and the fused features are output. Step S4: Dense fusion of multi-scale contextual features; The encoder inputs the fused features into the densely connected contextual fusion module of the bottleneck layer, which then passes through a lightweight depthwise separable convolution and multiple dilated convolutional layers with different dilation rates. The contextual feature matrices output by each layer are densely concatenated along the channel dimension. Step S5: Feature Decoding and Skip Connections; The result after dense stitching is input into the decoder, and the spatial resolution of the multimodal defect image is gradually restored using the deep shuffle upsampling module; At the same time, skip connections are used to integrate the model baseline features extracted by the encoder into the high-resolution features to restore the continuity of the multimodal defect features; Step S6: Defect segmentation prediction output; The model's prediction head generates the final pixel-level prediction result, which outputs a segmentation mask that accurately distinguishes the background, surface defects, dark line defects, and luminous aperture.

Images