A Defect Detection Method and System Based on Multimodal Semantic Segmentation Model VCSEL

Abstract

The present application relates to the field of vertical cavity surface emitting laser (VCSEL) chip reliability analysis, specifically to a VCSEL defect detection method and system based on a multi-modal semantic segmentation model, which solves the technical problems of low detection efficiency and poor detection accuracy caused by the complicated defect detection process and backward detection technology. The system comprises a probe station for placing VCSEL sample chips, a conveyor belt, an infrared CCD, a visible light CCD, a visible light source, an infrared light source and a computer; the signal output ends of the infrared CCD and the visible light CCD are connected to the computer; the computer is built-in with a multi-modal image fusion algorithm and a VCSEL multi-modal semantic segmentation model. The present application uses multi-modal data to train the semantic segmentation model, realizes accurate identification and positioning of VCSEL multi-class defects, improves detection accuracy and efficiency, and provides strong support for quality control and reliability evaluation of VCSEL.

Description

Technical Field

[0001] This invention relates to the field of reliability analysis of vertical cavity surface-emitting laser (VCSEL) chips, specifically to a VCSEL defect detection method and system based on a multimodal semantic segmentation model. Background Technology

[0002] Vertical-cavity surface-emitting lasers (VCSELs), as a type of high-performance semiconductor laser, are widely used in optical communication, 3D sensing, and lidar due to their advantages such as low power consumption, high modulation rate, and ease of two-dimensional integration. However, during the manufacturing and use of VCSEL chips, factors such as material defects, process fluctuations, and thermal stress can easily lead to damage such as electrode detachment, oxide layer damage, and active region defects, which seriously affect their performance and lifespan.

[0003] Traditional methods for detecting defects in VCSEL chips primarily rely on electrical testing and optical microscopy. Electrical testing methods assess the performance of VCSEL chips by measuring parameters such as IV and L1 characteristics, but they struggle to accurately pinpoint the location and type of defects. While optical microscopy allows for direct observation of the VCSEL surface morphology, it is ineffective at detecting internal defects, relies heavily on human experience, is inefficient, and is susceptible to subjective biases.

[0004] In recent years, with the rapid development of artificial intelligence technology, deep learning-based image recognition technology has shown great potential in the field of industrial defect detection. However, most existing deep learning-based VCSEL chip defect detection methods use single-modal data (such as optical images or electrical signals), which are difficult to comprehensively reflect the defect damage information of VCSEL chips, resulting in limited detection accuracy. Summary of the Invention

[0005] This invention addresses the technical problems of low detection efficiency and poor detection accuracy of VCSEL chips due to cumbersome defect detection processes and outdated detection technologies. It provides a VCSEL defect detection method and system based on a multimodal semantic segmentation model.

[0006] This invention is achieved using the following technical solution: a defect detection method based on the multimodal semantic segmentation model VCSEL, comprising the following steps:

[0007] S1. Obtain the multimodal defect dataset;

[0008] S11. Acquire visible light and infrared images of multiple VCSEL sample chips; the acquired VCSEL sample chips include defect-free VCSEL chips and defective VCSEL chips.

[0009] S12. The visible light image and infrared image of each VCSEL sample chip are fused using a multimodal image fusion algorithm to obtain a multimodal defect fusion image of each VCSEL sample chip.

[0010] S13. Mark the multimodal defect fusion images of the VCSEL sample chips obtained in step S12, remove the multimodal defect fusion images without defects, and mark the defect category, defect location and defect morphology of the multimodal defect fusion images with defects. The marked multimodal defect fusion images are used as chip defect samples to form a multimodal defect dataset.

[0011] Design of S2 and VCSEL multimodal semantic segmentation models;

[0012] S21. Use a pre-trained convolutional neural network ResNet as the encoder, which includes multiple layers of ordinary convolutions, with an SE attention module added after each ordinary convolution layer.

[0013] S22. The decoder is constructed using dilated convolutions that have the same number of layers as the encoder and are one-to-one with each other. Except for the last layer, each ordinary convolution layer of the encoder is connected to the dilated convolution layer of the corresponding layer of the decoder in a skip connection. The last ordinary convolution layer of the encoder is connected to the last dilated convolution layer of the decoder through a depth-separable convolution to form the VCSEL multimodal semantic segmentation model.

[0014] Training S3 and VCSEL multimodal semantic segmentation models

[0015] Chip defect samples from the multimodal defect dataset obtained in S1 are input into the VCSEL multimodal semantic segmentation model to train the model. The VCSEL multimodal semantic segmentation model can output a multimodal defect segmentation result map for each chip defect sample. The multimodal defect segmentation result map contains the defect category, defect location, and defect morphology of the chip defect sample. When the defect category, defect location, and defect morphology in the multimodal defect segmentation result map output by the VCSEL multimodal semantic segmentation model have an accuracy of more than 95% compared with the chip defect sample, the VCSEL multimodal semantic segmentation model is considered to have completed training.

[0016] Detection of S4 and VCSEL multimodal semantic segmentation models

[0017] Visible light and infrared images of the VCSEL chip under test are acquired. A multimodal defect fusion image of the VCSEL chip under test is obtained through a multimodal image fusion algorithm. Then, the multimodal defect fusion image of the VCSEL chip under test is input into the VCSEL multimodal semantic segmentation model trained in S3. The VCSEL multimodal semantic segmentation model outputs a multimodal defect segmentation result image containing the defect category, defect location and defect morphology of the VCSEL chip under test.

[0018] Furthermore, the defect categories include surface contamination, mechanical scratches, epitaxial defects, and dark defects in the light-emitting aperture.

[0019] Furthermore, in step S3, the VCSEL multimodal semantic segmentation model is configured with the following training parameters before training: the initial learning rate is set to 1e. -4 The learning rate is dynamically adjusted using a cosine annealing learning rate scheduler; the AdamW optimizer is used in conjunction with weight decay to prevent overfitting; the batch size is set to 16 and the number of training epochs is 150; and the model weights are saved after training.

[0020] A VCSEL defect detection system based on a multimodal semantic segmentation model is disclosed, which implements a VCSEL defect detection method based on a multimodal semantic segmentation model. The system includes a probe station for placing VCSEL sample chips, a conveyor belt for placing the VCSEL chip under test, an image acquisition system for acquiring images of the VCSEL sample chip and the VCSEL chip under test, and a computer. The image acquisition system includes an infrared CCD for acquiring infrared images of the VCSEL sample chip and the VCSEL chip under test, a visible light CCD for acquiring visible light images of the VCSEL sample chip and the VCSEL chip under test, and a visible light source and an infrared light source for providing light to the VCSEL sample chip and the VCSEL chip under test. The signal output terminals of the infrared CCD and the visible light CCD are connected to the computer. The probe station, conveyor belt, visible light source, and infrared light source are all electrically connected to the computer and controlled by the computer. The computer has a built-in multimodal image fusion algorithm and a VCSEL multimodal semantic segmentation model.

[0021] Furthermore, before the defect detection system performs its detection, it first sets the conveyor belt speed, the intensity of the visible light source and the infrared light source, and the shooting angles of the infrared CCD and the visible light CCD, and assigns an identification number to each VCSEL chip under test. Then, it starts automatic detection. The computer's built-in multimodal image fusion algorithm fuses the acquired visible light image and infrared image of the VCSEL chip under test to obtain a fused image of the VCSEL chip under test. After that, the computer's built-in trained VCSEL multimodal semantic segmentation model detects the fused image of the VCSEL chip under test. If the fused image of the VCSEL chip under test has no defects, the computer outputs the identification number of the VCSEL chip under test and marks it as qualified; if the fused image of the VCSEL chip under test has defects, the computer outputs the identification number of the VCSEL chip under test, the defect category, the defect location, and the defect morphology and marks it as unqualified.

[0022] This invention proposes a VCSEL multimodal semantic segmentation model targeting the multimodal defect image features of VCSELs. It designs an innovative feature extraction convolutional module, combines it with an SE attention mechanism, and simultaneously trains the semantic segmentation model using multimodal data (visible light defect images and infrared light defect images). This enables accurate identification and localization of multiple types of VCSEL defects and, for the first time, simultaneous detection of internal and external VCSEL defects, ultimately improving detection efficiency and accuracy. Addressing the problems of low efficiency and high cost of repetitive screening in traditional inspection processes, the defect detection system described in this invention effectively improves the automation level of the inspection process, significantly reduces production costs and inspection time, and provides an efficient and low-cost solution for laser chip defect detection. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the VCSEL defect detection system based on the multimodal semantic segmentation model of the present invention. Figure 1 .

[0024] Figure 2 This is a schematic diagram of the VCSEL defect detection system based on the multimodal semantic segmentation model of the present invention. Figure 2 .

[0025] Figure 3 This is a schematic diagram illustrating the process of obtaining a multimodal defect dataset according to the present invention.

[0026] Figure 4 This is a schematic diagram of defects in various VCSEL chips in a multimodal defect dataset.

[0027] Figure 5 This is a schematic diagram of the VCSEL multimodal semantic segmentation model.

[0028] Figure 6This is a flowchart illustrating the operation of the defect detection system.

[0029] Figure 7 An experimental comparison chart was created for the multimodal defect dataset in step S1 of this invention.

[0030] Figure 8 This is a schematic diagram of the multimodal defect detection results in step S4 of the present invention.

[0031] 1-VCSEL sample chip, 2-probe station, 3-VCSEL chip under test, 4-conveyor belt, 5-computer, 6-infrared CCD, 7-visible CCD, 8-surface defect, 9-internal defect, 10-light emission aperture. Detailed Implementation

[0032] A defect detection method based on the multimodal semantic segmentation model VCSEL includes the following steps:

[0033] S1. Obtain the multimodal defect dataset;

[0034] S11. Acquire visible light and infrared images of multiple VCSEL sample chips 1; the acquired VCSEL sample chips 1 include defect-free VCSEL chips and defective VCSEL chips.

[0035] S12. The visible light image and infrared image of each VCSEL sample chip 1 are fused using a multimodal image fusion algorithm to obtain a multimodal defect fusion image of each VCSEL sample chip 1.

[0036] S13. The multimodal defect fusion images of VCSEL sample chip 1 obtained in step S12 are labeled. Multimodal defect fusion images without defects are removed, and the defective multimodal defect fusion images are labeled with defect category, defect location and defect morphology. The labeled multimodal defect fusion images are used as chip defect samples to form a multimodal defect dataset. The defect categories include surface contamination, mechanical scratches, epitaxial defects and dark defects of light-emitting holes.

[0037] Design of S2 and VCSEL multimodal semantic segmentation models;

[0038] S21. Use a pre-trained convolutional neural network ResNet as the encoder, which includes multiple layers of ordinary convolutions, with an SE attention module added after each ordinary convolution layer.

[0039] S22. The decoder is constructed using dilated convolutions that have the same number of layers as the encoder and are one-to-one with each other. Except for the last layer, each ordinary convolution layer of the encoder is connected to the dilated convolution layer of the corresponding layer of the decoder in a skip connection. The last ordinary convolution layer of the encoder is connected to the last dilated convolution layer of the decoder through a depth-separable convolution to form the VCSEL multimodal semantic segmentation model.

[0040] Training S3 and VCSEL multimodal semantic segmentation models

[0041] Chip defect samples from the multimodal defect dataset obtained in S1 are input into the VCSEL multimodal semantic segmentation model for training. The VCSEL multimodal semantic segmentation model outputs a multimodal defect segmentation result map for each chip defect sample, which includes the defect category, defect location, and defect morphology of the chip defect sample. When the precision of the defect category, defect location, and defect morphology in the multimodal defect segmentation result map output by the VCSEL multimodal semantic segmentation model reaches 95% or more compared with the chip defect sample, the VCSEL multimodal semantic segmentation model is considered to have completed training. The following training parameters and configurations are set before training the VCSEL multimodal semantic segmentation model: the initial learning rate is set to 1e. -4 The learning rate was dynamically adjusted using a cosine annealing learning rate scheduler; the AdamW optimizer was used in conjunction with weight decay to prevent overfitting; the batch size was set to 16 and the number of training epochs was 150; and the model weights were saved after training.

[0042] Detection of S4 and VCSEL multimodal semantic segmentation models

[0043] Visible light and infrared images of the VCSEL chip 3 under test are acquired. A multimodal defect fusion image of the VCSEL chip 3 under test is obtained through a multimodal image fusion algorithm. Then, the multimodal defect fusion image of the VCSEL chip 3 under test is input into the VCSEL multimodal semantic segmentation model trained in S3. The VCSEL multimodal semantic segmentation model outputs a multimodal defect segmentation result image containing the defect category, defect location, and defect morphology of the VCSEL chip 3 under test. The multimodal image fusion algorithm adopts, but is not limited to, the method disclosed in Chinese Patent Publication No. CN117830235A.

[0044] Currently, quality screening on chip production lines is done independently, first detecting surface defects and then internal defects, which is very time-consuming. Furthermore, the detection method is manual, leading to a high false positive rate. The innovation of this invention lies in its ability to simultaneously detect both surface and internal defects in chips, significantly improving detection efficiency and accuracy. The significance of the VCSEL multimodal semantic segmentation model designed in this invention is twofold: first, it greatly simplifies the detection process on chip production lines, achieving "one-step" detection and reducing detection efficiency; second, the algorithm's detection avoids the high false positive rate problem associated with manual detection.

[0045] A VCSEL defect detection system based on a multimodal semantic segmentation model includes a probe station 2 for placing a VCSEL sample chip 1, a conveyor belt 4 for placing a VCSEL chip 3 under test, an image acquisition system for acquiring images of the VCSEL sample chip 1 and the VCSEL chip 3 under test, and a computer 5. The image acquisition system includes an infrared CCD 6 for acquiring infrared images of the VCSEL sample chip 1 and the VCSEL chip 3 under test, a visible light CCD 7 for acquiring visible light images of the VCSEL sample chip 1 and the VCSEL chip 3 under test, and a visible light source and an infrared light source for providing light to the VCSEL sample chip 1 and the VCSEL chip 3 under test. The signal output terminals of the infrared CCD 6 and the visible light CCD 7 are connected to the computer 5. The probe station 2, the conveyor belt 4, the visible light source, and the infrared light source are all electrically connected to the computer 5 and controlled by the computer 5 (connected via an HDMI cable). The computer 5 has a built-in multimodal image fusion algorithm and a VCSEL multimodal semantic segmentation model.

[0046] When acquiring a multimodal defect dataset, the VCSEL sample chip 1 is placed on the probe station 2, the light source is adjusted, and then infrared and visible light images of the VCSEL sample chip 1 are acquired using an infrared CCD 6 and a visible light CCD 7. The probe station 2, conveyor belt 4, visible light source, and infrared light source are all electrically connected to and controlled by the computer 5. This means that the computer can control the movement of the probe station 2 and conveyor belt 4 (such as the lifting and lowering of the probe station 2, the running direction and speed of the conveyor belt 4), as well as the intensity of the visible light and infrared light sources. The visible light and infrared light sources are supported by corresponding brackets, allowing for angle and height adjustment. Of course, the angles of the visible light and infrared light sources can also be adjusted via the computer 5 according to the needs of the actual application; these are all easily achievable by those skilled in the art.

[0047] Before the defect detection system performs its operation, an infrared CCD 6 and a visible light CCD 7 are placed above a conveyor belt. Then, the conveyor belt speed, the intensity of the visible and infrared light sources, and the shooting angles of the infrared and visible light CCD 7 are set. Each VCSEL chip 3 under test is assigned an identification number. Automatic detection then begins. The computer 5's built-in multimodal image fusion algorithm fuses the acquired visible light and infrared images of the VCSEL chip 3 under test, obtaining a multimodal defect fusion image. The computer 5's built-in, trained VCSEL multimodal semantic segmentation model then detects the multimodal defect fusion image. If the multimodal defect fusion image of the VCSEL chip 3 under test is free of defects, the computer 5 outputs the identification number of the VCSEL chip 3 and marks it as qualified. If the multimodal defect fusion image of the VCSEL chip 3 under test is defective, the computer outputs the identification number, defect type, defect location, and defect morphology of the VCSEL chip 3 under test and marks it as unqualified.

[0048] Due to the complexity of VCSEL chip manufacturing processes, various defects often arise, but these defects are characterized by low identification accuracy and small size. Traditional detection methods struggle to meet the requirements for both accuracy and speed. This invention improves the U-Net network and deploys the algorithm in a computer, connecting it to relevant equipment such as a light source, infrared CCD6, conveyor belt 4, and probe station 2, thus forming a complete VCSEL defect detection system based on a multimodal semantic segmentation model.

[0049] The present invention will be further described below with reference to the accompanying drawings.

[0050] 1. Preparation of multimodal defect dataset

[0051] like Figure 1 As shown, the defect detection system of the present invention comprises a probe station 2, an infrared CCD 6, a visible light CCD 7, a visible light source, an infrared light source, and a computer 5, constituting a VCSEL multimodal defect acquisition system. This system is used to acquire visible light and infrared images of multiple VCSEL sample chips 1. During acquisition, the VCSEL sample chip 1 is placed on the probe station 2, and the visible light source and infrared light source respectively illuminate the VCSEL sample chip 1. Simultaneously, the visible light CCD 7 and the infrared CCD 6 acquire the visible light and infrared images of the VCSEL sample chip 1, respectively.

[0052] like Figure 3 As shown, after acquiring a large dataset of visible light and infrared images of VCSEL sample chip 1, a multimodal defect dataset required for this invention is obtained using a multimodal image fusion algorithm. The defect types mainly include four categories: surface contamination, mechanical scratches, epitaxial defects, and dark defects in the light-emitting aperture, such as... Figure 4 As shown.

[0053] 2. Design and Training of VCSEL Multimodal Semantic Segmentation Model

[0054] like Figure 5 As shown, the improved U-Net model introduces the following optimizations based on the original U-Net:

[0055] (1) Encoder section

[0056] A pre-trained ResNet convolutional neural network is used as the encoder, replacing the simple convolutional layers of the original U-Net. This facilitates the extraction of multi-level feature information, from low-level features (defect edges, defect texture) to high-level semantic features (defect shape). An SE (Squeeze-and-Excitation) attention module is added after each layer of the encoder to enhance the weights of subtle defect features.

[0057] (2) Decoder section

[0058] Each layer uses dilated convolutions instead of ordinary convolutions to expand the receptive field and capture more contextual information. Features from each layer of the encoder are directly connected to the corresponding layer of the decoder to enhance defect feature reuse. Simultaneously, multi-scale features extracted from the encoder are fused to achieve accurate pixel-level segmentation of multimodal defects.

[0059] The last layer uses depthwise separable convolution to reduce the number of parameters, optimize computational efficiency, improve inference speed, and finally output high-resolution defect segmentation results.

[0060] (3) Model hyperparameter setting and training

[0061] Training parameters: Initial learning rate set to 1e -4 The model employs a cosine annealing LR scheduler to dynamically adjust the learning rate and avoid getting trapped in local optima. The AdamW optimizer, combined with weight decay, is used to prevent overfitting. The batch size is set to 16, and the number of training epochs is 150. After training, the model weights are saved, and the VCSEL multimodal semantic segmentation model is tested.

[0062] 3. VCSEL Defect Detection System Based on Multimodal Semantic Segmentation Model

[0063] The VCSEL multimodal semantic segmentation model is deployed on computer 5 and connected to visible light source, infrared light source, infrared CCD 6, visible light CCD 7, and conveyor belt 4 to form a complete automated VCSEL defect detection system. The specific implementation is as follows: Figure 2 As shown. The VCSEL chip 3 under test is placed on the conveyor belt 4, and the position and angle of the infrared light source and the visible light source are adjusted so that the infrared CCD 6 and the visible light CCD 7 can acquire clear infrared and visible light images.

[0064] The flowchart of the defect detection system operation is as follows: Figure 6 As shown:

[0065] 1) The system starts running;

[0066] 2) Parameter initialization: Set network parameters, conveyor belt 4 running speed, number of VCSEL chips 3 to be tested, light source intensity, voltage and current parameters, and shooting angle so that the light source and CCD can obtain the best image and work well with the operation of the conveyor belt 4.

[0067] 3) Automatic detection begins. Computer 3 uses its built-in VCSEL multimodal semantic segmentation model to determine whether the VCSEL chip 3 under test is qualified. The VCSEL multimodal semantic segmentation model is a pre-trained model.

[0068] 4) If the system determines that the VCSEL chip under test is qualified, it records the chip identification number; if it is not qualified, it saves the test log and outputs the chip identification number, defect type, defect location, defect morphology, test speed and accuracy.

[0069] 5) The system has finished running.

[0070] The present invention also provides relevant experimental results and data, which serve as evidence that the method described in the present invention has high detection accuracy.

[0071] Experimental results

[0072] 1) Creation of a multimodal defect dataset (corresponding to step S1):

[0073] This invention qualitatively and quantitatively compares the method proposed in step S1 with eight image fusion methods. These eight methods represent typical models in the field of deep learning image fusion. First, these methods are all based on deep learning technology and can effectively handle multi-objective image fusion tasks, such as adaptive extraction of multimodal defect features. Simultaneously, they achieve a good balance between preserving image details and enhancing image contrast, making them suitable for complex image scenes and multimodal object segmentation. Second, these eight methods differ in network architecture, loss function design, and feature fusion strategies, thereby enhancing the comparability of fusion techniques and making the fusion results more convincing. All eight algorithms were trained on the multimodal defect dataset (as described in step S1) built in this invention and fine-tuned according to their respective official training strategies to obtain the best fusion results. This invention selected six groups of defective chips for multimodal defect fusion operations, and the fusion results are as follows: Figure 7 And as shown in Table 1.

[0074] Table 1. Comparison of quality assessment results of the multimodal defect dataset in step S1.

[0075]

[0076] Table 1 and Figure 7 In this context, "Proposed" represents the method for obtaining a multimodal defect dataset designed in step S1 of the present invention. In Table 1, "EN" represents the image information entropy, "SD" represents the image standard deviation, "MI" represents the mutual information between the source image and the fused image, and "Q" represents the information information between the source image and the fused image. abf "" represents the fusion quality; as shown in Table 1, the multimodal defect dataset obtained by using step S1 of the present invention is superior to the current state-of-the-art fusion methods in terms of information entropy, standard deviation, mutual information and fusion quality indicators. This indicates that the multimodal defect dataset obtained by step S1 of the present invention has high image quality and lays a good foundation for improving the accuracy of VCSEL multimodal defect detection in the future.

[0077] 2) VCSEL multimodal defect model training and detection (corresponding to steps S3 and S4):

[0078] Based on the model training settings in step S3, the trained VCSEL multimodal semantic segmentation model was obtained. This invention compared the results of this model with seven high-performance segmentation models (U-Net, UNet++, Attention U-Net, Swin-Unet, SegNet, Fast-SegNet, and DeepLabv3). The detection results are shown in Table 2.

[0079] Table 2 Comparison of detection results of VCSEL multimodal semantic segmentation model in step S4

[0080]

[0081] In Table 2, "Proposed" represents the method of the VCSEL multimodal semantic segmentation model designed in step S4 of this invention. "DSC" is used to evaluate the similarity between predicted and true values, "mIoU" is used to evaluate the overall overlap between predicted and true results, "Recall" is used to measure the model's ability to identify defective positive examples, and "Precision" represents the prediction accuracy. Higher values ​​indicate higher reliability of the model in predicting defective positive examples.

[0082] Figure 8 This is a schematic diagram of the multimodal defect detection results in step S4 of this invention. During the dataset labeling process, this invention categorizes four types of defects—surface contamination, mechanical scratches, epitaxial defects, and dark defects at the light-emitting aperture—into two categories. Surface contamination, mechanical scratches, and epitaxial defects are all classified as surface defects (8), while dark defects at the light-emitting aperture are classified as internal defects (9). Furthermore, to enable the segmentation algorithm to accurately locate dark defects at the light-emitting aperture, this invention additionally labels the light-emitting aperture as category 10. Figure 8 The test results show that the VCSEL multimodal semantic segmentation model can accurately detect surface defect 8 and internal defect 9, and label 10 is the light-emitting hole of the chip.

[0083] The above results demonstrate that the VCSEL defect detection method based on a multimodal semantic segmentation model proposed in this invention significantly outperforms other models in four metrics, achieving a detection accuracy of 95.74%. This detection method can effectively segment dark areas with complex textures and blurred edges. Surface defect information and active area defect information within the dark areas help engineers conduct further failure analysis on failed VCSEL chips, contributing to improved stability and reliability of semiconductor laser chips.

Claims

1. A method for VCSEL defect detection based on a multi-modal semantic segmentation model, characterized in that: Comprising the following steps: S1, acquiring a multi-modal defect data set; S11, collecting visible light images and infrared images of a plurality of VCSEL sample chips (1); the collected VCSEL sample chips (1) include VCSEL chips without defects and VCSEL chips with defects; S12, fusing the visible light image and the infrared image of each VCSEL sample chip (1) respectively through a multi-modal image fusion algorithm to obtain a multi-modal defect fusion image of each VCSEL sample chip (1); S13, labeling the multi-modal defect fusion image of the VCSEL sample chip (1) obtained in step S12, removing the multi-modal defect fusion image without defects, and labeling the multi-modal defect fusion image with defects with defect categories, defect positions and defect morphologies; the labeled multi-modal defect fusion image constitutes a multi-modal defect data set as a chip defect sample; S2, design of a VCSEL multi-modal semantic segmentation model; S21, using a pre-trained convolutional neural network ResNet as an encoder, the encoder comprising a plurality of layers of ordinary convolution, and an SE attention module being added after each layer of ordinary convolution; S22, using a hollow convolution corresponding to each layer of the encoder to constitute a decoder; except for the last layer, each layer of ordinary convolution of the encoder is connected to the hollow convolution of the corresponding layer of the decoder through a skip connection; the last layer of ordinary convolution of the encoder is connected to the last layer of hollow convolution of the decoder through a depth separable convolution, thereby constituting the VCSEL multi-modal semantic segmentation model; S3, training of the VCSEL multi-modal semantic segmentation model The chip defect sample of the multi-modal defect data set obtained in S1 is input into the VCSEL multi-modal semantic segmentation model, and the VCSEL multi-modal semantic segmentation model is trained; the VCSEL multi-modal semantic segmentation model can output a multi-modal defect segmentation result image of each chip defect sample, the multi-modal defect segmentation result image comprising defect categories, defect positions and defect morphologies of the chip defect sample; when the defect categories, defect positions and defect morphologies in the multi-modal defect segmentation result image output by the VCSEL multi-modal semantic segmentation model have an accuracy of 95% or above compared with the chip defect sample, it is considered that the training of the VCSEL multi-modal semantic segmentation model is completed; S4, detection of the VCSEL multi-modal semantic segmentation model Visible light images and infrared images of a VCSEL chip (3) to be detected are collected, a multi-modal defect fusion image of the VCSEL chip (3) to be detected is obtained through a multi-modal image fusion algorithm, then the multi-modal defect fusion image of the VCSEL chip (3) to be detected is input into the VCSEL multi-modal semantic segmentation model trained in S3, and a multi-modal defect segmentation result image comprising defect categories, defect positions and defect morphologies of the VCSEL chip (3) to be detected is output by the VCSEL multi-modal semantic segmentation model.

2. The VCSEL defect detection method based on a multi-modal semantic segmentation model of claim 1, wherein, The defect categories include surface contamination, mechanical scratches, epitaxial defects and light-emitting hole dark defects. 3.The VCSEL defect detection method based on a multi-modal semantic segmentation model of claim 1, wherein, In step S3, the VCSEL multimodal semantic segmentation model is set with the following training parameters and configurations before training: the initial learning rate is set to 1e -4 , a cosine annealing learning rate scheduler is used to dynamically adjust the learning rate, an AdamW optimizer is used in combination with weight decay to prevent overfitting, the batch size is set to 16, the number of training rounds is 150, and the model weight is saved after training is completed.

4. A VCSEL defect detection system based on a multi-modal semantic segmentation model, configured to implement the VCSEL defect detection method based on a multi-modal semantic segmentation model according to any one of claims 1-3, characterized in that, The system comprises a probe station (2) for placing a VCSEL sample chip (1), a conveyor belt (4) for placing a VCSEL chip to be tested (3), an image acquisition system for acquiring images of the VCSEL sample chip (1) and the VCSEL chip to be tested (3), and a computer (5); the image acquisition system comprises an infrared CCD (6) for acquiring infrared images of the VCSEL sample chip (1) and the VCSEL chip to be tested (3), a visible light CCD (7) for acquiring visible light images of the VCSEL sample chip (1) and the VCSEL chip to be tested (3), and visible light sources and infrared light sources for providing light for the VCSEL sample chip (1) and the VCSEL chip to be tested (3); the signal output ends of the infrared CCD (6) and the visible light CCD (7) are connected to the computer (5), the probe station (2), the conveyor belt (4), the visible light sources, and the infrared light sources are electrically connected to the computer (5) and are controlled by the computer (5) to move; the computer (5) is built-in with a multi-modal image fusion algorithm and a VCSEL multi-modal semantic segmentation model.

5. The multi-modal semantic segmentation model based VCSEL defect detection system of claim 4, wherein, Before the defect detection system detects, the running speed of the conveyor belt (4), the intensity of the visible light sources and the infrared light sources, the shooting angle of the infrared CCD (6) and the visible light CCD (7) are set, and an identification number is assigned to each VCSEL chip to be tested (3); then the automatic detection is started, the multi-modal image fusion algorithm built-in the computer (5) fuses the visible light images and the infrared images of the VCSEL chip to be tested (3) acquired, and then the VCSEL multi-modal semantic segmentation model built-in the computer (5) and trained detects the multi-modal defect fusion image of the VCSEL chip to be tested (3); if the multi-modal defect fusion image of the VCSEL chip to be tested (3) has no defect, the computer (5) outputs the identification number of the VCSEL chip to be tested (3) and marks it as qualified; if the multi-modal defect fusion image of the VCSEL chip to be tested (3) has a defect, the computer outputs the identification number, the defect category, the defect position, and the defect morphology of the VCSEL chip to be tested (3) and marks it as unqualified.

Images