VCSEL chip appearance defect detection system and method based on visual recognition

Abstract

The present invention discloses a VCSEL chip appearance defect detection system and method based on visual recognition, belonging to the field of appearance defect detection technology. The present invention collects chip images, performs active area detection to obtain active area parameters, associates the chip images with the active area parameters to obtain a chip appearance detection data set; trains a convolutional neural network model, inputs the chip image to be detected into the trained convolutional neural network model to obtain an active area anomaly data set; integrates the anomaly types and anomaly locations in the active area anomaly data set for cluster analysis, and calculates the mean and standard deviation of the parameters within each cluster based on the active area parameters; treats each cluster as a node, and determines edges based on the similarity between clusters; calculates the degree of each node and identifies the subgraph structure in the graph; compares the subgraph structures at different time points to determine whether the anomaly data in the active area anomaly data set is reversible.

Description

Technical Field

[0001] The present invention relates to the technical field of appearance defect detection, and in particular to a VCSEL chip appearance defect detection system and method based on visual recognition. Background Art

[0002] With the rapid development of optical communications, LiDAR, and other fields, the quality and reliability of vertical-cavity surface-emitting laser (VCSEL) chips, as core components, directly impact the performance of the entire system. During VCSEL chip production, due to the complexity of the manufacturing process and the sensitivity of material properties, various anomalies such as scratches, holes, impurities, and lattice distortion are prone to appear in the active region. These anomalies can cause key parameters such as the chip's emission wavelength, optical power, and quantum efficiency to deviate from their normal ranges, thereby affecting the chip's operating performance and service life.

[0003] Traditional chip inspection methods rely heavily on manual visual inspection or simple optical imaging techniques, making it difficult to detect subtle anomalies in the chip's active area. Complex defect types and locations are prone to missed or false detections. Existing technologies can only identify the presence of an anomaly, but are limited in determining whether the anomaly data is reversible or distinguishing between reversible and irreversible failure modes. Faced with the massive amount of chip inspection data, existing technologies lack effective methods for data integration and analysis. Summary of the Invention

[0004] The purpose of the present invention is to provide a VCSEL chip appearance defect detection system and method based on visual recognition to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a method for detecting appearance defects of a VCSEL chip based on visual recognition, the method comprising the following steps:

[0007] Collect chip images, perform active area detection to obtain active area parameters, and associate the chip images and active area parameters to obtain a chip appearance detection dataset;

[0008] Based on the chip appearance inspection dataset, a convolutional neural network model is trained. The chip image to be inspected is input into the trained convolutional neural network model, and the image feature vector is extracted and classified and predicted to obtain the active area anomaly dataset.

[0009] The anomaly types and locations in the active area anomaly data set are integrated for cluster analysis. Combined with the active area parameters, the mean and standard deviation of the parameters within each cluster are calculated.

[0010] Treat each cluster as a node and determine the edge based on the similarity between clusters; calculate the degree of each node and identify the subgraph structure in the graph;

[0011] The subgraph structures at different time points are compared, and the node attributes and connection relationships in the graph are combined to determine whether the abnormal data in the active area abnormal data set are reversible.

[0012] In conjunction with the first aspect, in a first implementation of the first aspect of the present application, the step of acquiring chip images and performing active area detection to obtain active area parameters includes:

[0013] Control the stage so that the chip is located in the center of the camera's field of view. Take a vertical image of the entire chip. While keeping the light source and camera positions unchanged, take a close-up shot of a part of the chip.

[0014] Use an optical spectrum analyzer, aim the probe at the chip's active area, and set the wavelength scanning range. Use a calibrated optical power meter to measure the optical power under the same current conditions as the measured luminescence wavelength. Calculate the quantum efficiency based on the measured optical power and injection current value. Use a microscope to set the magnification, image the active area, select the edge of the active area on the image, and measure its length and width. Use an energy spectrum analyzer to inspect the active area, perform multi-point scanning of the active area, and obtain material composition and impurity concentration data.

[0015] In combination with the first aspect, in a second implementation of the first aspect of the present application, associating the chip image with the active area parameters to obtain a chip appearance inspection dataset includes:

[0016] The collected chip images are numbered, and the measured active area parameters are recorded in the measurement order and associated with the corresponding chip samples. The chip image number and the ID number in the active area parameter record table are used as unique identifiers to establish an association between the two and supplement the associated information; the associated information includes the acquisition time and measurement environment, and the chip appearance inspection data set is obtained.

[0017] In combination with the first aspect, in a third implementation of the first aspect of the present application, the training of the convolutional neural network model based on the chip appearance detection dataset includes:

[0018] The chip appearance inspection dataset is divided into a training set, a validation set, and a test set in proportion. In the training set, image data and their corresponding active area abnormality labels are sorted. ResNet50 is selected as the base model of the neural network model and the base model is adjusted according to the specific requirements of chip appearance inspection. Training parameters are set, and the cross entropy loss function is used to measure the difference between the model prediction value and the true label. Adam is selected as the optimizer.

[0019] The training set data is input in batches into the built convolutional neural network model. The model calculates the prediction results based on forward propagation, calculates the gradient of the loss function to the model parameters through back propagation, and uses the optimizer to update the parameters. After training is completed, the model is evaluated using the test set, and the model is optimized based on the evaluation results.

[0020] In combination with the first aspect, in a fourth implementation of the first aspect of the present application, the chip image to be detected is input into a trained convolutional neural network model, image feature vectors are extracted, and classification prediction is performed to obtain an active area abnormality dataset, including:

[0021] The chip image to be tested is preprocessed and input into the loaded convolutional neural network model. Through the forward propagation process, the feature vector of the image is extracted through the convolution layer, pooling layer and fully connected layer. Based on the extracted feature vector, the model performs classification prediction through the softmax classifier to determine whether there is an abnormality in the active area and obtain a prediction result. According to the actual application requirements and the model performance evaluation results, the threshold for abnormality judgment is determined. Based on the prediction result and the set threshold, the active area abnormality dataset is compiled and generated. The active area abnormality dataset includes the chip image identification, abnormality type, abnormality location and abnormality judgment threshold.

[0022] In combination with the first aspect, in a fifth implementation of the first aspect of the present application, the cluster analysis is performed by integrating the anomaly type and anomaly location in the active area anomaly data set, and the mean and standard deviation of the parameters within each cluster are calculated in combination with the active area parameters, including:

[0023] The anomaly type is converted into a numerical form; the DBSCAN clustering algorithm is selected for cluster analysis, and the neighborhood radius eps and the minimum sample number minPts are set. The neighborhood radius represents the neighborhood range of a data point, and the minimum sample number represents the number of data points required to become a core point in a neighborhood. According to the set eps and minPts, each data point in the data set is traversed; for a data point, when the number of data points contained in a neighborhood with a radius of eps centered on the data point is greater than or equal to minPts, the data point is determined to be a core point; in the active area anomaly data, the core point represents the center of data aggregation with similar anomaly characteristics;

[0024] Starting from a core point, all density-connected data points in its neighborhood are merged into a cluster. The other core points in the cluster are traversed, and the undivided data points in its neighborhood are added to the cluster, and the cluster is continuously expanded. In the process of expanding the cluster, by continuously merging density-connected data points, data with similar anomaly types, locations, and active area parameter characteristics are clustered to form different clusters. Data points that are not core points and are not density-connected to any core points are marked as noise points.

[0025] For each cluster, the active area parameters are extracted from the data points it contains, and the mean and standard deviation of the parameters within each cluster are calculated.

[0026] In combination with the first aspect, in a sixth implementation of the first aspect of the present application, treating each cluster as a node and determining edges based on similarities between clusters includes:

[0027] Each cluster is regarded as a node in the graph structure. Each node contains the cluster information of the cluster and the mean and standard deviation of the active area parameters. The cluster information includes the distribution of anomaly types, the central coordinates of the anomaly positions, and the degree of dispersion. The Euclidean distance is used to measure the similarity between clusters. A similarity threshold is set. When the Euclidean distance is less than the similarity threshold, an edge is established between the corresponding nodes. The weight of the edge is set to the inverse of the similarity measure value, reflecting the closeness between the clusters.

[0028] In combination with the first aspect, in a seventh implementation of the first aspect of the present application, calculating the degree of each node and identifying the subgraph structure in the graph includes:

[0029] The degree of the node reflects the degree of association between a cluster and other clusters. Each node in the graph structure is traversed and the number of edges connected to the node is counted to obtain the degree of the node.

[0030] Use the depth-first search algorithm to identify connected components in the graph, where each connected component is a subgraph. The method is as follows:

[0031] Starting from any unvisited node in the graph, mark the node as visited, and recursively visit all its unvisited adjacent nodes until all reachable nodes have been visited; when all adjacent nodes of a node have been visited, backtrack to the previous node to continue searching; repeat the process until all nodes have been visited, thereby determining all connected components.

[0032] In conjunction with the first aspect, in an eighth implementation of the first aspect of the present application, comparing the subgraph structures at different time points and combining the node attributes and connection relationships in the graph to determine whether the abnormal data in the active area abnormal data set is reversible includes:

[0033] The active area anomaly datasets at different time points are collected, and a graph structure is constructed for the active area anomaly dataset at each time point. The Jaccard similarity is used to calculate the anomaly type similarity of the nodes in the subgraphs at different time points. For the first node v1 and the second node v2 in the first subgraph S1 and the second subgraph S2, the anomaly type set of v1 is A1, and the anomaly type set of v2 is A2. The Jaccard similarity is used to calculate the anomaly type similarity of the nodes in the subgraphs at different time points. J type The closer it is to 1, the more similar the anomaly types of the two nodes are;

[0034] The weighted Euclidean distance is used to calculate the similarity of active area parameters. For the active area parameter mean vector of v1 and the active region parameter mean vector of v2 and the weight vector for each parameter The weight is determined according to the importance of the parameter to the chip performance, and the weighted Euclidean distance The smaller the distance, the higher the parameter similarity, where n is the number of active area parameters, p 11 , p 12 ,…,p 1n represent From the 1st element to the nth element, p 21 , p 22 ,…,p 2n represent From the 1st element to the nth element, w i is the weight vector The i-th element in represents the weight of the i-th active region parameter, p 1i is the mean vector of active region parameters The i-th element in represents The mean value of the parameters of the i-th active region, p 2i is the mean vector of active region parameters The i-th element in represents The mean value of the parameters of the i-th active region in ;

[0035] Calculate the similarity of the connection relationship between corresponding nodes in the subgraphs at different time points. Suppose the adjacent node sets of the first node v1 and the second node v2 in the first subgraph S1 and the second subgraph S2 are N1 and N2 respectively, and the connection relationship similarity The closer the value is to 1, the more similar the connection relationship is;

[0036] The comprehensive similarity of the subgraph is obtained by combining the node attribute similarity and the connection relationship similarity. The method is as follows: By weighted summation, let the node attribute similarity weight be w attr , the connection relationship similarity weight is w conn , comprehensive similarity Stotal =w attr ×(αJ type +(1-α)×(1-d param ))+w conn J conn , where α is the weight distribution coefficient of anomaly type similarity and active area parameter similarity;

[0037] A reversibility judgment threshold τ is set. When the comprehensive similarity of subgraphs at different time points is greater than τ and in the subgraphs at subsequent time points, the node attributes corresponding to the abnormal data change towards the normal state and the connection relationship tends to be stable, the abnormal data is judged to be reversible; otherwise, it is judged to be irreversible.

[0038] Secondly, the VCSEL chip appearance defect detection system based on visual recognition includes:

[0039] Data acquisition and association module: includes: image acquisition unit, active area parameter detection unit and data association unit; wherein, the image acquisition unit acquires chip images, the active area parameter detection unit performs active area detection to obtain active area parameters, and the data association unit associates the chip images and active area parameters to obtain a chip appearance detection data set;

[0040] Model training and prediction module: includes: model training unit and prediction unit; the model training unit trains the convolutional neural network model based on the chip appearance inspection dataset; the prediction unit inputs the chip image to be inspected into the trained convolutional neural network model, extracts the image feature vector, performs classification prediction, and obtains the active area anomaly dataset;

[0041] Cluster analysis module: includes: a cluster execution unit and a parameter calculation unit; wherein the cluster execution unit integrates the anomaly type and anomaly location in the active area anomaly data set for cluster analysis, and the parameter calculation unit combines the active area parameters to calculate the mean and standard deviation of the parameters within each cluster;

[0042] Graph structure construction module: includes: node definition unit, edge determination unit, node degree calculation unit and subgraph identification unit; wherein, the node definition unit regards each cluster as a node, the edge determination unit determines the edge based on the similarity between clusters; the node degree calculation unit calculates the degree of each node, and the subgraph identification unit identifies the subgraph structure in the graph;

[0043] Reversibility judgment module: includes: a subgraph comparison unit and a reversibility judgment unit; wherein, the subgraph comparison unit compares the subgraph structures at different time points, and the reversibility judgment unit combines the node attributes and connection relationships in the graph to judge whether the abnormal data in the active area abnormal data set is reversible.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The present invention uses high-resolution image acquisition equipment and advanced convolutional neural network models to accurately extract the feature vectors of chip images, achieve accurate classification and prediction of active area anomalies, greatly improve detection accuracy, and effectively reduce missed detections and false detections.

[0046] 2. Based on cluster analysis and graph structure construction, combined with subgraph comparison at different time points, the present invention can accurately determine the reversibility of abnormal data in the active area and effectively distinguish between reversible and irreversible failure modes; it provides clear guidance for chip quality assessment and subsequent processing, reducing production risks and costs.

[0047] 3. The present invention realizes the effective association between chip images and active area parameters, calculates the mean and standard deviation of the parameters in each cluster through cluster analysis, and deeply explores the intrinsic connection between data; it can grasp the quality status of the chip as a whole, provide strong data support for the optimization of production process, and help improve the production quality and efficiency of the chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Schematic diagram of the steps of the VCSEL chip appearance defect detection method based on visual recognition of the present invention;

[0049] Figure 2 This is a system structure diagram of the VCSEL chip appearance defect detection system based on visual recognition of the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution:

[0052] like Figure 1 The present invention provides a method for detecting VCSEL chip appearance defects based on visual recognition, as shown in a schematic diagram of the steps of the method. The method includes the following steps:

[0053] Step S100: collecting chip images, performing active area detection to obtain active area parameters, and correlating the chip images and active area parameters to obtain a chip appearance detection data set;

[0054] Specifically, the stage is controlled to position the chip in the center of the camera's field of view, and the entire chip image is captured from a vertical angle. While keeping the light source and camera positions unchanged, a close-up shot of a part of the chip is taken.

[0055] Use an optical spectrum analyzer, aim the probe at the chip's active area, and set the wavelength scanning range. Use a calibrated optical power meter to measure the optical power under the same current conditions as the measured luminescence wavelength. Calculate the quantum efficiency based on the measured optical power and injection current value. Use a microscope to set the magnification, image the active area, select the edge of the active area on the image, and measure its length and width. Use an energy spectrum analyzer to inspect the active area, perform multi-point scanning of the active area, and obtain material composition and impurity concentration data.

[0056] Furthermore, the collected chip images are numbered, and the measured active area parameters are recorded in the measurement order and associated with the corresponding chip samples; the chip image number and the ID number in the active area parameter record table are used as unique identifiers to establish an association relationship between the two, and supplement the associated information; the associated information includes the acquisition time and measurement environment, and a chip appearance inspection data set is obtained.

[0057] In a specific embodiment, 100 VCSEL chips were selected as experimental samples, and equipment such as a high-resolution industrial camera with a resolution of 5 million pixels, a translation stage, a ring-shaped LED light source, a spectrum analyzer with a wavelength scanning range of 800-1000nm and a resolution of 0.1nm, a calibrated optical power meter, a high-precision microscope with a magnification of 500-1000 times, an energy spectrum analyzer, and a high-precision constant current source were prepared.

[0058] Control the stage to position the chip in the center of the camera's field of view. Capture an image of the entire chip from a vertical angle, with a resolution of 2048 × 2048 pixels. Maintaining the same light source and camera position, capture close-ups of the chip, with each close-up overlapping by 10%-20%. Number the captured chip images chronologically, such as "20241001-001," "20241001-002," and so on.

[0059] Active area parameter detection: Use an optical spectrum analyzer, aim the probe at the chip's active area, set the wavelength scan range to 800-1000nm, and the scan resolution to 0.1nm. Measure the emission wavelength while maintaining a stable operating current of 10mA. For example, for the chip number "20241001-001," the measured emission wavelength is 850.5nm. Use a calibrated optical power meter to measure the optical power at the same 10mA current used to measure the emission wavelength. For example, the optical power of the chip "20241001-001" is 5mW. Calculate the quantum efficiency based on the measured optical power and injection current values. With an injection current of 10mA and an optical power of 5mW, the calculated quantum efficiency is 31.25%. Use a microscope with the magnification set to 800x to image the active area. Select the edge of the active area on the image and measure its length and width with an accuracy of 0.1μm. For example, the active area of ​​the "20241001-001" chip is 5.2μm long and 4.8μm wide. The active area is inspected using an energy spectrum analyzer, scanning five points within the active area to obtain material composition and impurity concentration data. For example, in the "20241001-001" chip, the primary material is GaAs, with an impurity concentration of 0.005at%. The measured active area parameters are recorded in the order in which they were measured, associated with the corresponding chip samples, and recorded in an Excel spreadsheet. Each row represents the parameter data for a single chip, including fields such as chip ID, emission wavelength, optical power, quantum efficiency, active area length, active area width, material composition, and impurity concentration.

[0060] Using the chip image number and the "chip ID" in the active area parameter record table as unique identifiers, a correlation is established between the two. Using the Python pandas library, the image number data and parameter record data are merged according to the "chip ID." Supplementary information is provided, including acquisition time and measurement environment. The resulting chip appearance inspection dataset contains chip image information, active area parameters, and related information.

[0061] Step S200: Based on the chip appearance inspection dataset, a convolutional neural network model is trained, the chip image to be inspected is input into the trained convolutional neural network model, the image feature vector is extracted, and classification prediction is performed to obtain an active area abnormality dataset;

[0062] Specifically, the chip appearance inspection dataset is divided into a training set, a validation set, and a test set in proportion. In the training set, image data and its corresponding active area anomaly labels are organized. ResNet50 is selected as the base model of the neural network model and the base model is adjusted according to the specific requirements of chip appearance inspection. Training parameters are set, and the cross-entropy loss function is used to measure the difference between the model prediction value and the true label. Adam is selected as the optimizer.

[0063] The training set data is input in batches into the built convolutional neural network model. The model calculates the prediction results based on forward propagation, calculates the gradient of the loss function to the model parameters through back propagation, and uses the optimizer to update the parameters. After training is completed, the model is evaluated using the test set, and the model is optimized based on the evaluation results.

[0064] Furthermore, the chip image to be detected is preprocessed and input into the loaded convolutional neural network model. Through the forward propagation process, the feature vector of the image is extracted through the convolution layer, pooling layer and fully connected layer. Based on the extracted feature vector, the model performs classification prediction through the softmax classifier to determine whether there is an abnormality in the active area and obtain a prediction result. According to the actual application requirements and the model performance evaluation results, the threshold for abnormality judgment is determined. According to the prediction results and the set threshold, the active area abnormality data set is compiled and generated. The active area abnormality data set includes the chip image identification, abnormality type, abnormality location and abnormality judgment threshold.

[0065] In one specific embodiment, the work of constructing a chip appearance inspection dataset for 100 VCSEL chips, completed in the previous phase, was continued. The chip appearance inspection dataset was divided into training, validation, and test sets in a 70% / 15% / 15% ratio. The training set contained data from 70 chips, while the validation and test sets each contained data from 15 chips. Within the training set, image data and corresponding active area anomaly labels, such as "scratch," "hole," and "normal," were collated and converted into numerical labels, with "scratch" assigned a value of 1, "hole" assigned a value of 2, and "normal" assigned a value of 0. ResNet50 was selected as the base neural network model. Based on the specific requirements of chip appearance inspection, the base model was adjusted as follows: the input layer was modified to accommodate the chip image size of 2048×2048 pixels and three channels (RGB); the number of neurons in the output layer was adjusted to 3 (corresponding to the three states of normal, scratch, and hole), and a softmax activation function was used. The learning rate was set to 0.001, the batch size to 32, and the number of training epochs to 50. The cross entropy loss function is used to measure the difference between the model prediction value and the true label. The optimizer is Adam, with default parameters beta1 = 0.9, beta2 = 0.999, and epsilon = 1e-8.

[0066] The training set data was input into the pre-built convolutional neural network model in batches. During training, the model calculated predictions using forward propagation. For example, during the 10th round of training, the model output predictions for a batch of 32 training samples were compared with the true labels to calculate the cross-entropy loss. The gradients of the loss function with respect to the model parameters were calculated using backpropagation, and the parameters were updated using the Adam optimizer. After every five rounds of training, the model was evaluated on the validation set, and the validation set loss and accuracy were recorded. By the 30th round of training, the validation set accuracy reached 80%, and the loss was reduced to approximately 0.5. After training, the model was evaluated on the test set. On the test set, the model achieved an accuracy of 85%, a recall of 82%, and an F1 score of 0.83. The evaluation results showed that the model's accuracy for identifying "scratches" anomalies was relatively low, at 75%. To optimize the model, data augmentation methods were used, such as rotation and flipping, to increase data diversity. After retraining the model, on the test set, the model's recognition accuracy for "scratch" anomalies increased to 80%, the overall accuracy reached 88%, the recall rate was 85%, and the F1 value increased to 0.86.

[0067] The chip image to be tested is preprocessed, including normalization (scaling the pixel values ​​to the range of 0-1) and resizing to 224×224 pixels (to meet the model input requirements). The preprocessed image is input into the loaded convolutional neural network model, and through the forward propagation process, the feature vector of the image is extracted through the convolution layer, pooling layer and fully connected layer. For example, for the chip image to be tested numbered "20241002-001", the model extracts a 512-dimensional feature vector. Based on the extracted feature vector, the model performs classification prediction through the softmax classifier to determine whether there is an abnormality in the active area. The probability of the predicted result being "normal" is 0.9, the probability of "scratch" is 0.05, and the probability of "hole" is 0.05.

[0068] Based on actual application requirements and model performance evaluation results, the anomaly detection threshold was determined to be 0.5. Based on the prediction results and the set threshold, an active area anomaly dataset was generated. For example, for the chip image numbered "20241002-001" mentioned above, its active area anomaly dataset would be recorded as follows: Chip Image ID: "20241002-001", Anomaly Type: "Normal", Anomaly Location: None (because it was determined to be normal), and Anomaly Detection Threshold: 0.5.

[0069] Step S300: integrating the anomaly types and locations in the active area anomaly data set for cluster analysis, and calculating the mean and standard deviation of the parameters within each cluster in combination with the active area parameters;

[0070] Specifically, for the anomaly type, the data is converted into a numerical form; the DBSCAN clustering algorithm is selected for cluster analysis, and the neighborhood radius eps and the minimum sample number minPts are set. The neighborhood radius represents the neighborhood range of a data point, and the minimum sample number represents the number of data points required to become a core point in a neighborhood; according to the set eps and minPts, each data point in the data set is traversed; for a data point, when the number of data points contained in a neighborhood with a radius of eps centered on the data point is greater than or equal to minPts, the data point is determined to be a core point; in the active area anomaly data, the core point represents the center of data aggregation with similar anomaly characteristics;

[0071] Starting from a core point, all density-connected data points in its neighborhood are merged into a cluster. The other core points in the cluster are traversed, and the undivided data points in its neighborhood are added to the cluster, and the cluster is continuously expanded. In the process of expanding the cluster, by continuously merging density-connected data points, data with similar anomaly types, locations, and active area parameter characteristics are clustered to form different clusters. Data points that are not core points and are not density-connected to any core points are marked as noise points.

[0072] For each cluster, the active area parameters are extracted from the data points it contains, and the mean and standard deviation of the parameters within each cluster are calculated.

[0073] In one specific embodiment, the DBSCAN clustering algorithm was used for cluster analysis. Through multiple experiments and preliminary observations of the data, a neighborhood radius of eps was set to 0.5, and a minimum sample number of minPts was set to 5. The neighborhood radius here represents a circular area with a radius of 0.5 centered on a data point in the data space; the minimum sample number indicates that at least five data points must exist within the neighborhood for the data point to be considered a core point.

[0074] Based on the set eps and minPts, each data point in the dataset is traversed. For example, for data point A, if the number of data points within a neighborhood with a radius of 0.5 centered on A is greater than or equal to 5, A is considered a core point. In the dataset, if data points B, C, D, E, and F are within A's neighborhood and meet the minimum sample count requirement, A becomes a core point.

[0075] Starting with core point A, all density-connected data points B, C, D, E, and F within its neighborhood are merged into a single cluster. Next, the cluster is expanded by traversing other core points within the cluster. For example, if B is also a core point, unclassified data points in B's neighborhood are added to the cluster. During this expansion process, data with similar anomaly types, locations, and active area parameter characteristics are clustered together. For example, data points with a "scratches" anomaly type and close locations, along with similar active area optical power parameters, are grouped together.

[0076] For data points that are neither core points nor densely connected to any core points, they are marked as noise points. Data point G is not in the neighborhood of any core point, and G is marked as a noise point.

[0077] Cluster analysis yielded multiple clusters. For each cluster, active region parameters, such as wavelength and optical power, were extracted from the data points within it. For example, a cluster contained 10 data points with optical power values ​​of 4.5mW, 5.0mW, 4.8mW, 5.2mW, 4.9mW, 5.1mW, 4.7mW, 5.3mW, 4.6mW, and 5.0mW, respectively.

[0078] Sum: 4.5 + 5.0 + 4.8 + 5.2 + 4.9 + 5.1 + 4.7 + 5.3 + 4.6 + 5.0 = 49.1; Calculate the average: 49.1 / 10 = 4.91 mW;

[0079] Calculate the difference between each data point and the mean: 4.5-4.91=-0.41, 5.0-4.91=0.09, ..., 5.0-4.91=0.09; calculate the square of the difference, the result is as follows: (-0.41) 2 =0.1681, (0.09) 2 =0.0081, ..., (0.09) 2 =0.0081; Sum of squared deviations: 0.1681+0.0081+0.0121+0.0841+0.0001+0.0361+0.0441+0.1521+0.0961+0.0081=0.609. Calculate the standard deviation: Similarly, the means and standard deviations of other active area parameters within each cluster are calculated.

[0080] Step S400: taking each cluster as a node, determining edges based on the similarity between clusters; calculating the degree of each node, and identifying the subgraph structure in the graph;

[0081] Specifically, each cluster is regarded as a node in the graph structure. Each node contains the cluster information of the cluster and the mean and standard deviation of the active area parameters. The cluster information includes the distribution of anomaly types, the central coordinates of the anomaly positions, and the degree of dispersion. The Euclidean distance is used to measure the similarity between clusters, and a similarity threshold is set. When the Euclidean distance is less than the similarity threshold, an edge is established between the corresponding nodes. The weight of the edge is set to the inverse of the similarity measure value, reflecting the closeness between the clusters.

[0082] Furthermore, the degree of the node reflects the closeness of the association between a cluster and other clusters. Each node in the graph structure is traversed and the number of edges connected to the node is counted to obtain the degree of the node.

[0083] Use the depth-first search algorithm to identify connected components in the graph, where each connected component is a subgraph. The method is as follows:

[0084] Starting from any unvisited node in the graph, mark the node as visited, and recursively visit all its unvisited adjacent nodes until all reachable nodes have been visited; when all adjacent nodes of a node have been visited, backtrack to the previous node to continue searching; repeat the process until all nodes have been visited, thereby determining all connected components.

[0085] In a specific embodiment, the clusters obtained by the cluster analysis in the previous stage are used, and each cluster contains the distribution of abnormal types, the central coordinates and the degree of dispersion of the abnormal position, as well as the mean and standard deviation of the active area parameters. After cluster analysis, a total of 5 clusters are obtained, which are recorded as C1, C2, C3, C4, and C5 respectively. Taking the two active area parameters of optical power and luminous wavelength as examples, the Euclidean distance between clusters is calculated. The average optical power of C1 is 4.91mW, and the average luminous wavelength is 850nm; the average optical power of C2 is 5.5mW, and the average luminous wavelength is 860nm. The Euclidean distance is calculated to obtain the distance matrix as shown in Table 1:

[0086] Table 1 Distance matrix table

[0087] C1 C2 C3 C4 C5 C1 0 10.02 12.5 8.3 15.2 C2 10.02 0 9.1 11.7 13.4 C3 12.5 9.1 0 10.05 11.2 C4 8.3 11.7 10.5 0 9.8 C5 15.2 13.4 11.2 9.8 0

[0088] Set the similarity threshold to 10. When the Euclidean distance is less than this threshold, an edge is established between the corresponding nodes, and the edge weight is set to the inverse of the similarity metric (Euclidean distance). For example, the Euclidean distance between C1 and C4 is 8.3, which is less than the threshold of 10, so an edge is established between nodes C1 and C4 with a weight of 1 / 8.3 ≈ 0.12.

[0089] After judgment, the established edges and weights are as follows: C1-C4: weight 0.12, C2-C3: weight 0.11, C2-C5: weight 0.07, C3-C5: weight 0.09, C4-C5: weight 0.10.

[0090] Traverse each node in the graph structure and count the number of edges connected to it to obtain the node's degree. Node C1: Connected to C4, with a degree of 1. Node C2: Connected to C3 and C5, with a degree of 2. Node C3: Connected to C2 and C5, with a degree of 2. Node C4: Connected to C1 and C5, with a degree of 2. Node C5: Connected to C2, C3, and C4, with a degree of 3.

[0091] Use a depth-first search algorithm to identify connected components in a graph.

[0092] Starting from node C1, mark C1 as visited, visit its adjacent node C4, mark C4 as visited. At this point, C4 has no other unvisited adjacent nodes, and backtracking to C1, C1 also has no other unvisited adjacent nodes.

[0093] Next, we start from the unvisited node C2, mark C2 as visited, visit its adjacent node C3, mark C3 as visited, and visit C3's adjacent node C5, mark C5 as visited. At this point, C5 has no other unvisited adjacent nodes. We trace back to C3, and C3 has no other unvisited adjacent nodes. We trace back to C2, and C2 has no other unvisited adjacent nodes.

[0094] Finally, it is determined that there are two connected components, namely two subgraphs: Figure 1 :Contains nodes C1 and C4. Figure 2 : Contains nodes C2, C3, and C5.

[0095] Step S500: comparing the subgraph structures at different time points, and combining the node attributes and connection relationships in the graph to determine whether the abnormal data in the active area abnormal data set is reversible.

[0096] Specifically, the active area anomaly data sets at different time points are collected, and a graph structure is constructed for the active area anomaly data sets at each time point; the Jaccard similarity is used to calculate the anomaly type similarity of the nodes in the subgraphs at different time points. For the first node v1 and the second node v2 in the first subgraph S1 and the second subgraph S2, the anomaly type set of v1 is A1, and the anomaly type set of v2 is A2. The Jaccard similarity is J type The closer it is to 1, the more similar the anomaly types of the two nodes are;

[0097] The weighted Euclidean distance is used to calculate the similarity of active area parameters. For the active area parameter mean vector of v1 and the active region parameter mean vector of v2 and the weight vector for each parameter The weight is determined according to the importance of the parameter to the chip performance, and the weighted Euclidean distance The smaller the distance, the higher the parameter similarity, where n is the number of active area parameters, p 11 , p 12 ,…,p 1n represent From the 1st element to the nth element, p 21 , p 22 ,…,p 2n represent From the 1st element to the nth element, w i is the weight vector The i-th element in represents the weight of the i-th active region parameter, p 1i is the mean vector of active region parameters The i-th element in represents The mean value of the parameters of the i-th active region, p 2i is the mean vector of active region parameters The i-th element in represents The mean value of the parameters of the i-th active region in ;

[0098] Calculate the similarity of the connection relationship between corresponding nodes in the subgraphs at different time points. Suppose the adjacent node sets of the first node v1 and the second node v2 in the first subgraph S1 and the second subgraph S2 are N1 and N2 respectively, and the connection relationship similarity The closer the value is to 1, the more similar the connection relationship is;

[0099] The comprehensive similarity of the subgraph is obtained by combining the node attribute similarity and the connection relationship similarity. The method is as follows: By weighted summation, let the node attribute similarity weight be w attr , the connection relationship similarity weight is w conn , comprehensive similarity S total =w attr ×(αJ type +(1-α)×(1-d param ))+w conn J conn , where α is the weight distribution coefficient of anomaly type similarity and active area parameter similarity;

[0100] A reversibility judgment threshold τ is set. When the comprehensive similarity of subgraphs at different time points is greater than τ and in the subgraphs at subsequent time points, the node attributes corresponding to the abnormal data change towards the normal state and the connection relationship tends to be stable, the abnormal data is judged to be reversible; otherwise, it is judged to be irreversible.

[0101] In a specific embodiment, an abnormal data set of the active area at two different time points t1 and t2 is collected. At time t1, after cluster analysis and graph structure construction, two subgraphs as described above are obtained: Figure 1 :Contains nodes C1 and C4. Figure 2 : Contains nodes C2, C3, and C5. At time t2, the same cluster analysis and graph structure construction yield two subgraphs: Subgraph 3: Contains nodes C1' and C4' (corresponding to C1 and C4 at time t1, but their attributes may have changed). Subgraph 4: Contains nodes C2', C3', and C5' (corresponding to C2, C3, and C5 at time t1, but their attributes may have changed).

[0102] t1 time Figure 1 The anomaly type set A1 of the node C1 is {scratch}, and the anomaly type set A2 of the corresponding node C1' in the subgraph 3 at time t2 is {scratch}.

[0103] According to the Jaccard similarity formula Here |A1∩A2|=1, |A1∪A2|=1, so J type =1.

[0104] Assume that the mean vector of the active area parameters of node C1 at time t1 is where p 11 The average optical power is 4.91mW, p 12 The average value of the emission wavelength is 850nm; the average value vector of the active area parameter corresponding to the node C1' at time t2 is where p 21 The average optical power is 5.0mW, p 22 The average emission wavelength is 852nm.

[0105] Weight vector of optical power and emission wavelength According to the weighted Euclidean distance formula, where n = 2, the calculation process is as follows:

[0106]

[0107] The active area parameter similarity is 1-d param The larger the value, the more similar it is, that is, 1-1.26=-0.26.

[0108] t1 time Figure 1 The adjacent node set N1 of the node C1 is {C4}, and the adjacent node set N2 of the corresponding node C1' in the subgraph 3 at time t2 is {C4'}.

[0109] According to the connection relationship similarity formula Here |N1∩N2|=1, |N1∪N2|=1, so J conn =1.

[0110] Assume the node attribute similarity weight w attr =0.6, connection relationship similarity weight w conn =0.4, and the weight distribution coefficient of anomaly type similarity and active area parameter similarity is α=0.5.

[0111] The node attribute similarity is:

[0112] αJ type +(1-α)×(1-d param )=0.5×1+(1-0.5)×(-0.26)=0.5-0.13=0.37.

[0113] The comprehensive similarity is:

[0114] S total =w attr ×0.37+w conn J conn =0.6×0.37+0.4×1=0.222+0.4=0.622.

[0115] Set the reversibility judgment threshold τ = 0.7. total =0.622<0.7, so the anomaly is initially determined to be irreversible. However, further observation is needed in the subgraphs at subsequent time points to see whether the node attributes corresponding to the anomaly change toward normality and whether the connection relationships stabilize. Observation shows that at subsequent time points, the anomaly type for node C1 remains a scratch, the active area parameters show no clear trend toward normality, and the connection relationships do not become more stable. Therefore, the anomaly is ultimately determined to be irreversible.

[0116] like Figure 2 The present invention provides a system structure diagram of a VCSEL chip appearance defect detection system based on visual recognition, which includes:

[0117] Data acquisition and association module: includes: image acquisition unit, active area parameter detection unit and data association unit; wherein, the image acquisition unit acquires chip images, the active area parameter detection unit performs active area detection to obtain active area parameters, and the data association unit associates the chip images and active area parameters to obtain a chip appearance detection data set;

[0118] Model training and prediction module: includes: model training unit and prediction unit; the model training unit trains the convolutional neural network model based on the chip appearance inspection dataset; the prediction unit inputs the chip image to be inspected into the trained convolutional neural network model, extracts the image feature vector, performs classification prediction, and obtains the active area anomaly dataset;

[0119] Cluster analysis module: includes: a cluster execution unit and a parameter calculation unit; wherein the cluster execution unit integrates the anomaly type and anomaly location in the active area anomaly data set for cluster analysis, and the parameter calculation unit combines the active area parameters to calculate the mean and standard deviation of the parameters within each cluster;

[0120] Graph structure construction module: includes: node definition unit, edge determination unit, node degree calculation unit and subgraph identification unit; wherein, the node definition unit regards each cluster as a node, the edge determination unit determines the edge based on the similarity between clusters; the node degree calculation unit calculates the degree of each node, and the subgraph identification unit identifies the subgraph structure in the graph;

[0121] Reversibility judgment module: includes: a subgraph comparison unit and a reversibility judgment unit; wherein, the subgraph comparison unit compares the subgraph structures at different time points, and the reversibility judgment unit combines the node attributes and connection relationships in the graph to judge whether the abnormal data in the active area abnormal data set is reversible.

[0122] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A VCSEL chip appearance defect detection method based on visual recognition, characterized in that: The method comprises the following steps: Collect chip images, perform active area detection to obtain active area parameters, and associate the chip images and active area parameters to obtain a chip appearance detection dataset; Based on the chip appearance inspection dataset, a convolutional neural network model is trained. The chip image to be inspected is input into the trained convolutional neural network model, and the image feature vector is extracted and classified and predicted to obtain the active area anomaly dataset. The anomaly types and locations in the active area anomaly data set are integrated for cluster analysis. Combined with the active area parameters, the mean and standard deviation of the parameters within each cluster are calculated. Treat each cluster as a node and determine the edge based on the similarity between clusters; calculate the degree of each node and identify the subgraph structure in the graph; The subgraph structures at different time points are compared, and the node attributes and connection relationships in the graph are combined to determine whether the abnormal data in the active area abnormal data set are reversible.

2. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The collecting chip image and performing active area detection to obtain active area parameters include: Control the stage so that the chip is located in the center of the camera's field of view. Take a vertical image of the entire chip. While keeping the light source and camera positions unchanged, take a close-up shot of a part of the chip. Use an optical spectrum analyzer, aim the probe at the chip's active area, and set the wavelength scanning range. Use a calibrated optical power meter to measure the optical power under the same current conditions as the measured luminescence wavelength. Calculate the quantum efficiency based on the measured optical power and injection current value. Use a microscope to set the magnification, image the active area, select the edge of the active area on the image, and measure its length and width. Use an energy spectrum analyzer to inspect the active area, perform multi-point scanning of the active area, and obtain material composition and impurity concentration data.

3. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The chip image and active area parameters are associated to obtain a chip appearance inspection data set, including: The collected chip images are numbered, and the measured active area parameters are recorded in the measurement order and associated with the corresponding chip samples. The chip image number and the ID number in the active area parameter record table are used as unique identifiers to establish an association between the two and supplement the associated information; the associated information includes the acquisition time and measurement environment, and the chip appearance inspection data set is obtained.

4. The method for detecting appearance defects of VCSEL chips based on visual recognition according to claim 1, wherein: The method of training a convolutional neural network model based on a chip appearance detection dataset includes: The chip appearance inspection dataset is divided into a training set, a validation set, and a test set in proportion. In the training set, image data and their corresponding active area abnormality labels are sorted. ResNet50 is selected as the base model of the neural network model and the base model is adjusted according to the specific requirements of chip appearance inspection. Training parameters are set, and the cross entropy loss function is used to measure the difference between the model prediction value and the true label. Adam is selected as the optimizer. The training set data is input in batches into the built convolutional neural network model. The model calculates the prediction results based on forward propagation, calculates the gradient of the loss function to the model parameters through back propagation, and uses the optimizer to update the parameters. After training is completed, the model is evaluated using the test set, and the model is optimized based on the evaluation results.

5. The method for detecting appearance defects of VCSEL chips based on visual recognition according to claim 1, characterized in that: The chip image to be detected is input into the trained convolutional neural network model, the image feature vector is extracted, and classification prediction is performed to obtain the active area abnormality data set, including: The chip image to be tested is preprocessed and input into the loaded convolutional neural network model. Through the forward propagation process, the feature vector of the image is extracted through the convolution layer, pooling layer and fully connected layer. Based on the extracted feature vector, the model performs classification prediction through the softmax classifier to determine whether there is an abnormality in the active area and obtain a prediction result. According to the actual application requirements and the model performance evaluation results, the threshold for abnormality judgment is determined. Based on the prediction result and the set threshold, the active area abnormality dataset is compiled and generated. The active area abnormality dataset includes the chip image identification, abnormality type, abnormality location and abnormality judgment threshold.

6. The method for detecting appearance defects of VCSEL chips based on visual recognition according to claim 1, characterized in that: The method integrates the anomaly types and locations in the active area anomaly data set to perform cluster analysis, combines the active area parameters, and calculates the mean and standard deviation of the parameters within each cluster, including: The anomaly type is converted into a numerical form; the DBSCAN clustering algorithm is selected for cluster analysis, and the neighborhood radius eps and the minimum sample number minPts are set. The neighborhood radius represents the neighborhood range of a data point, and the minimum sample number represents the number of data points required to become a core point in a neighborhood. According to the set eps and minPts, each data point in the data set is traversed; for a data point, when the number of data points contained in a neighborhood with a radius of eps centered on the data point is greater than or equal to minPts, the data point is determined to be a core point; in the active area anomaly data, the core point represents the center of data aggregation with similar anomaly characteristics; Starting from a core point, all density-connected data points in its neighborhood are merged into a cluster. The other core points in the cluster are traversed, and the undivided data points in its neighborhood are added to the cluster, and the cluster is continuously expanded. In the process of expanding the cluster, by continuously merging density-connected data points, data with similar anomaly types, locations, and active area parameter characteristics are clustered to form different clusters. Data points that are not core points and are not density-connected to any core points are marked as noise points. For each cluster, the active area parameters are extracted from the data points it contains, and the mean and standard deviation of the parameters within each cluster are calculated.

7. The method for detecting appearance defects of VCSEL chips based on visual recognition according to claim 1, characterized in that: The method of treating each cluster as a node and determining edges based on the similarity between clusters includes: Each cluster is regarded as a node in the graph structure. Each node contains the cluster information of the cluster and the mean and standard deviation of the active area parameters. The cluster information includes the distribution of anomaly types, the central coordinates of the anomaly positions, and the degree of dispersion. The Euclidean distance is used to measure the similarity between clusters. A similarity threshold is set. When the Euclidean distance is less than the similarity threshold, an edge is established between the corresponding nodes. The weight of the edge is set to the inverse of the similarity measure value, reflecting the closeness between the clusters.

8. The method for detecting appearance defects of VCSEL chips based on visual recognition according to claim 1, characterized in that: The step of calculating the degree of each node and identifying the subgraph structure in the graph includes: The degree of the node reflects the degree of association between a cluster and other clusters. Each node in the graph structure is traversed and the number of edges connected to the node is counted to obtain the degree of the node. Use the depth-first search algorithm to identify connected components in the graph, where each connected component is a subgraph. The method is as follows: Starting from any unvisited node in the graph, mark the node as visited, and recursively visit all its unvisited adjacent nodes until all reachable nodes have been visited; when all adjacent nodes of a node have been visited, backtrack to the previous node to continue searching; repeat the process until all nodes have been visited, thereby determining all connected components.

9. The method for detecting appearance defects of VCSEL chips based on visual recognition according to claim 1, characterized in that: The comparison of subgraph structures at different time points and the determination of whether the abnormal data in the active area abnormal data set is reversible by combining the node attributes and connection relationships in the graph include: The active area anomaly datasets at different time points are collected, and a graph structure is constructed for the active area anomaly dataset at each time point. The Jaccard similarity is used to calculate the anomaly type similarity of the nodes in the subgraphs at different time points. For the first node v1 and the second node v2 in the first subgraph S1 and the second subgraph S2, the anomaly type set of v1 is A1, and the anomaly type set of v2 is A2. The Jaccard similarity is used to calculate the anomaly type similarity of the nodes in the subgraphs at different time points. J type The closer it is to 1, the more similar the anomaly types of the two nodes are; The weighted Euclidean distance is used to calculate the similarity of active area parameters. For the active area parameter mean vector of v1 and the active region parameter mean vector of v2 and the weight vector for each parameter The weight is determined according to the importance of the parameter to the chip performance, and the weighted Euclidean distance The smaller the distance, the higher the parameter similarity, where n is the number of active area parameters, p 11 , p 12 ,…,p 1n represent From the 1st element to the nth element, p 21 , p 22 ,…,p 2n represent From the 1st element to the nth element, w i is the weight vector The i-th element in represents the weight of the i-th active region parameter, p 1i is the mean vector of active region parameters The i-th element in represents The mean value of the parameter of the i-th active region, p 2i is the mean vector of active region parameters The i-th element in represents The mean value of the parameters of the i-th active region in ; Calculate the similarity of the connection relationship between corresponding nodes in the subgraphs at different time points. Suppose the adjacent node sets of the first node v1 and the second node v2 in the first subgraph S1 and the second subgraph S2 are N1 and N2 respectively, and the connection relationship similarity The closer the value is to 1, the more similar the connection relationship is; The comprehensive similarity of the subgraph is obtained by combining the node attribute similarity and the connection relationship similarity. The method is as follows: By weighted summation, let the node attribute similarity weight be w attr , the connection relationship similarity weight is w conn , comprehensive similarity S total =w attr ×(αJ type +(1-α)×(1-d param ))+w conn J conn , where α is the weight distribution coefficient of anomaly type similarity and active area parameter similarity; A reversibility judgment threshold τ is set. When the comprehensive similarity of subgraphs at different time points is greater than τ and in the subgraphs at subsequent time points, the node attributes corresponding to the abnormal data change towards the normal state and the connection relationship tends to be stable, the abnormal data is judged to be reversible; otherwise, it is judged to be irreversible.

10. A VCSEL chip appearance defect detection system based on visual recognition, using the VCSEL chip appearance defect detection method based on visual recognition according to any one of claims 1 to 9, characterized in that: include: Data acquisition and association module: includes: image acquisition unit, active area parameter detection unit and data association unit; wherein, the image acquisition unit acquires chip images, the active area parameter detection unit performs active area detection to obtain active area parameters, and the data association unit associates the chip images and active area parameters to obtain a chip appearance detection data set; Model training and prediction module: includes: model training unit and prediction unit; the model training unit trains the convolutional neural network model based on the chip appearance inspection dataset; the prediction unit inputs the chip image to be inspected into the trained convolutional neural network model, extracts the image feature vector, performs classification prediction, and obtains the active area anomaly dataset; Cluster analysis module: includes: a cluster execution unit and a parameter calculation unit; wherein the cluster execution unit integrates the anomaly type and anomaly location in the active area anomaly data set for cluster analysis, and the parameter calculation unit combines the active area parameters to calculate the mean and standard deviation of the parameters within each cluster; Graph structure construction module: includes: node definition unit, edge determination unit, node degree calculation unit and subgraph identification unit; wherein, the node definition unit regards each cluster as a node, the edge determination unit determines the edge based on the similarity between clusters; the node degree calculation unit calculates the degree of each node, and the subgraph identification unit identifies the subgraph structure in the graph; Reversibility judgment module: includes: a subgraph comparison unit and a reversibility judgment unit; wherein, the subgraph comparison unit compares the subgraph structures at different time points, and the reversibility judgment unit combines the node attributes and connection relationships in the graph to judge whether the abnormal data in the active area abnormal data set is reversible.

Images