VCSEL chip appearance defect detection system and method based on visual identification

Abstract

The invention discloses a VCSEL chip appearance defect detection system and method based on visual identification, and belongs to the technical field of appearance defect detection. The method comprises the following steps: acquiring a chip image, performing active area detection to obtain an active area parameter, and associating the chip image with the active area parameter to obtain a chip appearance detection data set; training the convolutional neural network model, and inputting a to-be-detected chip image into the trained convolutional neural network model to obtain an active region abnormal data set; integrating abnormal types and abnormal positions in the active area abnormal data set for clustering analysis, and calculating a mean value and a standard deviation of parameters in each cluster by combining active area parameters; taking each cluster as a node, and determining an edge according to the similarity between the clusters; calculating the degree of each node, and identifying a sub-graph structure in the graph; and comparing the sub-graph structures at different time points, and judging whether the abnormal data in the abnormal data set of the active region is reversible or not.

Description

Technical Field

[0001] The present invention relates to the technical field of appearance defect detection, and in particular to a VCSEL chip appearance defect detection system and method based on visual recognition. Background Art

[0002] With the rapid development of optical communications, laser radar and other fields, the quality and reliability of vertical cavity surface emitting laser (VCSEL) chips, as core components, directly affect the performance of the entire system. In the production process of VCSEL chips, due to the complexity of the manufacturing process and the sensitivity of material properties, various anomalies are prone to occur in the active area, such as scratches, holes, impurities, lattice distortion, etc. These anomalies will cause the chip's key parameters such as emission wavelength, optical power, quantum efficiency, etc. to deviate from the normal range, thereby affecting the chip's working performance and service life.

[0003] Traditional chip inspection methods mostly rely on manual visual inspection or simple optical imaging technology, which makes it difficult to detect subtle anomalies in the chip's active area. For some complex defect types and locations, missed detection and false detection are prone to occur. When judging chip anomalies, existing technologies can often only identify the existence of anomalies, but have weak capabilities in determining whether the abnormal data is reversible and distinguishing between reversible and irreversible failure modes. Faced with a large amount of chip inspection data, existing technologies lack effective data integration and analysis methods. Summary of the invention

[0004] The purpose of the present invention is to provide a VCSEL chip appearance defect detection system and method based on visual recognition to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a method for detecting appearance defects of a VCSEL chip based on visual recognition, the method comprising the following steps:

[0007] Collect chip images, perform active area detection to obtain active area parameters, associate the chip images with the active area parameters, and obtain a chip appearance detection data set;

[0008] Based on the chip appearance inspection data set, a convolutional neural network model is trained, and the chip image to be inspected is input into the trained convolutional neural network model to extract the image feature vector and perform classification prediction to obtain the active area abnormality data set;

[0009] The anomaly types and locations in the active area anomaly data set are integrated for cluster analysis, and the mean and standard deviation of the parameters in each cluster are calculated in combination with the active area parameters.

[0010] Treat each cluster as a node and determine the edge based on the similarity between clusters; calculate the degree of each node and identify the subgraph structure in the graph;

[0011] The subgraph structures at different time points are compared, and the node attributes and connection relationships in the graph are combined to determine whether the abnormal data in the active area abnormal data set are reversible.

[0012] In combination with the first aspect, in a first implementation of the first aspect of the present application, the collecting chip images and performing active area detection to obtain active area parameters include:

[0013] Control the translation stage to position the chip in the center of the camera's field of view, capture the entire chip image from a vertical angle, and take a partial close-up shot of the chip while keeping the light source and camera positions unchanged;

[0014] Use a spectrum analyzer, aim the probe at the active area of ​​the chip, and set the wavelength scanning range; use a calibrated optical power meter to measure the optical power under the same current conditions as the measured luminescent wavelength; calculate the quantum efficiency based on the measured optical power and injection current value; use a microscope, set the magnification, image the active area, select the edge of the active area on the image, and measure its length and width; use an energy spectrum analyzer to detect the active area, perform multi-point scanning on the active area, and obtain material composition and impurity concentration data.

[0015] In combination with the first aspect, in a second implementation of the first aspect of the present application, associating the chip image with the active area parameter to obtain a chip appearance inspection data set includes:

[0016] The collected chip images are numbered, and the measured active area parameters are recorded in the measurement order and associated with the corresponding chip samples; the chip image number and the ID number in the active area parameter record table are used as unique identifiers to establish an association relationship between the two and supplement the associated information; the associated information includes the acquisition time and the measurement environment, and the chip appearance detection data set is obtained.

[0017] In combination with the first aspect, in a third implementation of the first aspect of the present application, the training of the convolutional neural network model based on the chip appearance detection dataset includes:

[0018] The chip appearance inspection data set is divided into a training set, a validation set, and a test set according to a certain ratio. In the training set, the image data and its corresponding active area abnormality labels are sorted. ResNet50 is selected as the basic model of the neural network model, and the basic model is adjusted according to the specific requirements of chip appearance inspection. The training parameters are set, and the cross entropy loss function is used to measure the difference between the model prediction value and the true label. Adam is selected as the optimizer.

[0019] The training set data is input in batches into the built convolutional neural network model. The model calculates the prediction results based on forward propagation, calculates the gradient of the loss function to the model parameters through back propagation, and uses the optimizer to update the parameters. After the training is completed, the model is evaluated using the test set, and the model is optimized based on the evaluation results.

[0020] In combination with the first aspect, in a fourth implementation of the first aspect of the present application, the chip image to be detected is input into a trained convolutional neural network model, the image feature vector is extracted, and classification prediction is performed to obtain an active area abnormal data set, including:

[0021] The chip image to be detected is preprocessed and input into the loaded convolutional neural network model. Through the forward propagation process, the feature vector of the image is extracted through the convolution layer, the pooling layer and the fully connected layer. Based on the extracted feature vector, the model performs classification prediction through the softmax classifier to determine whether there is an abnormality in the active area and obtain the prediction result. According to the actual application requirements and the model performance evaluation results, the threshold of abnormality judgment is determined, and according to the prediction results and the set threshold, the active area abnormality data set is sorted and generated, and the active area abnormality data set includes chip image identification, abnormality type, abnormality location and abnormality judgment threshold.

[0022] In combination with the first aspect, in a fifth implementation of the first aspect of the present application, the abnormal type and abnormal position in the active area abnormal data set are integrated for cluster analysis, and the mean and standard deviation of the parameters in each cluster cluster are calculated in combination with the active area parameters, including:

[0023] According to the anomaly type, it is converted into a numerical form; the DBSCAN clustering algorithm is selected for clustering analysis, and the neighborhood radius eps and the minimum number of samples minPts are set. The neighborhood radius represents the neighborhood range of a data point, and the minimum number of samples represents the number of data points required to become a core point in a neighborhood; according to the set eps and minPts, each data point in the data set is traversed; for a data point, when the number of data points contained in the neighborhood with a radius of eps centered on the data point is greater than or equal to minPts, the data point is determined to be a core point; in the active area abnormal data, the core point represents the data aggregation center with similar abnormal characteristics;

[0024] Starting from a core point, all density-connected data points in its neighborhood are merged into a cluster, and other core points in the cluster are traversed to add undivided data points in its neighborhood to the cluster, and the cluster is continuously expanded; in the process of expanding the cluster, by continuously merging density-connected data points, data with similar anomaly types, locations and active area parameter characteristics are clustered to form different clusters; data points that are not core points and are not density-connected to any core points are marked as noise points;

[0025] For each cluster, the active area parameters are extracted from the data points it contains, and the mean and standard deviation of the parameters within each cluster are calculated.

[0026] In combination with the first aspect, in a sixth implementation of the first aspect of the present application, taking each cluster as a node and determining an edge according to the similarity between the clusters includes:

[0027] Each cluster is regarded as a node in the graph structure. Each node contains the cluster information of the cluster and the mean and standard deviation of the active area parameters. The cluster information includes the distribution of abnormal types, the central coordinates of the abnormal position and the degree of dispersion. The Euclidean distance is used to measure the similarity between clusters, and a similarity threshold is set. When the Euclidean distance is less than the similarity threshold, an edge is established between the corresponding nodes, and the weight of the edge is set to the inverse of the similarity measurement value, which reflects the closeness between the clusters.

[0028] In combination with the first aspect, in a seventh implementation manner of the first aspect of the present application, the step of calculating the degree of each node and identifying a subgraph structure in the graph includes:

[0029] The degree of the node reflects the degree of association between a cluster and other clusters. Each node in the graph structure is traversed, and the number of edges connected to the node is counted to obtain the degree of the node.

[0030] Use a depth-first search algorithm to identify connected components in the graph, each connected component is a subgraph, as follows:

[0031] Starting from any unvisited node in the graph, mark the node as visited, and recursively visit all its unvisited adjacent nodes until all reachable nodes are visited; when all adjacent nodes of a node have been visited, backtrack to the previous node to continue searching; repeat the process until all nodes have been visited, thereby determining all connected components.

[0032] In combination with the first aspect, in an eighth implementation of the first aspect of the present application, comparing the subgraph structures at different time points, and combining the node attributes and connection relationships in the graph to determine whether the abnormal data in the active area abnormal data set is reversible, includes:

[0033] The anomaly data sets of active areas at different time points are collected, and a graph structure is constructed for the anomaly data sets of active areas at each time point. The similarity of the anomaly types of nodes in the subgraphs at different time points is calculated using the Jaccard similarity. For the first subgraph S 1 and the second subgraph S 2 The first node v in 1 and the second node v 2 , v 1 The exception type set is A 1 , v 2 The exception type set is A 2 , Jaccard similarity J type The closer it is to 1, the more similar the anomaly types of the two nodes are;

[0034] The weighted Euclidean distance is used to calculate the similarity of active area parameters. 1 The mean vector of active region parameters and v 2 The mean vector of active region parameters and the weight vector for each parameter The weight is determined according to the importance of the parameter to the chip performance, and the weighted Euclidean distance The smaller the distance, the higher the parameter similarity, where n is the number of active area parameters and p is 11 , p 12 ,…,p 1n represent From the 1st element to the nth element, p 21 , p 22 ,…,p 2n represent From the 1st element to the nth element, w i is the weight vector The i-th element in represents the weight of the i-th active region parameter, p 1i is the mean vector of active region parameters The i-th element in represents The mean value of the parameter of the i-th active region, p 2i is the mean vector of active region parameters The i-th element in represents The mean value of the parameters of the i-th active region in ;

[0035] Calculate the similarity of the corresponding node connection relationship in the subgraphs at different time points. Suppose the first subgraph S 1 and the second subgraph S 2 The first node v 1 and the second node v 2 The adjacent node sets are N1 and N 2 , connection relationship similarity The closer the value is to 1, the more similar the connection relationship is;

[0036] The comprehensive similarity of the subgraph is obtained by combining the node attribute similarity and the connection relationship similarity. The method is as follows: By weighted summation, let the node attribute similarity weight be w attr , the connection relationship similarity weight is w conn , comprehensive similarity S total =w attr ×(αJ type +(1-α)×(1-d param ))+w conn J conn , where α is the weight distribution coefficient of anomaly type similarity and active area parameter similarity;

[0037] A reversibility judgment threshold τ is set. When the comprehensive similarity of subgraphs at different time points is greater than τ and in the subgraphs at subsequent time points, the node attributes corresponding to the abnormal data change toward the normal state and the connection relationship tends to be stable, the abnormal data is judged to be reversible; otherwise, it is judged to be irreversible.

[0038] Second, the VCSEL chip appearance defect detection system based on visual recognition includes:

[0039] Data acquisition and association module: including: image acquisition unit, active area parameter detection unit and data association unit; wherein the image acquisition unit acquires chip images, the active area parameter detection unit performs active area detection to obtain active area parameters, and the data association unit associates the chip image with the active area parameters to obtain a chip appearance detection data set;

[0040] Model training and prediction module: including: model training unit and prediction unit; the model training unit trains the convolutional neural network model based on the chip appearance detection data set, and the prediction unit inputs the chip image to be detected into the trained convolutional neural network model, extracts the image feature vector, performs classification prediction, and obtains the active area abnormality data set;

[0041] Cluster analysis module: including: a cluster execution unit and a parameter calculation unit; wherein the cluster execution unit integrates the abnormal type and abnormal position in the active area abnormal data set for cluster analysis, and the parameter calculation unit combines the active area parameters to calculate the mean and standard deviation of the parameters in each cluster cluster;

[0042] Graph structure construction module: including: node definition unit, edge determination unit, node degree calculation unit and subgraph identification unit; wherein the node definition unit regards each cluster as a node, the edge determination unit determines the edge according to the similarity between clusters; the node degree calculation unit calculates the degree of each node, and the subgraph identification unit identifies the subgraph structure in the graph;

[0043] Reversibility judgment module: includes: subgraph comparison unit and reversibility judgment unit; wherein, the subgraph comparison unit compares the subgraph structures at different time points, and the reversibility judgment unit combines the node attributes and connection relationships in the graph to judge whether the abnormal data in the active area abnormal data set is reversible.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The present invention uses high-resolution image acquisition equipment and advanced convolutional neural network models to accurately extract the feature vectors of chip images, achieve accurate classification and prediction of active area anomalies, greatly improve detection accuracy, and effectively reduce missed detections and false detections.

[0046] 2. The present invention is based on cluster analysis and graph structure construction, combined with sub-graph comparison at different time points, which can accurately determine the reversibility of abnormal data in the active area and effectively distinguish reversible and irreversible failure modes; it provides clear guidance for chip quality assessment and subsequent processing, reducing production risks and costs.

[0047] 3. The present invention realizes the effective association between chip images and active area parameters, calculates the mean and standard deviation of the parameters in each cluster through cluster analysis, and deeply explores the intrinsic connection between data; it can grasp the quality status of the chip as a whole, provide strong data support for the optimization of production process, and help improve the production quality and efficiency of the chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the steps of the VCSEL chip appearance defect detection method based on visual recognition of the present invention;

[0049] Figure 2 It is a system structure diagram of the VCSEL chip appearance defect detection system based on visual recognition of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution:

[0052] like Figure 1 As shown in the schematic diagram of the steps of the VCSEL chip appearance defect detection method based on visual recognition of the present invention, the present invention provides a VCSEL chip appearance defect detection method based on visual recognition, and the method comprises the following steps:

[0053] Step S100: collecting chip images, performing active area detection to obtain active area parameters, and correlating the chip images with the active area parameters to obtain a chip appearance detection data set;

[0054] Specifically, the translation stage is controlled so that the chip is located at the center of the camera's field of view, and the overall image of the chip is captured from a vertical angle. While keeping the light source and camera positions unchanged, a local close-up shot of the chip is taken;

[0055] Use a spectrum analyzer, aim the probe at the active area of ​​the chip, and set the wavelength scanning range; use a calibrated optical power meter to measure the optical power under the same current conditions as the measured luminescent wavelength; calculate the quantum efficiency based on the measured optical power and injection current value; use a microscope, set the magnification, image the active area, select the edge of the active area on the image, and measure its length and width; use an energy spectrum analyzer to detect the active area, perform multi-point scanning on the active area, and obtain material composition and impurity concentration data.

[0056] Furthermore, the collected chip images are numbered, and the measured active area parameters are recorded in the measurement order and linked to the corresponding chip samples; the chip image number and the ID number in the active area parameter record table are used as unique identifiers to establish an association relationship between the two, and supplement the associated information; the associated information includes the acquisition time and the measurement environment, and the chip appearance detection data set is obtained.

[0057] In a specific embodiment, 100 VCSEL chips are selected as experimental samples, and a high-resolution industrial camera with a resolution of 5 million pixels, a translation stage, a ring-shaped LED light source, a spectrum analyzer with a wavelength scanning range of 800-1000nm and a resolution of 0.1nm, a calibrated optical power meter, a high-precision microscope with a magnification of 500-1000 times, an energy spectrum analyzer, a high-precision constant current source and other equipment are prepared.

[0058] Control the stage so that the chip is located in the center of the camera's field of view, and take a vertical image of the entire chip with a resolution of 2048×2048 pixels. While keeping the light source and camera positions unchanged, take a local close-up shot of the chip, with each close-up area overlapping by 10%-20%. Number the collected chip images in chronological order, such as "20241001-001", "20241001-002", etc.

[0059] Active area parameter detection: Use a spectrum analyzer, aim the probe at the chip active area, set the wavelength scanning range to 800-1000nm, and the scanning resolution to 0.1nm. Under the condition that the chip working current is stable at 10mA, measure the luminous wavelength. For example, the chip with the chip number "20241001-001" has a measured luminous wavelength of 850.5nm. Use a calibrated optical power meter to measure the optical power under the same 10mA current condition as the measured luminous wavelength. For example, the optical power of the "20241001-001" chip is 5mW. Calculate the quantum efficiency based on the measured optical power and injection current value. The injection current is 10mA, the optical power is 5mW, and the quantum efficiency is 31.25% calculated by the formula. Use a microscope, set the magnification to 800 times, and image the active area. Select the edge of the active area on the image and measure its length and width with an accuracy of 0.1μm. For example, the active area length of the "20241001-001" chip is 5.2μm and the width is 4.8μm. Use an energy spectrum analyzer to detect the active area, scan the active area at 5 points, and obtain material composition and impurity concentration data. For example, in the "20241001-001" chip, the main material is GaAs, and the impurity concentration is 0.005at%. The measured active area parameters are recorded in the measurement order, and are linked to the corresponding chip samples and recorded in an Excel table. Each row represents the parameter data of a chip, including chip ID, emission wavelength, optical power, quantum efficiency, active area length, active area width, material composition, impurity concentration and other fields.

[0060] The chip image number and the "chip ID" in the active area parameter record table are used as unique identifiers to establish an association between the two. Using Python's pandas library, the image number data and parameter record data are merged according to the "chip ID". Supplement the associated information, including the acquisition time and measurement environment. Finally, a chip appearance inspection dataset containing chip image information, active area parameters, and associated information is obtained.

[0061] Step S200: Based on the chip appearance detection data set, a convolutional neural network model is trained, the chip image to be detected is input into the trained convolutional neural network model, the image feature vector is extracted, and classification prediction is performed to obtain an active area abnormality data set;

[0062] Specifically, the chip appearance inspection data set is divided into a training set, a validation set, and a test set according to a ratio. In the training set, the image data and its corresponding active area abnormality labels are sorted; ResNet50 is selected as the basic model of the neural network model, and the basic model is adjusted according to the specific needs of chip appearance inspection; the training parameters are set, and the cross entropy loss function is used to measure the difference between the model prediction value and the true label, and Adam is selected as the optimizer;

[0063] The training set data is input in batches into the built convolutional neural network model. The model calculates the prediction results based on forward propagation, calculates the gradient of the loss function to the model parameters through back propagation, and uses the optimizer to update the parameters. After the training is completed, the model is evaluated using the test set, and the model is optimized based on the evaluation results.

[0064] Furthermore, the chip image to be detected is preprocessed and input into the loaded convolutional neural network model. Through the forward propagation process, the feature vector of the image is extracted through the convolution layer, the pooling layer and the fully connected layer. Based on the extracted feature vector, the model performs classification prediction through the softmax classifier to determine whether there is an abnormality in the active area and obtain a prediction result. According to the actual application requirements and the model performance evaluation results, the threshold for abnormality judgment is determined, and according to the prediction results and the set threshold, the active area abnormality data set is sorted and generated, and the active area abnormality data set includes the chip image identification, abnormality type, abnormality location and abnormality judgment threshold.

[0065] In a specific embodiment, the chip appearance inspection data set construction work of 100 VCSEL chips completed in the previous stage is taken over. The chip appearance inspection data set is divided into a training set, a validation set, and a test set in a ratio of 70%, 15%, and 15%. That is, the training set contains data of 70 chips, and the validation set and the test set each contain data of 15 chips. In the training set, the image data and its corresponding active area abnormal labels, such as "scratches", "holes", "normal", etc., are sorted and converted into numerical labels, "scratches" are 1, "holes" are 2, and "normal" is 0. ResNet50 is selected as the basic model of the neural network model. According to the specific needs of chip appearance inspection, the basic model is adjusted as follows: the input layer of the model is modified to adapt to the chip image size of 2048×2048 pixels and the characteristics of 3 channels (RGB); the number of neurons in the output layer is adjusted to 3 (corresponding to the three states of normal, scratches, and holes), and the activation function uses softmax. Set the learning rate to 0.001, the batch size to 32, and the number of training rounds to 50. The cross entropy loss function is used to measure the difference between the model prediction value and the true label. The optimizer selects Adam with default parameters beta1=0.9, beta2=0.999, and epsilon=1e-8.

[0066] The training set data is input into the built convolutional neural network model in batches. During the training process, the model calculates the prediction results based on the forward propagation. For example, in the 10th round of training, for a batch of training data containing 32 samples, the prediction results output by the model are compared with the true labels to calculate the cross entropy loss. The gradient of the loss function to the model parameters is calculated by back propagation, and the parameters are updated using the Adam optimizer. After every 5 rounds of training, an evaluation is performed on the validation set, and the loss and accuracy of the validation set are recorded. In the 30th round of training, the accuracy of the validation set reached 80%, and the loss was reduced to about 0.5. After the training is completed, the model is evaluated using the test set. On the test set, the accuracy of the model reached 85%, the recall rate was 82%, and the F1 value was 0.83. According to the evaluation results, it was found that the model's recognition accuracy for the "scratch" anomaly was relatively low, at 75%. In order to optimize the model, the data enhancement method is used to rotate and flip the training set images to increase the diversity of the data. After retraining the model, on the test set, the model's recognition accuracy for "scratch" anomalies increased to 80%, the overall accuracy reached 88%, the recall rate was 85%, and the F1 value increased to 0.86.

[0067] The chip image to be tested is preprocessed, including normalization (scaling the pixel value to the range of 0-1) and resizing to 224×224 pixels (to meet the model input requirements). The preprocessed image is input into the loaded convolutional neural network model, and the feature vector of the image is extracted through the forward propagation process, the convolution layer, the pooling layer, and the fully connected layer. For example, for the chip image to be tested numbered "20241002-001", the model extracts a 512-dimensional feature vector. Based on the extracted feature vector, the model performs classification prediction through the softmax classifier to determine whether there is an abnormality in the active area. The probability of the predicted result being "normal" is 0.9, the probability of "scratch" is 0.05, and the probability of "hole" is 0.05.

[0068] According to the actual application requirements and the model performance evaluation results, the threshold for abnormal judgment is determined to be 0.5. According to the prediction results and the set threshold, the active area abnormal data set is sorted and generated. For example, for the chip image numbered "20241002-001" above, its active area abnormal data set is recorded as follows: chip image identification: "20241002-001", abnormal type: "normal", abnormal location: none (because it is judged to be normal), abnormal judgment threshold: 0.5.

[0069] Step S300: Integrate the abnormal types and abnormal positions in the active area abnormal data set for cluster analysis, and calculate the mean and standard deviation of the parameters in each cluster cluster in combination with the active area parameters;

[0070] Specifically, for the anomaly type, convert it into a numerical form; select the DBSCAN clustering algorithm for clustering analysis, set the neighborhood radius eps and the minimum number of samples minPts, the neighborhood radius represents the neighborhood range of a data point, and the minimum number of samples represents the number of data points required to become a core point in a neighborhood; according to the set eps and minPts, traverse each data point in the data set; for a data point, when the number of data points contained in the neighborhood with a radius of eps centered on the data point is greater than or equal to minPts, the data point is determined to be a core point; in the active area abnormal data, the core point represents the data aggregation center with similar abnormal characteristics;

[0071] Starting from a core point, all density-connected data points in its neighborhood are merged into a cluster, and other core points in the cluster are traversed to add undivided data points in its neighborhood to the cluster, and the cluster is continuously expanded; in the process of expanding the cluster, by continuously merging density-connected data points, data with similar anomaly types, locations and active area parameter characteristics are clustered to form different clusters; data points that are not core points and are not density-connected to any core points are marked as noise points;

[0072] For each cluster, the active area parameters are extracted from the data points it contains, and the mean and standard deviation of the parameters within each cluster are calculated.

[0073] In a specific embodiment, the DBSCAN clustering algorithm is selected for cluster analysis. Through multiple experiments and preliminary observation of the data, the neighborhood radius eps=0.5 and the minimum number of samples minPts=5 are set. The neighborhood radius here represents a circular area with a radius of 0.5 centered on a certain data point in the data space; the minimum number of samples means that at least 5 data points are required in the neighborhood before the data point can be determined as a core point.

[0074] According to the set eps and minPts, traverse each data point in the data set. For example, for data point A, when the number of data points contained in the neighborhood with A as the center and a radius of 0.5 is greater than or equal to 5, A is determined to be a core point. In the data set, there are data points B, C, D, E, and F in the neighborhood of A and meet the minimum sample number requirement, and A becomes a core point.

[0075] Starting from the core point A, all the density-connected data points B, C, D, E, and F in its neighborhood are merged into one cluster. Then traverse other core points in the cluster. If B is also a core point, the undivided data points in the neighborhood of B are added to the cluster, and the cluster is continuously expanded. During the expansion process, data with similar anomaly types, locations, and active area parameter characteristics are clustered together. For example, data points with anomaly types of "scratches" and close locations, and similar active area optical power parameters, will be clustered into the same cluster.

[0076] For those data points that are neither core points nor densely connected to any core points, they are marked as noise points. Data point G is not in the neighborhood of any core point, and G is marked as a noise point.

[0077] After cluster analysis, multiple clusters were obtained. For each cluster, active area parameters such as luminous wavelength and optical power were extracted from the data points contained in it. Taking optical power as an example, a cluster contains 10 data points, and their optical power values ​​are: 4.5mW, 5.0mW, 4.8mW, 5.2mW, 4.9mW, 5.1mW, 4.7mW, 5.3mW, 4.6mW, and 5.0mW.

[0078] Sum: 4.5+5.0+4.8+5.2+4.9+5.1+4.7+5.3+4.6+5.0=49.1; Calculate the mean: 49.1 / 10=4.91mW;

[0079] Calculate the difference between each data point and the mean: 4.5-4.91=-0.41, 5.0-4.91=0.09, ..., 5.0-4.91=0.09; calculate the square of the difference, the result is as follows: (-0.41) 2 =0.1681, (0.09) 2 =0.0081, ..., (0.09) 2 =0.0081; Sum of squared deviations: 0.1681+0.0081+0.0121+0.0841+0.0001+0.0361+0.0441+0.1521+0.0961+0.0081=0.609. Calculate the standard deviation: Similarly, the means and standard deviations of other active area parameters in each cluster are calculated.

[0080] Step S400: taking each cluster as a node, determining the edge according to the similarity between the clusters; calculating the degree of each node, and identifying the subgraph structure in the graph;

[0081] Specifically, each cluster is regarded as a node in the graph structure, and each node contains the cluster information of the cluster and the mean and standard deviation of the active area parameters. The cluster information includes the distribution of abnormal types, the central coordinates of the abnormal positions, and the degree of dispersion. The Euclidean distance is used to measure the similarity between clusters, and a similarity threshold is set. When the Euclidean distance is less than the similarity threshold, an edge is established between the corresponding nodes, and the weight of the edge is set to the inverse of the similarity measurement value, reflecting the closeness between the clusters.

[0082] Furthermore, the degree of the node reflects the degree of association between a cluster and other clusters. Each node in the graph structure is traversed, and the number of edges connected to the node is counted to obtain the degree of the node.

[0083] Use a depth-first search algorithm to identify connected components in the graph, each connected component is a subgraph, as follows:

[0084] Starting from any unvisited node in the graph, mark the node as visited, and recursively visit all its unvisited adjacent nodes until all reachable nodes are visited; when all adjacent nodes of a node have been visited, backtrack to the previous node to continue searching; repeat the process until all nodes have been visited, thereby determining all connected components.

[0085] In a specific embodiment, the clusters obtained by the cluster analysis in the previous stage are used, and each cluster contains the distribution of abnormal types, the central coordinates and the degree of dispersion of the abnormal position, and the mean and standard deviation of the active area parameters. After cluster analysis, a total of 5 clusters are obtained, which are recorded as C1, C2, C3, C4, and C5. Taking the two active area parameters of optical power and luminous wavelength as examples, the Euclidean distance between clusters is calculated. The mean optical power of C1 is 4.91mW, and the mean luminous wavelength is 850nm; the mean optical power of C2 is 5.5mW, and the mean luminous wavelength is 860nm. The Euclidean distance is calculated to obtain the distance matrix as shown in Table 1:

[0086] Table 1 Distance matrix table

[0087] C1 C2 C3 C4 C5 C1 0 10.02 12.5 8.3 15.2 C2 10.02 0 9.1 11.7 13.4 C3 12.5 9.1 0 10.05 11.2 C4 8.3 11.7 10.5 0 9.8 C5 15.2 13.4 11.2 9.8 0

[0088] Set the similarity threshold to 10. When the Euclidean distance is less than the threshold, an edge is established between the corresponding nodes, and the weight of the edge is set to the inverse of the similarity metric (Euclidean distance). For example, the Euclidean distance between C1 and C4 is 8.3, which is less than the threshold of 10, so an edge is established between nodes C1 and C4 with a weight of 1 / 8.3≈0.12.

[0089] After judgment, the established edges and weights are as follows: C1-C4: weight 0.12, C2-C3: weight 0.11, C2-C5: weight 0.07, C3-C5: weight 0.09, C4-C5: weight 0.10.

[0090] Traverse each node in the graph structure, count the number of edges connected to the node, and get the degree of the node. Node C1: connected to C4, degree 1. Node C2: connected to C3 and C5, degree 2. Node C3: connected to C2 and C5, degree 2. Node C4: connected to C1 and C5, degree 2. Node C5: connected to C2, C3, and C4, degree 3.

[0091] Use a depth-first search algorithm to identify connected components in a graph.

[0092] Starting from node C1, mark C1 as visited, visit its adjacent node C4, mark C4 as visited. At this time, C4 has no other unvisited adjacent nodes, and backtracking to C1, C1 also has no other unvisited adjacent nodes.

[0093] Next, we start from the unvisited node C2, mark C2 as visited, visit its adjacent node C3, mark C3 as visited, and then visit C3's adjacent node C5, mark C5 as visited. At this point, C5 has no other unvisited adjacent nodes, and we trace back to C3, and C3 has no other unvisited adjacent nodes, and we trace back to C2, and C2 has no other unvisited adjacent nodes.

[0094] Finally, it is determined that there are two connected components, namely two subgraphs: Figure 1 : Contains nodes C1 and C4. Figure 2 : Contains nodes C2, C3, and C5.

[0095] Step S500: Compare the subgraph structures at different time points, and determine whether the abnormal data in the active area abnormal data set is reversible by combining the node attributes and connection relationships in the graph.

[0096] Specifically, the anomaly data sets of the active area at different time points are collected, and a graph structure is constructed for the anomaly data sets of the active area at each time point; the similarity of the anomaly types of the nodes in the subgraphs at different time points is calculated using the Jaccard similarity. For the first subgraph S 1 and the second subgraph S 2 The first node v in 1 and the second node v 2 , v 1 The exception type set is A 1 , v 2 The exception type set is A 2 , Jaccard similarity J type The closer it is to 1, the more similar the anomaly types of the two nodes are;

[0097] The weighted Euclidean distance is used to calculate the similarity of active area parameters. 1 The mean vector of active region parameters and v 2 The mean vector of active region parameters and the weight vector for each parameter The weight is determined according to the importance of the parameter to the chip performance, and the weighted Euclidean distance The smaller the distance, the higher the parameter similarity, where n is the number of active area parameters and p is 11 , p 12 ,…,p 1n represent From the 1st element to the nth element, p 21 , p 22 ,…,p 2n represent From the 1st element to the nth element, w i is the weight vector The i-th element in represents the weight of the i-th active region parameter, p 1i is the mean vector of active region parameters The i-th element in represents The mean value of the parameter of the i-th active region, p 2iis the mean vector of active region parameters The i-th element in represents The mean value of the parameters of the i-th active region in ;

[0098] Calculate the similarity of the corresponding node connection relationship in the subgraphs at different time points. Suppose the first subgraph S 1 and the second subgraph S 2 The first node v 1 and the second node v 2 The adjacent node sets are N 1 and N 2 , connection relationship similarity The closer the value is to 1, the more similar the connection relationship is;

[0099] The comprehensive similarity of the subgraph is obtained by combining the node attribute similarity and the connection relationship similarity. The method is as follows: By weighted summation, let the node attribute similarity weight be w attr , the connection relationship similarity weight is w conn , comprehensive similarity S total =w attr ×(αJ type +(1-α)×(1-d param ))+w conn J conn , where α is the weight distribution coefficient of anomaly type similarity and active area parameter similarity;

[0100] A reversibility judgment threshold τ is set. When the comprehensive similarity of subgraphs at different time points is greater than τ and in the subgraphs at subsequent time points, the node attributes corresponding to the abnormal data change toward the normal state and the connection relationship tends to be stable, the abnormal data is judged to be reversible; otherwise, it is judged to be irreversible.

[0101] In one embodiment, two different time points t are collected. 1 and t 2 The active area anomaly data set of 1 At this moment, after cluster analysis and graph structure construction, we get the two subgraphs as mentioned above: Figure 1 : Contains nodes C1 and C4. Figure 2 : Contains nodes C2, C3, and C5. 2 At this moment, after cluster analysis and graph structure construction, two subgraphs are obtained: Subgraph 3: Contains nodes C1', C4' (with t 1 Subgraph 4: Contains nodes C2', C3', C5' (corresponding to C1 and C4 at time t, but the attributes may change). 1 C2, C3, and C5 at the same time correspond, but the properties may change).

[0102] t1 Moment Figure 1 The abnormal type set A of node C1 1 ={scratch}, t 2 The exception type set A corresponding to node C1' in subgraph 3 at time 2 ={scratch}.

[0103] According to the Jaccard similarity formula Here | A 1 ∩A 2 |=1,|A 1 ∪A 2 |=1, so J type =1.

[0104] Assume t 1 The mean vector of active area parameters of node C1 at time where p 11 The average optical power is 4.91mW, p 12 The average emission wavelength is 850nm; t 2 The mean value vector of the active area parameters corresponding to node C1' at the moment where p 21 The average optical power is 5.0mW, p 22 The average emission wavelength is 852nm.

[0105] Weight vector of optical power and emission wavelength According to the weighted Euclidean distance formula, where n = 2, the calculation process is as follows:

[0106]

[0107] The similarity of active area parameters is 1-d param The larger the value, the more similar they are, that is, 1-1.26=-0.26.

[0108] t 1 Moment Figure 1 The set of adjacent nodes N of the middle node C1 1 ={C4},t 2 The adjacent node set N corresponding to node C1' in subgraph 3 at time 2 ={C4′}.

[0109] According to the connection relationship similarity formula Here | N 1 ∩N 2 |=1,|N 1 ∪N 2 |=1, so J conn =1.

[0110] Let the node attribute similarity weight w attr=0.6, connection relationship similarity weight w conn =0.4, the weight allocation coefficient of anomaly type similarity and active area parameter similarity α=0.5.

[0111] The node attribute similarity is:

[0112] αJ type +(1-α)×(1-d param )=0.5×1+(1-0.5)×(-0.26)=0.5-0.13=0.37.

[0113] The comprehensive similarity is:

[0114] S total =w attr ×0.37+w conn J conn =0.6×0.37+0.4×1=0.222+0.4=0.622.

[0115] Set the reversibility judgment threshold τ = 0.7. total =0.622<0.7, so it is preliminarily judged that the abnormal data is irreversible. However, it is necessary to observe whether the node attributes corresponding to the abnormal data change towards the normal state and whether the connection relationship tends to be stable in the subgraph at the subsequent time point. After observation, at the subsequent time point, the abnormal type of node C1 is still scratch, the active area parameters do not have an obvious trend towards the normal state, and the connection relationship is not more stable, so it is finally judged that the abnormal data is irreversible.

[0116] like Figure 2 The present invention provides a system structure diagram of a VCSEL chip appearance defect detection system based on visual recognition, and the present invention provides a VCSEL chip appearance defect detection system based on visual recognition, including:

[0117] Data acquisition and association module: including: image acquisition unit, active area parameter detection unit and data association unit; wherein the image acquisition unit acquires chip images, the active area parameter detection unit performs active area detection to obtain active area parameters, and the data association unit associates the chip image with the active area parameters to obtain a chip appearance detection data set;

[0118] Model training and prediction module: including: model training unit and prediction unit; the model training unit trains the convolutional neural network model based on the chip appearance detection data set, and the prediction unit inputs the chip image to be detected into the trained convolutional neural network model, extracts the image feature vector, performs classification prediction, and obtains the active area abnormality data set;

[0119] Cluster analysis module: including: a cluster execution unit and a parameter calculation unit; wherein the cluster execution unit integrates the abnormal type and abnormal position in the active area abnormal data set for cluster analysis, and the parameter calculation unit combines the active area parameters to calculate the mean and standard deviation of the parameters in each cluster cluster;

[0120] Graph structure construction module: including: node definition unit, edge determination unit, node degree calculation unit and subgraph identification unit; wherein the node definition unit regards each cluster as a node, the edge determination unit determines the edge according to the similarity between clusters; the node degree calculation unit calculates the degree of each node, and the subgraph identification unit identifies the subgraph structure in the graph;

[0121] Reversibility judgment module: includes: subgraph comparison unit and reversibility judgment unit; wherein, the subgraph comparison unit compares the subgraph structures at different time points, and the reversibility judgment unit combines the node attributes and connection relationships in the graph to judge whether the abnormal data in the active area abnormal data set is reversible.

[0122] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A VCSEL chip appearance defect detection method based on visual recognition, characterized in that: The method comprises the following steps: Collect chip images, perform active area detection to obtain active area parameters, associate the chip images with the active area parameters, and obtain a chip appearance detection data set; Based on the chip appearance inspection data set, a convolutional neural network model is trained, and the chip image to be inspected is input into the trained convolutional neural network model to extract the image feature vector and perform classification prediction to obtain the active area abnormality data set; The anomaly types and locations in the active area anomaly data set are integrated for cluster analysis, and the mean and standard deviation of the parameters in each cluster are calculated in combination with the active area parameters. Treat each cluster as a node and determine the edge based on the similarity between clusters; calculate the degree of each node and identify the subgraph structure in the graph; The subgraph structures at different time points are compared, and the node attributes and connection relationships in the graph are combined to determine whether the abnormal data in the active area abnormal data set are reversible.

2. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The collecting chip image and performing active area detection to obtain active area parameters includes: Control the translation stage to position the chip in the center of the camera's field of view, capture the entire chip image from a vertical angle, and take a partial close-up shot of the chip while keeping the light source and camera positions unchanged; Use a spectrum analyzer, aim the probe at the active area of ​​the chip, and set the wavelength scanning range; use a calibrated optical power meter to measure the optical power under the same current conditions as the measured luminescent wavelength; calculate the quantum efficiency based on the measured optical power and injection current value; use a microscope, set the magnification, image the active area, select the edge of the active area on the image, and measure its length and width; use an energy spectrum analyzer to detect the active area, perform multi-point scanning on the active area, and obtain material composition and impurity concentration data.

3. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The chip image and the active area parameters are associated to obtain a chip appearance detection data set, including: The collected chip images are numbered, and the measured active area parameters are recorded in the measurement order and associated with the corresponding chip samples; the chip image number and the ID number in the active area parameter record table are used as unique identifiers to establish an association relationship between the two and supplement the associated information; the associated information includes the acquisition time and the measurement environment, and the chip appearance detection data set is obtained.

4. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The method of training a convolutional neural network model based on a chip appearance detection dataset includes: The chip appearance inspection data set is divided into a training set, a validation set, and a test set according to a certain ratio. In the training set, the image data and its corresponding active area abnormality labels are sorted. ResNet50 is selected as the basic model of the neural network model, and the basic model is adjusted according to the specific requirements of chip appearance inspection. The training parameters are set, and the cross entropy loss function is used to measure the difference between the model prediction value and the true label. Adam is selected as the optimizer. The training set data is input in batches into the built convolutional neural network model. The model calculates the prediction results based on forward propagation, calculates the gradient of the loss function to the model parameters through back propagation, and uses the optimizer to update the parameters. After the training is completed, the model is evaluated using the test set, and the model is optimized based on the evaluation results.

5. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The chip image to be detected is input into the trained convolutional neural network model, the image feature vector is extracted, and classification prediction is performed to obtain the active area abnormal data set, including: The chip image to be detected is preprocessed and input into the loaded convolutional neural network model. Through the forward propagation process, the feature vector of the image is extracted through the convolution layer, the pooling layer and the fully connected layer. Based on the extracted feature vector, the model performs classification prediction through the softmax classifier to determine whether there is an abnormality in the active area and obtain the prediction result. According to the actual application requirements and the model performance evaluation results, the threshold of abnormality judgment is determined, and according to the prediction results and the set threshold, the active area abnormality data set is sorted and generated, and the active area abnormality data set includes chip image identification, abnormality type, abnormality location and abnormality judgment threshold.

6. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The method integrates the abnormal type and abnormal position in the active area abnormal data set for cluster analysis, combines the active area parameters, and calculates the mean and standard deviation of the parameters in each cluster, including: According to the anomaly type, it is converted into a numerical form; the DBSCAN clustering algorithm is selected for clustering analysis, and the neighborhood radius eps and the minimum number of samples minPts are set. The neighborhood radius represents the neighborhood range of a data point, and the minimum number of samples represents the number of data points required to become a core point in a neighborhood; according to the set eps and minPts, each data point in the data set is traversed; for a data point, when the number of data points contained in the neighborhood with a radius of eps centered on the data point is greater than or equal to minPts, the data point is determined to be a core point; in the active area abnormal data, the core point represents the data aggregation center with similar abnormal characteristics; Starting from a core point, all density-connected data points in its neighborhood are merged into a cluster, and other core points in the cluster are traversed to add undivided data points in its neighborhood to the cluster, and the cluster is continuously expanded; in the process of expanding the cluster, by continuously merging density-connected data points, data with similar anomaly types, locations and active area parameter characteristics are clustered to form different clusters; data points that are not core points and are not density-connected to any core points are marked as noise points; For each cluster, the active area parameters are extracted from the data points it contains, and the mean and standard deviation of the parameters within each cluster are calculated.

7. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The method of treating each cluster as a node and determining edges according to the similarities between clusters includes: Each cluster is regarded as a node in the graph structure. Each node contains the cluster information of the cluster and the mean and standard deviation of the active area parameters. The cluster information includes the distribution of abnormal types, the central coordinates of the abnormal position and the degree of dispersion. The Euclidean distance is used to measure the similarity between clusters, and a similarity threshold is set. When the Euclidean distance is less than the similarity threshold, an edge is established between the corresponding nodes, and the weight of the edge is set to the inverse of the similarity measurement value, which reflects the closeness between the clusters.

8. The method for detecting appearance defects of VCSEL chips based on visual recognition according to claim 1, characterized in that: The step of calculating the degree of each node and identifying the subgraph structure in the graph includes: The degree of the node reflects the degree of association between a cluster and other clusters. Each node in the graph structure is traversed, and the number of edges connected to the node is counted to obtain the degree of the node. Use a depth-first search algorithm to identify connected components in the graph, each connected component is a subgraph, as follows: Starting from any unvisited node in the graph, mark the node as visited, and recursively visit all its unvisited adjacent nodes until all reachable nodes are visited; when all adjacent nodes of a node have been visited, backtrack to the previous node to continue searching; repeat the process until all nodes have been visited, thereby determining all connected components.

9. The VCSEL chip appearance defect detection method based on visual recognition according to claim 1, characterized in that: The subgraph structures at different time points are compared, and the node attributes and connection relationships in the graph are combined to determine whether the abnormal data in the active area abnormal data set is reversible, including: The anomaly data sets of active areas at different time points are collected, and a graph structure is constructed for the anomaly data sets of active areas at each time point. The similarity of the anomaly types of nodes in the subgraphs at different time points is calculated using the Jaccard similarity. For the first node v1 and the second node v2 in the first subgraph S1 and the second subgraph S2, the anomaly type set of v1 is A1, and the anomaly type set of v2 is A2. The Jaccard similarity J type The closer it is to 1, the more similar the anomaly types of the two nodes are; The weighted Euclidean distance is used to calculate the similarity of active area parameters. For the active area parameter mean vector of v1 and the active region parameter mean vector of v2 and the weight vector for each parameter The weight is determined according to the importance of the parameter to the chip performance, and the weighted Euclidean distance The smaller the distance, the higher the parameter similarity, where n is the number of active area parameters and p is 11 , p 12 ,…,p 1n represent From the 1st element to the nth element, p 21 , p 22 ,…,p 2n represent From the 1st element to the nth element, w i is the weight vector The i-th element in represents the weight of the i-th active region parameter, p 1i is the mean vector of active region parameters The i-th element in represents The mean value of the parameter of the i-th active region, p 2i is the mean vector of active region parameters The i-th element in represents The mean value of the parameters of the i-th active region in ; Calculate the similarity of the connection relationship between the corresponding nodes in the subgraphs at different time points. Suppose the adjacent node sets of the first node v1 and the second node v2 in the first subgraph S1 and the second subgraph S2 are N1 and N2 respectively, and the connection relationship similarity The closer the value is to 1, the more similar the connection relationship is; The comprehensive similarity of the subgraph is obtained by combining the node attribute similarity and the connection relationship similarity. The method is as follows: By weighted summation, let the node attribute similarity weight be w attr , the connection relationship similarity weight is w conn , comprehensive similarity S total =w attr ×(αJ type +(1-α)×(1-d param ))+w conn J conn , where α is the weight distribution coefficient of anomaly type similarity and active area parameter similarity; A reversibility judgment threshold τ is set. When the comprehensive similarity of subgraphs at different time points is greater than τ and in the subgraphs at subsequent time points, the node attributes corresponding to the abnormal data change toward the normal state and the connection relationship tends to be stable, the abnormal data is judged to be reversible; otherwise, it is judged to be irreversible.

10. A VCSEL chip appearance defect detection system based on visual recognition, using the VCSEL chip appearance defect detection method based on visual recognition according to any one of claims 1 to 9, characterized in that: include: Data acquisition and association module: including: image acquisition unit, active area parameter detection unit and data association unit; wherein the image acquisition unit acquires chip images, the active area parameter detection unit performs active area detection to obtain active area parameters, and the data association unit associates the chip image with the active area parameters to obtain a chip appearance detection data set; Model training and prediction module: including: model training unit and prediction unit; the model training unit trains the convolutional neural network model based on the chip appearance detection data set, and the prediction unit inputs the chip image to be detected into the trained convolutional neural network model, extracts the image feature vector, performs classification prediction, and obtains the active area abnormality data set; Cluster analysis module: including: a cluster execution unit and a parameter calculation unit; wherein the cluster execution unit integrates the abnormal type and abnormal position in the active area abnormal data set for cluster analysis, and the parameter calculation unit combines the active area parameters to calculate the mean and standard deviation of the parameters in each cluster cluster; Graph structure construction module: including: node definition unit, edge determination unit, node degree calculation unit and subgraph identification unit; wherein the node definition unit regards each cluster as a node, the edge determination unit determines the edge according to the similarity between clusters; the node degree calculation unit calculates the degree of each node, and the subgraph identification unit identifies the subgraph structure in the graph; Reversibility judgment module: includes: subgraph comparison unit and reversibility judgment unit; wherein, the subgraph comparison unit compares the subgraph structures at different time points, and the reversibility judgment unit combines the node attributes and connection relationships in the graph to judge whether the abnormal data in the active area abnormal data set is reversible.

Images