An AI detection method for surface defects of electroluminescent semiconductor sheets

Abstract

The present invention relates to an AI detection method for surface defects of electroluminescent semiconductor light-emitting plates, the method steps comprising performing a first image enhancement process, performing image defect annotation and conversion, random distribution of training verification test images, performing a second image enhancement process, establishing a target detection neural network, performing data training to obtain optimal weight parameters, and AI detection of plate images. The present invention aims to solve the technical problems in the prior art of performing defect detection on the surface of electroluminescent semiconductor plates, which are limited by the quality and quantity of image data sets, the single image data preprocessing method, and the small size of the pixel algorithm network, resulting in slow defect recognition speed, low accuracy, and lack of accurate classification and positioning capabilities. The method is suitable for the detection of semiconductor chips and panels such as silicon, germanium, gallium arsenide, and silicon carbide.

Description

Technical Field

[0001] The invention relates to an AI detection method for surface defects, in particular to an AI detection method for surface defects of an electroluminescent semiconductor plate, and belongs to the field of image recognition and nondestructive testing. Background Art

[0002] Electroluminescence (EL) is a testing technology that uses near-infrared images taken by a high-resolution infrared camera to detect surface defects of panels. It plays a key role in photovoltaic power station operation and maintenance. The recognition of EL image defect feature information is the core part of the detection process.

[0003] Chinese patent 201810794758.5 ​​discloses a method for detecting and identifying electroluminescent images of solar cells, which specifically includes the following steps: obtaining an electroluminescent image of a solar cell to be detected, locating the grid lines and performing region segmentation; deleting the grid line area and recombining the image, calculating the image grayscale value and performing a two-dimensional construction; calculating the inter-class discreteness matrix of the particle swarm and determining the current optimal position; updating the optimal individual of the particle swarm and the historical optimal individual of the particle; generating new chaotic variables using a chaotic model; updating the position and velocity of all particles in the particle swarm and recalculating until the number of iterations is reached; obtaining a defective image of the cell according to the obtained optimal position segmentation and performing defect identification.

[0004] Chinese patent 201911187612.5 discloses an EL defect detection method applied to photovoltaic power station sites, comprising the following steps: preprocessing the acquired original EL image of the photovoltaic module to obtain a corrected electroluminescent image; cropping the corrected EL image to form an m×n specification cell; graying, corroding, dilating, binarizing, and pixel computing the obtained cell, finding possible EL defects based on pixel projection, and performing defect analysis.

[0005] The above method can segment, classify and combine electroluminescent images, and determine the types of defects on the plate surface through pixel operations. However, due to the limitations of the quality and quantity of image data sets, the single image data preprocessing method and the small size of the pixel algorithm network, the recognition speed of plate defects displayed by electroluminescent images is slow and the accuracy is low, and there is a lack of accurate classification and positioning capabilities for different types of defects on the same plate surface. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention aims to provide an AI detection method for surface defects of electroluminescent semiconductor plates, in order to solve the technical problems of single image data, small pixel algorithm network scale, slow recognition process, low accuracy and lack of accurate positioning of defects in the prior art. This method is suitable for the detection of semiconductor chips and panels such as silicon, germanium, gallium arsenide, and silicon carbide.

[0007] In order to solve the above technical problems, the present invention discloses an AI detection method for surface defects of electroluminescent semiconductor plates, comprising the following steps in sequence:

[0008] 1. The first image enhancement process

[0009] (1) Input an electroluminescent image of a semiconductor plate containing defects, and use an image cropping function to resize the input image into a square image of 640×640 pixels;

[0010] (2) Write an image random flipping algorithm, set the image horizontal and vertical flipping probability to 0.5, flip the image and save it;

[0011] (3) Write an image random rotation algorithm, set the step size to 2°-10°, and set the probability of whether each rotation direction is rotated to 0.5 to ensure the randomness of the image change, rotate the image and save it;

[0012] (4) Obtain a total of 4 times the number of original input images, with different pixels of 640×640 for semiconductor plate defect images, and perform batch renaming.

[0013] 2. Image defect annotation and conversion

[0014] (1) Mark the defect locations of semiconductor sheets and obtain image information corresponding to each defect;

[0015] (2) Write a format conversion algorithm to convert the defect information in the XML format into a tensor format. The coefficients before the tensor represent the defect type, which corresponds to the format required by the input end of the convolutional neural network.

[0016] (3) The defect image labels and tensor names are normalized and arranged in order of the same label name through a sorting algorithm.

[0017] 3. Random assignment of training, validation, and test images

[0018] (1) Write a random sampling algorithm, traverse the ordered images and corresponding labels through the random.Sample function, and sample with a random step size with a sampling ratio of 5-8:2-4:1;

[0019] (2) Determine the file path and save the defect information in the corresponding data set in the form of tensor.

[0020] 4. Second image enhancement processing

[0021] (1) Set the program to read the data of 4-9 pictures each time it normalizes the image parameters;

[0022] (2) Randomly select 4-9 images from the training set and perform random distribution, random scaling, and random splicing.

[0023] 5. Build a target detection neural network

[0024] (1) Adopting the adaptive anchor box calculation method, the predicted box is output based on the initial anchor box, and then compared with the ground truth box, the difference between the two is calculated, and then the network weight parameters are adjusted by reverse update and iteration. During each training, the best anchor box value in the training set is output in an adaptive manner;

[0025] (2) Input the preprocessed image with the size of 3x640x640, copy it 4 times, and then use the slicing operation to crop the copied image into 4 slices of 3x320x320;

[0026] (3) Use the concat function to connect the four slices in the depth direction of the image to obtain an output image with a size of 12x320x320;

[0027] (4) Then pass through the convolution layer with 32 convolution kernels to generate a 32x320x320 image, and then pass through the batch_borm function and leaky_relu function to input the output result into the next convolution layer;

[0028] (5) The backbone network adopts the Resnet residual network structure. After convolution with 32 convolution kernels, the residual structure is added to the initial input, and the depth of the model is controlled by the depth multiple function.

[0029] (6) The original input is divided into two branches, and convolution operations are performed on each branch to halve the number of channels. Then, the input branch 1 is multiplied by the bottleneck network. Then, the output is divided into output branch 1 and output branch 2 using the concat function, so that the input and output image feature sizes of the local network across stages are the same;

[0030] (7) Perform deconvolution and depooling on the high-level feature map, and use the direct addition method to fuse features. Use adaptive feature pooling to restore the destroyed information path between each candidate region and all feature levels, aggregate each candidate region on each feature level, and avoid arbitrary allocation;

[0031] (8) The feature image is input into the spp layer, output after passing through a 1x1 convolution layer, and then downsampled through three parallel Maxpool layers. The result is added to its initial feature and output.

[0032] 6. Data training to obtain the best weight parameters

[0033] (1) Input the preprocessed electroluminescent image of the semiconductor sheet and perform iterative training;

[0034] (2) In each iteration, calculate the loss function value, recall (positive class found / all positive classes that should have been found) value and mean average precision map (MeanAverage Precision) value, and draw a line chart;

[0035] (3) Obtain the optimal parameter weight file, overwrite and save the optimal parameters after each iteration, and obtain the optimal parameter set file for AI detection of plate images.

[0036] 7. AI detection of plate images

[0037] Input the EL image of the semiconductor plate directly taken by the near-infrared camera without pre-processing, and obtain the defect recognition result containing the defect type and location information.

[0038] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0039] (1) The present invention adopts AI detection technology, which can detect whether the plate has hidden cracks or black spots based on electroluminescent images, and locate and mark the defect positions.

[0040] (2) In the case where the image data set of the electroluminescent semiconductor plate is extremely limited, the present invention performs two image enhancement processes on the defect image, which significantly improves the richness of the data set and the recognition accuracy without affecting the accuracy of defect positioning.

[0041] (3) The present invention adopts a random classification algorithm for data sets, and performs tensor conversion and classification processing on the data between two image enhancements, thereby ensuring the exclusivity of the data used for training, verification and testing, thereby improving the stability of the detection model and the generalization ability of target recognition.

[0042] (4) The present invention adopts a method combining residual neural network, feature fusion method and cross-stage local network to construct a target detection network, which increases the model complexity and expands the network scale, while improving the recognition accuracy, enhancing the model robustness, reducing memory consumption and increasing the recognition speed.

[0043] (5) The present invention adopts a method of covering and saving the optimal weight configuration, which ensures the effectiveness of the training results and improves the recognition accuracy and recognition speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is the mean graph of the classification loss function;

[0045] Figure 2 is the recall rate;

[0046] Figure 3 is the average precision value;

[0047] Figure 4 It is the detection result of hidden crack photovoltaic panel;

[0048] Figure 5 It is the result of the black spot photovoltaic panel detection;

[0049] Figure 6 This is the detection result of photovoltaic panels with coexistence of hidden cracks and black spots. DETAILED DESCRIPTION

[0050] In order to make the purpose of the present invention, the principle and advantages of the technical solution clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation schemes. In this implementation scheme, the specific implementation scheme described is only used to explain the present invention and is not used to limit the present invention.

[0051] Example

[0052] Electroluminescence technology is used to take a picture on the surface of the silicon carbide semiconductor plate, and then defect detection is performed using the method of the present invention. The detection steps are as follows:

[0053] Electroluminescence technology is used to take photos on the surface of silicon semiconductor photovoltaic modules, and then defect detection is performed using the method described in the present invention. The detection steps are as follows:

[0054] 1. Perform the first image enhancement process

[0055] (1) Input 500 electroluminescent images of semiconductor sheets containing black spots and hidden cracks, and use the image cropping function to resize the input images into square images with pixels of 640×640;

[0056] (2) Design an image random flipping algorithm and set the probability of flipping the image horizontally and vertically to 0.5. Flip the image and save it;

[0057] (3) Design an image random rotation algorithm, set the step size to 5°, and set the probability of whether each rotation direction is rotated to 0.5 to ensure the randomness of image changes. Rotate the image and save it;

[0058] (4) A total of 2000 different images of semiconductor plate black spots and hidden crack defects with a pixel size of 640×640 are obtained, and a batch renaming operation from 0 to 2000 is performed.

[0059] 2. Perform image defect annotation and conversion

[0060] (1) Mark the positions of black spots on semiconductor sheets to obtain image information corresponding to each defect;

[0061] (2) Design a format conversion algorithm to convert the defect information in XML format into a tensor format. The coefficients before the tensor represent the defect type, which corresponds to the format required by the input end of the convolutional neural network.

[0062] (3) The defect image labels and tensor names are normalized and arranged in order of the same label name through a sorting algorithm.

[0063] 3. Random assignment of training, validation, and test images

[0064] (1) Design a random sampling algorithm, traverse the ordered images and corresponding labels through the random.Sample function, and sample with a random step size at a sampling ratio of 6:3:1;

[0065] (2) Determine the file path and save the defect information in the corresponding data set in the form of tensor.

[0066] 4. Perform the second image enhancement process

[0067] (1) Set the program to read the data of six images each time it performs image parameter normalization processing;

[0068] (2) Randomly select 6 images from the training set and perform random distribution, random scaling and random splicing.

[0069] 5. Build a target detection neural network

[0070] (1) Adopting the adaptive anchor box calculation method, the predicted box is output based on the initial anchor box, and then compared with the ground truth box, the difference between the two is calculated, and then the network weight parameters are adjusted by reverse update and iteration. During each training, the best anchor box value in the training set is output in an adaptive manner;

[0071] (2) Input a preprocessed image with a size of 3x640x640, copy it four times, and then use the slicing operation to crop the four copied images into four slices of 3x320x320;

[0072] (3) Use the concat function to connect the four slices in the depth direction of the image to obtain an output image with a size of 12x320x320;

[0073] (4) Then pass through the convolution layer with 32 convolution kernels to generate a 32x320x320 image, and then pass through the batch_borm function and leaky_relu function to input the output result into the next convolution layer;

[0074] (5) The backbone network adopts the Resnet residual network structure. After convolution with 32 convolution kernels, the residual structure is added to the initial input, and the depth of the model is controlled by the depth multiple function.

[0075] (6) The original input is divided into two branches, and convolution operations are performed on each branch to halve the number of channels. Then, the bottleneck network is multiplied on branch one, and the output is divided into branch one and branch two using the concat function, so that the input and output image feature sizes of the local network across stages are the same;

[0076] (7) Perform deconvolution and depooling on the high-level feature map, and use the direct addition method to fuse features. Use adaptive feature pooling to restore the destroyed information path between each candidate region and all feature levels, aggregate each candidate region on each feature level, and avoid arbitrary allocation;

[0077] (8) The feature image is input into the spp layer, output after passing through a 1x1 convolution layer, and then downsampled through three parallel Maxpool layers. The result is added to its initial feature and output.

[0078] 6. Data training to obtain the best weight parameters

[0079] (1) Input 2000 pre-processed electroluminescent images of semiconductor sheets and perform 100 iterations of training;

[0080] (2) In each iteration, calculate the loss function value, recall value and map (mean Average Precision) value, and draw a line chart;

[0081] (3) Obtain the optimal parameter weight file, overwrite and save the optimal parameters after each iteration, and obtain the optimal parameter set file for AI detection of plate images.

[0082] 7. AI detection of plate images

[0083] Input the EL image of the semiconductor plate directly taken by the near-infrared camera without pre-processing, and obtain the defect recognition result including the defect type and location information. The average recognition time for each 640×640 pixel image is 0.016 seconds.

[0084] Test results: Figure 1-Figure 6 As shown in the figure, after more than 70 iterations, the recall rate (correct positive class / all positive classes that should have been found) and the map value reached about 99.5% and tended to be stable. The loss function value dropped to 0.02 after 100 iterations, which shows that the model has excellent defect recognition ability. Not only can the photovoltaic panels with hidden cracks and black spots exist alone, but also the photovoltaic panels with hidden cracks and black spots coexisting are detected, and the defects are marked with text to facilitate the analysis of the results.

Claims

1. An AI detection method for surface defects of electroluminescent semiconductor sheets, characterized in that The method comprises the following steps in order: (1) first image enhancement processing; (2) image defect annotation and conversion; (3) random allocation of training, verification and test images; (4) second image enhancement processing; (5) establishment of target detection neural network; (6) data training to obtain optimal weight parameters; (7) AI detection of plate images; The first image enhancement process comprises the following steps in order: (1) inputting an electroluminescent image of a semiconductor plate containing defects, and using an image cropping function to adjust the size of the input image to a square image with pixels of 640×640; (2) writing an image random flipping algorithm, setting the probability of flipping the image in the horizontal and vertical directions to 0.5, flipping the image and saving it; (3) writing an image random rotation algorithm, setting the step size to 2°-10°, and setting the probability of whether each rotation direction is rotated to 0.5 to ensure the randomness of the image change, rotating the image and saving it; (4) obtaining a total of 4 times the number of original input images, which are different from each other and have pixels of 640×640 and are semiconductor plate defect images, and performing batch renaming; The second image enhancement process comprises the following steps in order: (1) setting the program to read data of 4-9 pictures each time the image parameters are normalized; (2) randomly selecting 4-9 pictures from the training set, and performing random distribution, random scaling and random splicing processes.

2. The AI ​​detection method according to claim 1, characterized in that The image defect annotation and conversion includes the following steps in order: (1) annotating the defect positions of the semiconductor plate to obtain the image information corresponding to each defect; (2) writing a format conversion algorithm to convert the defect information in the extensible markup language format into a tensor format, where the coefficients before the tensor represent the defect type and correspond to the format required by the input end of the convolutional neural network; (3) normalizing the defect image labels and tensor names, and arranging them in order of the same label name through a sorting algorithm.

3. The AI ​​detection method according to claim 1, characterized in that The random allocation of training, validation and test images includes the following steps in order: (1) writing a random sampling algorithm, traversing the ordered images and corresponding labels through the random.Sample function, sampling with a random step size at a sampling ratio of 5-8:2-4:1; (2) determining the file path and saving the defect information in the form of a tensor in the corresponding data set.

4. The AI ​​detection method according to claim 1, characterized in that The object detection neural network described in the invention comprises the following steps in order: (1) using an adaptive anchor frame calculation method, outputting a predicted frame based on an initial anchor frame, and then comparing it with a ground truth frame, calculating the difference between the two, and then reversely updating and iterating to adjust the network weight parameters; each time during training, outputting the best anchor frame value in the training set in an adaptive manner; (2) Input a preprocessed image of size 3x640x640, copy it 4 times, and then use the slicing operation to crop the copied image into 4 slices of size 3x320x320; (3) Use the concat function to connect the 4 slices in the image depth direction to obtain an output image of size 12x320x320; (4) Generate an image of size 32x320x320 through a convolution layer with 32 convolution kernels, and then input the output result to the next convolution layer through batch_borm function and leaky_relu function; (5) The backbone network adopts the Resnet residual network structure. After convolution of the convolution layer with 32 convolution kernels, it is added to the initial input through the residual structure, and the depth of the model is controlled by the depthmultiple function. ; (6) Divide the original input into two branches, perform convolution operations on each branch to halve the number of channels, then input branch one to perform bottleneck network doubling operation, and then use the concat function to divide the output into output branch one and output branch two, so that the input and output image feature sizes of the local network across stages are the same; (7) Perform deconvolution and depooling upsampling on the high-level feature map, and use the direct addition method to fuse the features. Use adaptive feature pooling to restore the destroyed information path between each candidate region and all feature levels, aggregate each candidate region on each feature level, and avoid arbitrary allocation; (8) Input the feature image in the spp layer, output it after passing through the 1x1 convolution layer, and then downsample it through three parallel Maxpools. Add the result to its initial feature and output the result.

5. The AI ​​detection method according to claim 1, characterized in that The data training to obtain the optimal weight parameters includes the following steps in order: (1) inputting the pre-processed electroluminescent image of the semiconductor plate and performing iterative training; (2) in each iteration process, calculating the loss function value, the recall value and the average precision map value, and drawing a line chart; (3) Obtain the optimal parameter weight file, overwrite and save the optimal parameters after each iteration, and obtain the optimal parameter set file for AI detection of plate images.

6. The AI ​​detection method according to claim 1, characterized in that The AI ​​detection of plate images is as follows: an EL image of a semiconductor plate directly taken by a near-infrared camera without preprocessing is input to obtain a defect recognition result including defect type and location information.

Images