Chip surface defect recognition method based on information entropy convolutional neural network

Abstract

The present application relates to image processing technology, a chip surface defect recognition method based on information entropy convolutional neural network, a data set is constructed: a preset category to which the chip belongs is obtained by using chip production process, an optical image of the wafer surface of the chip sample is collected by using a visible light imaging device to construct a data set, a training set and a test set are divided, and a convolutional neural network initial model is constructed and trained: the images of the training set are input into the convolutional neural network initial model, the cross-entropy loss function is used as the optimization target for training, and the convolutional neural network intermediate model is obtained, the information entropy convolutional neural network initial model is constructed, the information entropy convolutional neural network initial model is trained, and the chip surface defect recognition: the images of the test set in the data set are input into the information entropy convolutional neural network final model, and the recognition result is output by the output layer, so that the chip surface defect recognition is completed.

Description

TECHNICAL FIELD

[0001] The present application relates to image processing technology, in particular to a chip surface defect recognition method based on information entropy convolutional neural network. BACKGROUND

[0002] Chip surface defect recognition is crucial to control its production capacity and its product quality. Using convolutional neural network to process chip surface optical image instead of chip performance test in traditional production process can improve the efficiency of chip surface defect recognition.

[0003] However, the performance of convolutional neural network is heavily dependent on sample quality. Affected by industrial cost and process precision, chip surface optical image has low resolution, defect features are not obvious, and there is little feature information available for model learning; it is difficult to collect a large number of defect samples, and the number of defect-free samples is much larger than the number of defect samples, which misleads the model learning direction. The current convolutional neural network model has not yet specifically addressed the impact of chip sample features. Therefore, it has become a problem to be solved in the field to research and design a new convolutional neural network model. SUMMARY

[0004] In order to solve the problems in the prior art, the present application provides a chip surface defect recognition method based on information entropy convolutional neural network, which has the advantages of high recognition accuracy, high speed, good robustness, and can be used for identifying defective chips with surface defects in chip production process.

[0005] In order to achieve the above purpose, the present application provides a chip surface defect recognition method based on information entropy convolutional neural network, which comprises the following steps:

[0006] (1) Constructing a data set: using chip production process to obtain a preset category to which the chip belongs, using a visible light imaging device to take pictures to collect optical images of the wafer surface of the chip sample, to construct a data set, and dividing the training set and the test set:

[0007] (2) Constructing a convolutional neural network initial model and training: constructing a convolutional neural network initial model including an input layer, a convolutional layer, a down-sampling layer, an activation layer, a full connection layer and an output layer, the input layer, the convolutional layer, the down-sampling layer, the activation layer, the full connection layer and the output layer are arranged in sequence, the convolutional layer, the down-sampling layer and the activation layer are multiple and are arranged alternately, and the images of the training set of the data set constructed in step (1) are input into the convolutional neural network initial model, and the cross-entropy loss function is used as the optimization objective to train, to obtain a convolutional neural network intermediate model;

[0008] (3) Constructing the initial model of the information entropy convolutional neural network: Based on the intermediate model of the convolutional neural network obtained in step (2), a feature fusion module guided by information entropy is added. The feature fusion module guided by information entropy includes determining the introduced edge information and texture information, using feature entropy consistency to select the features to be fused and using the global attention mechanism to perform feature fusion. Feature entropy consistency is the variance of the information entropy of the feature map. Construct the initial model of the information entropy convolutional neural network.

[0009] (4) Train the initial model of the information entropy convolutional neural network: Input the images of the training set in the dataset constructed in step (1) into the initial model of the information entropy convolutional neural network constructed in step (3), and train it with the gradient-weighted cross-entropy loss function as the optimization objective to obtain the final model of the information entropy convolutional neural network.

[0010] (5) Chip surface defect identification: The images of the test set in the dataset constructed in step (1) are input into the final model of the information entropy convolutional neural network obtained in step (4). The image features are extracted through convolutional layer, downsampling layer, activation layer, feature fusion module and fully connected layer. The identification result is output by the output layer. Chip surface defect identification is completed in this way.

[0011] The optical image of the chip sample wafer surface is captured by a visible light imaging device to obtain the image to be identified. The image to be identified is input into the final model of the information entropy convolutional neural network obtained in step (4). The image features are extracted through convolutional layer, downsampling layer, activation layer, feature fusion module and fully connected layer. The recognition result is output by the output layer. In this way, the chip surface defect identification is completed.

[0012] As a further preferred option, the preset categories mentioned in step (1) include the following two categories: normal chips and defective chips.

[0013] As a further preferred embodiment, the cross-entropy loss function described in step (2) is specifically as follows: For each sample in a dataset with a total of 2 categories, i is one of the 2 categories, i = 0 represents the normal chip category, i = 1 represents the defective chip category, j is the true category of the sample, and y is the label description of the sample. When the i-th component y of the sample label... i If the value is the true label of the sample, it is 1; otherwise, it is 0.

[0014]

[0015] The sample input model yields feature descriptors z, z i Let 'a' be the i-th component of the feature descriptor z, 'a' be the result of the feature descriptor z after passing through the softmax function, and 'e' be the natural logarithm. The i-th component of 'a' is 'a'. iThis represents the class confidence score of the model predicting that the sample belongs to class i.

[0016]

[0017] Cross-entropy loss of a single sample ce The calculation is as follows:

[0018]

[0019] As a further preferred embodiment, step (2) involves inputting the images from the training set of the dataset constructed in step (1) into the initial model of the convolutional neural network, and training it with the cross-entropy loss function as the optimization objective to obtain the intermediate model of the convolutional neural network. Specifically, this is as follows:

[0020] The images in the training set are randomly sorted and divided into multiple batches according to a preset number of batches;

[0021] Initialize the parameters of the initial convolutional neural network model constructed in step (2), and then input the images in the training set into the initial convolutional neural network model with these parameters in batches. Iterate the training with the cross-entropy loss function as the optimization objective, and obtain the final convolutional neural network model after the iteration training is completed.

[0022] As a further preferred embodiment, the determination of the introduced edge information and texture information in step (3) specifically involves: extracting the edge information F of the input image using the Laplacian operator and the local binary mode operator, respectively. e and texture information F t As introduced edge information and texture information.

[0023] As a further preferred embodiment, step (3) involves selecting features to be fused using feature entropy consistency. Feature entropy consistency is the variance of the information entropy of the feature map, specifically:

[0024] First, construct the edge information F. e and texture information F t The feature pyramid has L+1 layers, where L is the number of downsampling layers in the initial model of the convolutional neural network described in step (2), and the Gaussian kernel K is used for edge information F. e and texture information F t After convolution, it undergoes L downsampling pooling (·) to obtain F. e and F t Features at any level l (0≤l≤L) in the feature pyramid and Calculation formula, pool l (·) indicates that l consecutive downsampling operations are performed:

[0025]

[0026] Then the convolutional neural network feature list List cnn and edge and texture information feature list List e，t , the convolutional neural network feature list List cnn is composed of the layer feature set of all convolutional layers, down-sampling layers and activation layers of the convolutional neural network intermediate model in step (2) k layers, that is, it satisfies h t represents the number of channels of the t-th (0≤t≤k-1) layer, and the layer feature set of the t-th layer is composed of the feature maps extracted from the h t channels of the layer, which satisfies represents the feature map of the h-th (0≤h≤h t ) channel of the t-th layer, and the feature maps of the t-th layer have the same size Edge and texture information feature list List e，t is composed of the layer feature set of all layers in the L+1 layer feature pyramid of the edge information F e and the texture information F t , that is, it satisfies The edge and texture information layer feature set of the l-th layer is composed of the feature maps of the edge information and the texture information corresponding to the layer, which satisfies The edge information and the texture information have the same size

[0027] Finally, the feature set group with the optimal feature entropy consistency is obtained from the convolutional neural network feature list List cnn and the edge and texture information feature list List e，t using feature entropy consistency. The image X has N pixel values in total, and the probability of the occurrence of pixel value n (0≤n≤N-1) is p n , and the information entropy En of the image X is:

[0028]

[0029] The feature sets with the same feature map size are selected from the neural network feature list List cnn and the edge and texture information feature list List e，t respectively and satisfy:

[0030]

[0031] 0≤t≤k-1

[0032] 0≤l≤L

[0033] Calculate the feature set and CCP h t Information entropy and average information entropy of the two feature maps And calculate the feature entropy consistency Ec among these feature maps:

[0034]

[0035] Record the feature sets and their corresponding feature entropy consistency to obtain a feature set G with optimal (minimum) feature entropy consistency. cnn and G e，t G cnn It is the feature set of the convolutional neural network feature layer with optimal feature entropy consistency, G e，t It is the corresponding set of edge information and texture information layer features.

[0036] More preferably, the feature fusion using the global attention mechanism in step (3) specifically involves: for the feature group G with optimal feature entropy consistency obtained in step (6.3) cnn and G e，t , for G cnn The results obtained by performing global pooling (·) and convolution (·) are the same as those obtained by G. e，t Multiply the results after performing convolution Conv(·) and then add G. cnn The feature fusion result Y is obtained:

[0037] Y = Conv(Globalpooling(G cnn ))×Conv(G e，t )+G cnn

[0038] This enables feature fusion using a global attention mechanism.

[0039] Further preferred, the gradient-weighted cross loss function mentioned in step (4) specifically involves: extracting the intermediate model of the convolutional neural network in step (2), and extracting the gradient g of the backpropagation of the model with the cross-entropy loss function as the optimization objective. ce :

[0040]

[0041] Infer all samples in the training set of the data set constructed in step (1) in the intermediate model of the convolutional neural network in step (2), and perform back propagation with the cross-entropy loss function as the optimization objective to obtain a gradient set Grad ce Divide the interval [0, 1] into m equal length gradient subintervals, and count the number of gradients in the gradient set Grad ce | falling into the g-th (1≤g≤m) gradient subinterval u g , calculate the average number of gradients in each gradient subinterval

[0042]

[0043] The gradient subinterval of the gradient of a single sample s is g s , and the number of gradients in this interval is Then the gradient weighting factor a of sample s s :

[0044]

[0045] The gradient-weighted cross-entropy loss loss of a single sample s wce is calculated as:

[0046]

[0047] Further preferably, in step (4), the images of the training set of the data set constructed in step (1) are input into the information entropy convolutional neural network initial model constructed in step (3), and the gradient-weighted cross-entropy loss function is used as the optimization objective to obtain the information entropy convolutional neural network final model, which is specifically:

[0048] Randomly sort the images in the training set and divide them into multiple batches according to a predetermined batch number;

[0049] Initialize the parameters of the information entropy convolutional neural network initial model constructed in step (3), then input the images in the training set into the parameter-initialized convolutional neural network model in batches, and perform iterative training with the gradient-weighted cross-entropy loss function as the optimization objective. After the iterative training is completed, the final convolutional neural network model is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A chip surface defect recognition method based on an information entropy convolutional neural network is provided for the embodiments of the present application.

[0051] Figure 2Part of the surface images of Mini LED chips of the data set of the embodiment of the present application, (a) is a surface image of Mini LED chips of part of the normal chip category, (b) is a surface image of Mini LED chips of part of the defective chip category;

[0052] Figure 3 The Mini-LED-CNN structure diagram constructed in the embodiment of the present application;

[0053] Figure 4 The edge information and texture information of part of the surface images of Mini LED chips of the data set of the embodiment of the present application, (a) is the edge information and texture information of Mini LED chips of part of the normal chip category, (b) is the edge information and texture information of Mini LED chips of part of the defective chip category;

[0054] Figure 5 The Mini-LED-ENCNN structure diagram constructed in the embodiment of the present application; DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0057] The specific scheme of the chip defect recognition method based on information entropy convolutional neural network provided by the present application will be specifically described below in combination with the drawings and the embodiment of Mini LED chip surface defect recognition.

[0058] Please refer to Figure 1 which shows a chip surface defect recognition method flowchart based on information entropy convolutional neural network provided by an embodiment of the present application, and the method comprises the following steps:

[0059] (1) Step s1, obtaining the preset category to which the chip belongs by using the chip production process, collecting the optical image of the surface of the chip sample to construct a data set, and dividing it into a training set and a test set.

[0060] Specifically, 5000 chip surface optical images collected by the visible light imaging device are used to obtain the categories of the chips, including normal chips and defective chips, to form a dataset Mini-LED-DS. The chip image format is PNG, and the chip image is a single-channel grayscale image. Some chip images are as shown in Figure 2 (a)-(b), and each chip image is normalized to the same pixel size of 32x32 (unit: pixel). The images are randomly selected from the original dataset at a ratio of 4:1 to form a training set and a test set, as shown in Table 1. Each chip image is labeled as a normal chip or a defective chip. The normal chip is represented by 0, and the expected output is (1, 0). The defective chip is represented by 1, and the expected output is (0, 1).

[0061] Table 1: Dataset Mini-LED-DS

[0062] Type Training set Test set Total Normal chips 3680 920 4600 Defective chips 320 80 400 Total 4000 1000 5000

[0063] (2) Step s2, an initial model of the convolutional neural network is constructed, the training set is used as the input, and the cross-entropy loss function is used as the optimization target to train the initial model, so as to obtain an intermediate model of the convolutional neural network.

[0064] Specifically, in this embodiment, an initial model Mini-LED-CNN of the convolutional neural network is constructed, and the structure thereof is as shown in Figure 3 It includes 1 input layer, 4 learning layers, and 1 output layer. The feature map size of the input layer is 32x32. The 4 learning layers are 3 convolutional layers and 1 fully connected layer. A max-pooling layer and an activation layer are arranged after the 1st and 2nd convolutional layers. The convolution kernel size of the convolutional layers is 3x3, the feature map padding width is 1, and the step is 1. The output channel of the 1st convolutional layer is h1=6, the output channel of the 2nd convolutional layer is h2=12, and the output channel of the 3rd convolutional layer is h3=24. The pooling window size of the max-pooling layers is 2x2, and the step is 2. The activation layers are linear rectifier functions. The number of neurons of the output layer is 3. The output feature size of the 1st layer convolutional layer is 32x32. The output feature size of the 1st layer max-pooling layer, the 1st layer activation layer, and the 2nd layer convolutional layer is 16x16. The output feature size of the 2nd layer max-pooling layer, the 2nd activation layer, and the 3rd convolutional layer is 8x8.

[0065] Mini-LED-CNN is constructed in the Pytorch environment. The images in the training set are randomly sorted, and the training set is divided into 63 batches according to 64 as 1 batch. The number of images in the last batch is 32. The training images are input into the Mini-LED-CNN model in batches, the model parameters are initialized, and when all the samples in a batch are calculated, the cross-entropy loss function Calculate the loss value and perform a weight update using backpropagation. After all batches are updated, proceed to the next iteration. The maximum number of iterations for training samples is 200. After iteration, a parametric network model is obtained, named Mini-LED-CNN200model. The trained model is tested using the test set, and various recognition results are obtained, which are listed in Table 2.

[0066] Table 2. Classification Results of Mini-LED-CNN200 Models

[0067] Normal chips Defective chips Total Number 920 80 1000 Recognized number 876 52 928 Misrecognition rate 8％ 35％ 7.2％

[0068] (3) Step s3: Based on the intermediate model of the convolutional neural network, add a feature fusion module guided by information entropy to construct the initial model of the information entropy convolutional neural network.

[0069] Specifically, the edge information F of all training set inputs is first obtained using the Laplacian operator and the local binary mode operator. e and texture information F t As edge and texture information introduced into the model, some results are as follows: Figure 4 As shown. Secondly, using the formula:

[0070]

[0071] A three-layer feature pyramid is constructed to represent edge and texture information, from bottom to top as layer 0, layer 1, and layer 2. The feature map size of layer 0 is 32×32, layer 1 is 16×16, and layer 2 is 8×8. Then, the training set is input into the Mini-LED-CNN200 model to obtain feature maps from all convolutional layers, max-pooling layers, and activation layers to construct a convolutional neural network feature list. The feature pyramid is then used to construct feature lists for edge and texture information. Finally, the average information entropy of the feature maps from the 1st, 2nd, and 3rd convolutional layers, the 1st and 2nd max-pooling layers, and the 1st and 2nd activation layers in the convolutional neural network feature list is calculated. Calculate the average information entropy of the edge features and texture features in layers 1-3 of the feature pyramid, respectively. Based on the relationship between feature map sizes, utilize... Calculate the feature consistency between the 6 features of the first convolutional layer and the 8 features (edge ​​features and texture features) of the 0th layer; utilize The feature consistency among the 12 features of the first max pooling layer, the first activation layer, and the second convolutional layer, and the 14 features of the first edge feature and texture feature, is calculated separately; using... Feature consistency was calculated among 24 features from the second max pooling layer, the second activation layer, and the third convolutional layer, and among 26 features from the edge and texture features of the second layer. The results are shown in Table 3.

[0072] Table 3. Calculation results of feature consistency

[0073]

[0074]

[0075] The calculation shows that the feature map of the second convolutional layer has the optimal feature consistency with the edge and texture features of the first layer. Therefore, in this embodiment, the feature map of the second convolutional layer and the edge and texture features of the first layer of the feature pyramid are selected for feature fusion.

[0076] In this embodiment, an initial model of the information entropy convolutional neural network, Mini-LED-ENCNN, is constructed, and its structure is as follows: Figure 4 As shown, it is in Figure 3 A feature fusion module was added to the basic structure. After the input layer, a branch is formed. In the branch network, the Laplacian operator and the local binary pattern operator are used to perform sliding convolution on the input image to obtain 32×32 layer 0 edge and texture features. These are then passed through a Gaussian kernel and a downsampling layer to obtain 16×16 layer 1 edge and texture features. The edge and texture features are merged into a 2-channel 16×16 first feature group. This feature group is then passed through a 4th convolutional layer to obtain a 12-channel 16×16 second feature group. In the main branch network, a branch is formed at the 2nd convolutional layer. The branch network undergoes a global average pooling to obtain a 12-channel 1×1 second feature group, which is then passed through a 5th convolutional layer to obtain a 12-channel 1×1 fourth feature group. The second and fourth feature groups are multiplied to obtain a 12-channel 16×16 fifth feature group. The fifth feature group is then added to the features from the 2nd convolutional layer to obtain the final fused feature group, which is returned to the main network and then connected to a second max pooling layer. The kernel size of the 4th convolutional layer is 3×3, the feature map padding width is 1, and the stride is 1; the kernel size of the 5th convolutional layer is 1×1, and the stride is 1.

[0077] (4) Step s4 uses the training set as input and the gradient-weighted cross-entropy loss function as the optimization objective to train the initial model of the information entropy convolutional neural network to obtain the final model of the information entropy convolutional neural network.

[0078] Specifically, the training set is input into the model for forward inference to obtain the gradients of all training sets during backpropagation. The interval [0,1] is divided into 20 gradient sub-intervals, and the number of samples in each gradient sub-interval is counted. The results are shown in Table 4.

[0079] Table 4. Number of samples within the gradient subinterval

[0080] Interval 1 2 3 4 5 6 7 8 9 10 Sample 2803 748 190 68 30 14 10 8 9 5 Interval 11 12 14 14 15 16 17 18 19 20 Sample 4 5 6 6 8 14 9 12 24 27

[0081] Gradient sub-interval average sample is 20, and the gradient weighting value of each sub-interval is obtained, and the results are shown in Table 5.

[0082] Table 5 Gradient weighting value

[0083] Interval 1 2 3 4 5 6 7 8 9 10 α 0.010 0.038 0.143 0.367 0.718 1.222 1.494 1.688 1.584 2.115 Interval 11 12 13 14 15 16 17 18 19 20 α 2.321 2.115 1.947 1.947 1.688 1.222 1.584 1.343 0.847 0.777

[0084] The gradient weighted cross-entropy loss function is obtained as follows The gradient weighting value a of each sample in the formula is s .

[0085] In the Pytorch environment, the Mini-LED-ENCNN is constructed. The training images are input into the Mini-LED-ENCNN model in batches, the model parameters of the feature fusion module are initialized, and the remaining parameters are imported into the model parameters of the Mini-LED-CNN200 model. After all the samples in a batch are calculated, the loss value is calculated using the gradient weighted cross-entropy loss function , and the weight is updated once using the back propagation mechanism. After all batches are updated, the next iteration is performed, and the maximum iteration number of the training sample is 200 times. After the iteration is completed, the parametric network model is obtained, which is named Mini-LED-ENCNN200 model.

[0086] (5) Step s5, input the test set (or the image to be recognized, in this patent, the test set is taken as the image to be recognized) into the final model of the information entropy convolutional neural network to extract features, and output the recognition result. In this way, the chip defect recognition is completed.

[0087] Specifically, the test level is input into the Mini-LED-ENCNN200 model to obtain the recognition result of each class, and the results are listed in the table.

[0088] Table Mini-LED-ENCNN200 model recognition effect

[0089] Normal chips Defective chips Total Number 920 80 1000 Recognized number 920 78 998 Misrecognition rate 0 2.5％ 0.2％

[0090] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying chip surface defects based on information entropy convolutional neural networks, characterized in that, The method includes the following: (1) Dataset Construction: The chip manufacturing process is used to obtain the preset category to which the chip belongs. Optical images of the chip sample wafer surface are captured using a visible light imaging device to construct the dataset, which is then divided into a training set and a test set. (2) Constructing and training an initial convolutional neural network model: Construct an initial convolutional neural network model including an input layer, a convolutional layer, a downsampling layer, an activation layer, a fully connected layer, and an output layer. The input layer, convolutional layer, downsampling layer, activation layer, fully connected layer, and output layer are set sequentially. There are multiple convolutional layers, downsampling layers, and activation layers, and they are set alternately. Input the images of the training set of the dataset constructed in step (1) into the initial convolutional neural network model, and train it with the cross-entropy loss function as the optimization objective to obtain an intermediate convolutional neural network model. (3) Constructing the initial model of the information entropy convolutional neural network: Based on the intermediate model of the convolutional neural network obtained in step (2), a feature fusion module guided by information entropy is added. The feature fusion module guided by information entropy includes determining the introduced edge information and texture information, using feature entropy consistency to select the features to be fused and using the global attention mechanism to perform feature fusion. Feature entropy consistency is the variance of the information entropy of the feature map. Construct the initial model of the information entropy convolutional neural network. (4) Training the initial model of the information entropy convolutional neural network: Input the images from the training set in the dataset constructed in step (1) into the initial model of the information entropy convolutional neural network constructed in step (3), and train it with the gradient-weighted cross-entropy loss function as the optimization objective to obtain the final model of the information entropy convolutional neural network; the gradient-weighted cross-entropy loss function is as follows: Extract the intermediate model of the convolutional neural network in step (2), and extract the gradient g of the backpropagation of the model with the cross-entropy loss function as the optimization objective. ce : The training set of the dataset from step (1) is used for inference in the intermediate model of the convolutional neural network in step (2), and the gradient set Grad is obtained during backpropagation with the cross-entropy loss function as the optimization objective. ce Divide the interval [0,1] into m equal-length gradient subintervals, and count the absolute value of the gradient |g| in the gradient set. ce The number of gradients falling within the g-th (1≤g≤m) gradient subinterval u g Calculate the average number of gradients within each gradient subinterval. The gradient subinterval containing the gradient of a single sample s is g. s The number of gradients in this interval is Then the gradient weighting factor α of sample s s : Gradient-weighted cross-entropy loss for a single sample s wce The calculation is as follows: Among them, y i Let 'a' be the i-th component of the sample label, and 'z' be the result of the feature descriptor z after passing through the softmax function. i Let a be the i-th component; (5) Chip surface defect identification: Input the images of the test set in the dataset constructed in step (1) into the final model of the information entropy convolutional neural network obtained in step (4). The image features are extracted through convolutional layer, downsampling layer, activation layer, feature fusion module and fully connected layer. The identification result is output by the output layer. Chip surface defect identification is completed in this way.

2. The chip surface defect identification method based on information entropy convolutional neural network as described in claim 1, characterized in that: The preset categories mentioned in step (1) include two categories: normal chips and defective chips.

3. The chip surface defect identification method based on information entropy convolutional neural network as described in claim 1, characterized in that: The cross-entropy loss function mentioned in step (2) is as follows: For each sample in a dataset with a total of 2 categories, i is one of the 2 categories, i = 0 represents the normal chip category, i = 1 represents the defective chip category, j is the true category of the sample, and y is the label description of the sample. When the i-th component y of the sample label... i When the label is the true label of the sample, its value is 1; otherwise, its value is 0. The sample input model yields feature descriptors z, z i Let 'a' be the i-th component of the feature descriptor z, 'a' be the result of the feature descriptor z after passing through the softmax function, and 'e' be the natural logarithm. The i-th component of 'a' is 'a'. i This represents the class confidence score of the model predicting that the sample belongs to class i. Cross-entropy loss of a single sample ce The calculation is as follows:

4. The chip surface defect identification method based on information entropy convolutional neural network as described in claim 1, characterized in that: Step (2) involves inputting the training set images from the dataset constructed in step (1) into the initial model of the convolutional neural network, and training it using the cross-entropy loss function as the optimization objective to obtain the intermediate model of the convolutional neural network. (4.1) Randomly sort the images in the training set and divide them into multiple batches according to the preset batch number; (4.2) Initialize the parameters of the initial convolutional neural network model constructed in step (2), and then input the images in the training set into the initial convolutional neural network model after parameter initialization in batches. Iterative training is performed with the cross-entropy loss function as the optimization objective. After the iterative training is completed, the intermediate convolutional neural network model is obtained.

5. The chip surface defect identification method based on information entropy convolutional neural network as described in claim 1, characterized in that: The determination of the introduced edge information and texture information in step (3) specifically involves: extracting the edge information F of the input image using the Laplacian operator and the local binary mode operator, respectively. e and texture information F t As introduced edge information and texture information.

6. The chip surface defect identification method based on information entropy convolutional neural network as described in claim 1, characterized in that: In step (3), the feature entropy consistency is used to select the features to be fused. Feature entropy consistency is the variance of the information entropy of the feature map, specifically: (6.1) First, construct the edge information F e and texture information F t The feature pyramid has L+1 layers, where l is the number of downsampling layers in the initial model of the convolutional neural network described in step (2), and the Gaussian kernel K is used for edge information F. e and texture information F t After convolution, it undergoes L downsampling pooling (·) to obtain F. e and F t Features at any level l (0≤l≤L) in the feature pyramid and Calculation formula, pool l (·) indicates that l consecutive downsampling operations are performed: (6.2) Then construct the feature list of the convolutional neural network. cnn List of features containing edge and texture information e,t Convolutional Neural Network Feature List cnn It consists of the feature set of all k layers (convolutional layers, downsampling layers, and activation layers) of the intermediate model of the convolutional neural network in step (2), that is, it satisfies h t This represents the number of channels in the t-th layer (0≤t≤k-1), and the layer feature set of the t-th layer. From the h of this layer t The feature maps extracted from each channel are combined to satisfy both Denotes the h-th level of the t-th layer (0≤h≤h) t The feature maps of the t-th layer have the same size. Feature list of edge and texture information e,t The edge information F constructed in step (6.1) e and texture information F t The feature set of all layers in the L+1 layer feature pyramid constitutes the feature set, that is, satisfying The feature set of the edge and texture information layer of layer l The edge information corresponding to this layer and texture information The feature maps are combined to satisfy both Edge information and texture information Having the same size (6.3) Finally, feature entropy consistency is used to extract features from the convolutional neural network feature list List. cnn List of features containing edge and texture information e,t To obtain the feature set group with optimal feature entropy consistency, there are N possible pixel values ​​for image X, and the probability of pixel value n (0≤n≤N-1) appearing is p. n The information entropy En of image X is: From the feature list of the convolutional neural network respectively cnn List of features containing edge and texture information e,t Select layer feature sets with the same feature map size. and satisfy: 0≤t≤k-1 0≤l≤L Computational layer feature set and CCP h t The information entropy En and average information entropy of the two feature maps. And calculate the feature entropy consistency Ec among these feature maps: Record the feature groups and their corresponding feature entropy consistency to obtain the layer feature set G with optimal feature entropy consistency. cnn and G e,t G cnn It is the feature set of the convolutional neural network feature layer with optimal feature entropy consistency, G e,t It is the corresponding set of edge information and texture information layer features.

7. The chip surface defect identification method based on information entropy convolutional neural network as described in claim 6, characterized in that: The feature fusion using the global attention mechanism in step (3) specifically involves: for the layer feature set G with optimal feature entropy consistency obtained in step (6.3) cnn and G e,t , for G cnn The results obtained by performing global pooling (·) and convolution (·) are the same as those obtained by G. e,t Multiply the results after performing convolution Conv(·) and add G. cnn The feature fusion result Y is obtained: Y=Conv(Globalpooling(G cnn ))×Conv(G e,t )+G cnn This enables feature fusion using a global attention mechanism.

8. The chip surface defect identification method based on information entropy convolutional neural network as described in claim 1, characterized in that: In step (4), the images from the training set in the dataset constructed in step (1) are input into the initial model of the information entropy convolutional neural network constructed in step (3), and trained with the gradient-weighted cross-entropy loss function as the optimization objective to obtain the final model of the information entropy convolutional neural network. Specifically, the process is as follows: (9.1) Randomly sort the images in the training set and divide them into multiple batches according to a preset batch number; (9.2) Initialize the parameters of the initial model of the information entropy convolutional neural network constructed in step (3), and then input the images in the training set into the initial model of the information entropy convolutional neural network after parameter initialization in batches. Iterative training is performed with gradient-weighted cross-entropy loss function as the optimization objective. After the iterative training is completed, the final model of the information entropy convolutional neural network is obtained.

Images