Electronic circuit defect detection method based on non-negative constraint sparse autoencoder

Abstract

The application discloses an electronic circuit defect detection method based on a non-negative constraint sparse self-encoder, which comprises the following steps: collecting and clipping an electronic circuit image, enhancing image data and extracting features, determining a defect data set and a defect type. The electronic circuit image is denoised based on a group-based graph model and a kernel specification image denoising method, a Laplace graph matrix of the image is obtained through an optimization learning strategy, and the Laplace graph matrix is used for encapsulating a graph structure. A deep learning model of a non-negative constraint sparse self-encoder is proposed and used for extracting a defect area. A circuit image is predicted by using a successfully trained self-encoder model, the predicted image is subtracted from an original defect input image, a defect detection image is generated, and a correct defect type is obtained by setting a proper threshold. Compared with the prior art, the application can effectively detect a real defect of an electronic circuit with a large color change and a pseudo defect with a small color change, and improve the reliability and precision of electronic circuit defect identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic circuit defect classification, and particularly relates to an electronic circuit defect detection method based on a non-negative constraint sparse autoencoder. BACKGROUND

[0002] It is well known that electronic circuit boards mechanically support the connection of electronic components through conductive lines, pads and solder joints. With the progress of science and technology and the rapid development of the mobile electronic product market, electronic circuit boards are more diversified and complex, and more electronic devices are integrated into electronic circuit boards, and the layout of electronic circuit boards is increasing, and the problems caused thereby are increasingly obvious. A typical problem is that defects in the signal transmission tracks on the circuit board have a huge impact on the entire system, thereby causing the performance of the connected electronic components to decrease, and these electronic components are the key to affecting the function of the entire system, and thus will eventually cause circuit failure and defects in the performance of the circuit system. At present, it is difficult to identify the small defects of electronic circuit boards by means of human eye observation, and therefore, the research demand for an electronic circuit board defect automatic detection method is very urgent, which is one of the most important ways to control the quality of the circuit.

[0003] The defect detection method of electronic circuit boards can be generally divided into two categories: direct detection method and camera-based machine vision method. The direct detection method is a manual operation inspection, which allows the operator to use visual inspection to easily perform. However, the operator is easily tired due to repetitive work, and the detection results of each person are inconsistent, which is a fundamental limitation of human judgment. In order to overcome these limitations, researchers have studied defect detection based on machine vision, and the defect detection system generally includes a camera, a light source and an operating system. The main purpose is to use an automatic optical detection system to improve the detection quality, and generally uses an industrial camera such as a charge-coupled device (CCD) to obtain high-quality circuit defect images, which is intuitive and easy to understand, but requires high shooting alignment accuracy and sensitive light environment. In addition to the above method, developers also use various machine vision and image processing algorithms to complete the defect detection of electronic circuits. Generally, these methods need to report all defect types in advance. However, we cannot guarantee that the detection system will only encounter defects that have been determined in advance. In actual production environment, various sudden defects are often encountered, which cannot be correctly detected by the conventional machine vision-based detection method. In this case, the defect inspection system must be able to use new sample data to recalibrate when facing changes in circuit manufacturing conditions. This is a major drawback of the traditional machine vision detection system. SUMMARY

[0004] Invention purposes: In view of the current situation that the electronic circuit board is increasingly diversified and complicated, and it is difficult to identify its tiny defects by the conventional way of observing with the naked eye, the present application provides an electronic circuit defect detection method based on non-negative constraint sparse autoencoder, which captures the image of the electronic circuit by using an image sensor (industrial camera), trains a deep autoencoder model using these images, decodes the original non-defect image from the defect circuit image, and then compares the decoded circuit image with the input circuit image to determine the defect position, thereby achieving effective detection of circuit defects, without the need to know the defect type or the normal / defect evaluation standard of the expert system in advance, and overcoming the problem of small and unbalanced data set existing in the early manufacturing stage.

[0005] Technical scheme: The present application discloses an electronic circuit defect detection method based on non-negative constraint sparse autoencoder, comprising the following steps:

[0006] (1) Preprocessing of electronic circuit defect data set; collecting and cropping the electronic circuit image, determining the electronic circuit defect data set and defect type, and completing the enhancement and feature selection of image data;

[0007] (2) Completing the electronic circuit image denoising based on group graph model and kernel specification method GNN, and using the Laplace graph matrix and kernel specification of the image to encapsulate the graph structure;

[0008] (3) Constructing a deep learning model based on non-negative constraint sparse autoencoder FFSAE, and using it for defect area extraction;

[0009] (4) Generating a defect detection map, using the successfully trained autoencoder FFSAE model to predict a high-quality circuit image, subtracting the original defect input image from the predicted image to generate a defect detection map, and finally highlighting the defect position by setting an appropriate threshold for the defect detection map, thereby completing the correct classification of the electronic circuit defect type.

[0010] Further, the preprocessing of the electronic circuit defect data set in step (1) comprises the following specific steps:

[0011] Step (1)a: Collecting and cropping the electronic circuit image to determine the electronic circuit defect data set;

[0012] Step (1)b: Determine the electronic circuit defect type, set two kinds of defects, one is true defect, the other is pseudo defect; wherein the true defect is a defect caused by the change of lead shape, which is specifically set as disconnection defect, connection defect, protrusion defect and crack defect; the pseudo defect is characterized by only color change, while the shape feature of the lead and the basic component is unchanged, which is specifically set as oxidation defect and dust defect;

[0013] Step (1) c: complete image data enhancement and feature selection

[0014] Data enhancement: data enhancement is completed by applying geometric transformation and adding noise: random rotation is applied to overcome the position deviation of image data; a random matrix with noise distribution is multiplied with the original data;

[0015] Feature selection: determine the feature parameters as color information and shape information, wherein the color information is 30 kinds, and the following features are extracted from the RGB and HSV color models, respectively, 1) maximum value, 2) minimum value, 3) average value, 4) proportional high value, 5) ratio of lead area to candidate area and lead, 6) ratio of basic component to candidate component, 7) position difference between numerical barycenter and maximum value, 8) variance, 9) standard deviation, 10) kurtosis, 11) skewness, 12) entropy, 13) difference between maximum value and minimum value, 14) median value, 15) correlation between test image and reference image; the shape information includes 8 types, including 1) area, 2) perimeter, 3) x direction size, 4) y direction size, 5) aspect ratio, 6) diagonal length, 7) complexity, 8) roundness.

[0016] Further, the step (2) completes the circuit image denoising based on the group-based graph model and the kernel specification method GNN, and the specific steps are as follows:

[0017] Step (2) a: use Laplace matrix L to represent the collected patch image

[0018] (1) Obtain the weighted adjacency matrix W according to the image data

[0019] The weighted adjacency matrix W of the undirected weighted graph is non-negative, and has equal diagonal elements, that is, W ij = w ji , W ij ≥ 0, and the edge weight matrix W is constructed by using the threshold Gaussian kernel, which is as follows:

[0020]

[0021] Wherein, is the Euclidean distance between image vertices v i and v j , σ is a speed control parameter for controlling the weight decay with the increase of distance, and ε is a threshold parameter representing an ε-neighborhood graph;

[0022] (2) Obtain the image represented by the Laplace matrix L

[0023] L = Δ - W

[0024] Wherein, Δ is a diagonal matrix, which satisfies the equation Δ ii=∑ j W ij ;

[0025] Step (2) b: Establishing the combination optimization formula based on the group-based graph model and kernel norm

[0026] (1) Constructing the basic optimization formula

[0027] Let be the regularization term associated with the Laplace matrix, then the basic optimization formula based on image denoising is as follows:

[0028]

[0029] Where x and y are both n x 1 vectors representing image blocks, L is an n x n Laplace matrix, and θ is a regularization parameter;

[0030] (2) Constructing the optimization formula based on the group-based dual graph

[0031] Considering that each group is a matrix, a dual graph T including a row graph and a column graph is constructed m×n Therefore, the optimization expression of the dual graph model is defined as follows:

[0032]

[0033] Where X and Y are m x n image row and column data matrices, θ r and θ c are regularization control parameters used to determine the influence degree of the regularization term, i.e., the row graph and the column graph

[0034] is the group-based row graph regularization term, which is defined by the similarity of pixel intensities located at the same position of all similar patch images, specifically:

[0035]

[0036] is the group-based column graph regularization term, which is defined by the similarity of pixel intensities located at all positions corresponding to each patch image, specifically:

[0037]

[0038] L r and L c are the row Laplace matrix and the column Laplace matrix, respectively;

[0039] (3) Constructing the optimization formula of kernel norm

[0040] Introducing low-order optimization processing, the conventional replacement of low-rank data matrix X is called kernel norm or trace norm ||X|| * , which is defined as follows:

[0041] ||X|| * = tr((XX T ) 1 / 2 ) = å k å k

[0042] Wherein, å k is the singular value of X;

[0043] (4) construct a combination optimization formula based on grouping graph model and kernel norm

[0044] The specific definition is as follows:

[0045]

[0046] Wherein, å n , å r and å c are the control parameters of kernel norm, row graph and column graph, it can be seen that the regularization term reflects the non-local self-similarity, and the kernel norm reflects the low-rank characteristics of the image that can use a large amount of information.

[0047] Further, the combination optimization formula based on grouping graph model and kernel norm in step (2) is optimized and solved by KNN algorithm, and the specific steps are as follows:

[0048] (1) calculate the optimization formula value between the current patch image and all patch images;

[0049] (2) arrange in ascending order according to the optimization value;

[0050] (3) select the K nearest patch images of the optimization value;

[0051] (4) count the frequency of the category in which the K patch images appear, and take the category with the highest frequency among the K patch images as the result image after denoising.

[0052] Further, the step (3) based on non-negative constraint sparse autoencoder (FFSAE) deep learning model is specifically:

[0053] Step (3) a: using the encoder to encode the input data

[0054] First, a set of encoders å E ={W E , b E} is used to convert the input data into "compressed" representation of image features; the input signal X m∈ R d Transforming into hidden layer feature vector h m ∈ R s :

[0055] h m = E(X m , θ E ) = sigm(W E X E + b E )

[0056] where θ E represents the encoder parameters consisting of weight matrix W E and bias vector b E , and the encoder is a nonlinear transformation function: E(): R d → R s (d > s).

[0057] Step (3) b: define the cost function η AE (W, b)

[0058] The average reconstruction error of all training samples is defined as the cost function η AE (W, b), while increasing the weight decay penalty term α, which is defined as follows:

[0059]

[0060] where W = {W E , W D}, b = {b E , b D}, M is the number of training samples, and α is the regularization penalty term that controls the reduction of weights.

[0061] Step (3) c: build a sparse autoencoder

[0062] (1) Solve the average activation value of the hidden layer unit

[0063] A sparse autoencoder can be constructed by imposing sparsity on the hidden layer units of the autoencoder, and the sparse autoencoder expects the average activation value of each hidden layer unit to be close to zero; let [h m ] j be the activation value of the jth hidden unit related to X m , then the average activation value of the jth hidden layer unit on the entire training set is calculated as follows:

[0064]

[0065] (2) Define the penalty term

[0066] The sparsity constraint of the sparse autoencoder is enforced by adding an extra penalty term to increase the cost function when the average activation of hidden units deviates from ε, which is defined as the Kullback-Leibler (KL) divergence as follows:

[0067]

[0068] where is the average activation vector of hidden units, s is the number of hidden units, is a standard function to measure the difference between two distributions;

[0069] It can be seen that when the minimum value of 0 can be reached, and when deviates from ε upwards, it will be invalid, so minimizing this penalty term can make close to ε;

[0070] (3) Define the sparsity cost function η SAE (W, b)

[0071] The training objective optimization function of the sparse autoencoder is to minimize the average reconstruction error η AE (W, b) and the sparsity penalty term Therefore, the sparsity cost function η SAE (W, b) is defined as:

[0072]

[0073] where β is the weight used to control the sparsity penalty term; since ε represents the average activation of all hidden units, and the activation of hidden units depends on the parameters {W, b}, therefore, the ε term also depends on {W, b};

[0074] Step (3) d: Establish the cost function η FFSAE (W, b) of the non-negative constraint standard autoencoder

[0075] Propose a non-negative constraint autoencoder FFSAE, modify the cost function to η FFSAE (W, b), as follows:

[0076]

[0077] where

[0078] Step (3) e: Update and ​

[0079] First, initialize parameters and a random value close to zero, and then apply a gradient descent optimization algorithm to train, updating parameters in each iteration and Specifically as follows:

[0080]

[0081]

[0082] where λ>0 is the learning rate;

[0083] Step (3) f: use the decoder to reconstruct the input signal

[0084] Use the decoder θ D ={W D , b D} to recover the hidden features in reverse, so as to reconstruct the input signal, and in implementation, through the decoder D(): R s →R d , the hidden feature vector h m is recovered in reverse to a reconstructed vector with similar structure The implementation formula is as follows:

[0085]

[0086] where θ D represents the decoder parameters composed of the weight matrix W D and the bias vector b D .

[0087] Further, when determining the defect type by using the defect detection map, the performance evaluation index needs to be defined, and the structural similarity measurement index SSI is used as the evaluation index to measure the degradation degree of the structure information of a certain image relative to another image, and the specific calculation is as follows:

[0088]

[0089] In the formula, μ and σ respectively represent the average value and variance of each pixel parameter; σ xy is the covariance; c1 and c2 are constants to prevent the divisor from being zero.

[0090] Beneficial effects:

[0091] 1. The present application utilizes the non-negative constrained sparse autoencoder (FFSAE) and the group-based graph model and kernel specification image denoising method (GNN) to achieve automatic and effective detection of electronic circuit defect types. By using an image sensor (industrial camera) to capture images of the electronic circuit, the deep autoencoder model is trained using these images, and the original non-defect image is decoded from the defect circuit image, and then the decoded circuit image is compared with the input circuit image to determine the defect location, thereby achieving effective detection of circuit defects. Therefore, this method does not require prior knowledge of defect types or expert system normal / defect evaluation standards, and can overcome the problem of small and unbalanced data sets existing in the early manufacturing stage.

[0092] 2. The present application can design a suitable data set for training through appropriate preprocessing, thereby improving the performance of the classification model. The preprocessing techniques such as cropping and enhancement of the circuit image are completed, and the group-based graph model and kernel specification method (GNN) is used for image denoising. This method obtains the Laplacian graph matrix of the image by optimizing the learning strategy, and the Laplacian matrix encapsulates the graph structure of the data matrix, thus reflecting the topological structure of the image, thereby ensuring good application effect in image smoothing and denoising, further effectively improving the quality of the circuit image, and ensuring the establishment of a more perfect defect data set. The present application uses the non-negative constrained sparse autoencoder (FFSAE) to extract the defect area. The FFSAE algorithm emphasizes that the neuron weight is non-negative, converts the input encoding into a low-dimensional space encoding form, and reconstructs the original data through decoding. This method allows useful features to be extracted from unlabeled data, and the original input data can be reproduced through latent vector expansion. Based on the non-negative value of the neuron weight, the FFSAE algorithm can improve the interpretability of the recognition network operation, enhance the distinguishability of the learned features, and further improve the reliability and precision of electronic circuit defect recognition. BRIEF DESCRIPTION OF DRAWINGS

[0093] Figure 1 is the processing process of the electronic circuit defect detection method based on the non-negative constrained sparse autoencoder (FFSAE);

[0094] Figure 2 is a true defect image;

[0095] Figure 3 is a pseudo-defect image;

[0096] Figure 4 is the electronic circuit image denoising process based on the group-based graph model and kernel specification method (GNN);

[0097] Figure 5 is the image denoising process based on the group-based graph model and kernel specification;

[0098] Figure 6 is a structure of an autoencoder in an image denoising recovery application;

[0099] Figure 7 is a standard autoencoder structure;

[0100] Figure 8 is a circuit defect detection experimental result image based on FFSAE. DETAILED DESCRIPTION

[0101] In order to better explain the present application, the technical solutions of the present application are described in detail below. The following examples are an explanation of the present application, and the present application is not limited to the following examples.

[0102] The present application proposes an electronic circuit defect detection method based on non-negative constraint sparse autoencoder (FFSAE). The electronic circuit defect detection process has two main purposes. First, the electronic circuit defect data set is preprocessed. Since the amount of training data determines the performance of the defect detection classification model, through appropriate preprocessing process, the data set suitable for training can be designed, thereby improving the performance of the classification model. In view of this problem, the patent completes the preprocessing technology such as cropping and enhancing of the circuit image, and uses group-based graph model and kernel specification method (GNN) to denoise the image. This method obtains the Laplace graph matrix of the image through the optimization learning strategy. The Laplace matrix encapsulates the graph structure of the data matrix, so it can reflect the topological structure of the image, thereby ensuring its good application effect in image smoothing and denoising, further effectively improving the quality of the circuit image, and ensuring to establish a more perfect defect data set. The second is to extract the defect area and correctly identify it. The present application uses non-negative constraint sparse autoencoder (FFSAE) to extract the defect area. The FFSAE algorithm emphasizes that the neuron weight is non-negative, converts the input encoding into a low-dimensional space encoding form, and reconstructs the original data through decoding. This method allows useful features to be extracted from unlabeled data, can reproduce the original input data through latent vector expansion, based on the non-negative value of the neuron weight, the FFSAE algorithm can improve the interpretability of the recognition network operation, can enhance the distinguishability of the learned features, further, can improve the reliability and precision of the electronic circuit defect recognition.

[0103] The whole electronic circuit defect detection processing process mainly includes a model training stage and a defect recognition stage. In the model training stage, first, the image is preprocessed, the original electronic circuit image is segmented into patch images with a size of 500*500 pixels through the cropping operation of the image, and then the noise suppression processing is performed on each image, the group-based graph model and the kernel specification method (GNN) are used to complete the noise suppression of the image, so as to improve the quality of the original electronic circuit image data set. Next, the denoised electronic circuit image is subjected to data enhancement. In the data enhancement module, the defect patch image is subjected to random rotation, flipping and noise addition operations to complete the enhancement of the image data set, so as to effectively overcome the problem of data class imbalance caused by insufficient data. Finally, the enhanced patch image is used to train the non-negative constraint sparse autoencoder model (FFSAE), so as to predict the input non-defect patch image. After the model training is successful, the recognition stage of the electronic circuit defect is entered. The defect electronic circuit image for detection is sent into the model which has been successfully trained, once the high-quality image is predicted from the defect electronic circuit image by the trained model, the defect detection image is generated by subtracting the defect input image from the predicted image. Finally, the threshold of the defect detection image is set appropriately, so as to highlight the defect position and complete the effective and intuitive recognition of the defect.

[0104] Figure 1 The proposed electronic circuit defect detection method based on the non-negative constraint sparse autoencoder (FFSAE) in the embodiment of the application is shown, which combines Figure 1 The electronic circuit defect detection method based on the non-negative constraint sparse autoencoder (FFSAE) disclosed in the application specifically includes the following steps:

[0105] Step 1: realizing the preprocessing of the electronic circuit defect image. The specific steps are as follows:

[0106] Step (1)a: collecting and cropping the electronic circuit image, and determining the electronic circuit defect data set

[0107] To verify the effectiveness of the electronic circuit defect detection method, it is necessary to create an electronic circuit defect image dataset. In implementation, ten reference electronic circuit image datasets are collected, each dataset is collected by a million-pixel high-definition industrial camera equipped with a CMOS sensor. The original image is 4608*3456 pixels, the dataset contains 600 electronic circuit board defect images, which can be further divided into color images and grayscale images. Since the electronic circuit images collected by the industrial camera are high-resolution images, the data volume of processing these images is large, and the computational complexity of processing is very large, which will cause the processing time of training the classification model to be longer. To improve its computing power, in preprocessing, according to the size of each electronic circuit, the electronic circuit image is cropped, and the defect area is cropped into a patch image with a size of 500*500.

[0108] Step (1)b: Determine the type of electronic circuit defect

[0109] After collecting the electronic circuit image, it is necessary to manually process the defect image to determine the type of defect present in the electronic circuit board. In implementation, two types of defects are set, one is a true defect, and the other is a false defect. Among them, the true defect is a defect caused by the change of the lead shape, and the true defect image is as shown in Figure 3 , which is specifically set as a broken defect (see Figure 2 (a)), a connected defect (see Figure 2 (b)), a protruding defect (see Figure 2 (c)), and a crack defect (see Figure 2 (d)). The characteristic of the false defect is only color change, and the shape feature of the lead and the basic component is unchanged, and the false defect definition image is as shown in Figure 3 , which is specifically divided into oxidation defect (see Figure 3 (a)) and dust defect (see Figure 3 (b)).

[0110] Step (1)c: Complete image data augmentation and feature selection

[0111] Generally, training deep classification algorithms requires large-scale training data. However, in the electronic circuit manufacturing process, the probability of generating circuit defects is usually small, and in the mass production of electronic circuits, the types of defects also change. This imbalance of data is a fundamental problem that limits the application of electronic circuit defect detection systems. If these imbalanced data are applied to deep defect detection models, problems such as overfitting and performance degradation will occur. In order to avoid these problems, we apply data augmentation to supplement the small amount of defect data, thereby improving the performance of the model. Considering the redundancy of augmented images, geometric transformation and noise addition are applied to complete data augmentation. Geometric transformation is an effective method to use shape, direction or part feature position, and random rotation is applied to overcome the position bias of image data. Noise addition is to add noise to the original image, and multiply the original data with a random matrix with noise distribution. Through the above data augmentation techniques, the classification model can learn more robust features.

[0112] After preprocessing, image features need to be extracted from the defect candidate region and sent to the non-negative constrained autoencoder (FFSAE) for learning and classification. In implementation, the feature parameters are determined as color information and shape information. Among them, the first is color information, a total of 30 kinds, which are extracted from the RGB, HSV color model as follows: (1) maximum value, (2) minimum value, (3) average value, (4) proportional high value, (5) lead area and candidate area and lead ratio, (6) basic component and candidate component ratio, (7) position difference between numerical barycenter and maximum value, (8) variance, (9) standard deviation, (10) kurtosis, (11) skewness, (12) entropy, (13) difference between maximum value and minimum value, (14) median value, (15) correlation between test image and reference image. The second is shape information, a total of 8 types, including (1) area, (2) perimeter, (3) x direction size, (4) y direction size, (5) aspect ratio, (6) diagonal length, (7) complexity, (8) roundness.

[0113] Step (2), complete the electronic circuit image denoising based on group-based graph model and kernel specification method (GNN).

[0114] In image denoising, a group-based graph model and kernel specification image denoising method (GNN) is proposed. First, the original noisy image is converted into a patch image vector by using group sampling, and then GNN is used for processing. The dual graph based on grouping and the kernel specification optimization formula are constructed, and the KNN algorithm is used to search for similar patches, that is, by block matching each noisy patch image, m patch images similar to the original image are obtained, and finally all similar patch images are stacked to construct a group-based graph model. In the construction of the group-based graph model, a Laplacian matrix graph strategy is proposed, which first establishes a learning strategy through the optimization formula, and then obtains the Laplacian graph matrix using the optimization learning strategy. The Laplacian weighted matrix graph regards each pixel in the data matrix as a node, and explores the similarity between patch image pixels, so it can reflect the topological structure of the image and achieve effective smoothing of the image, further enhancing the denoising and smoothing effect of the electronic circuit image.

[0115] As Figure 4 is the electronic circuit image denoising process based on the group-based graph model and kernel specification method (GNN), the specific steps are as follows:

[0116] Step (2) a: Use the Laplacian matrix L to represent the collected patch image

[0117] (1) Obtain the weighted adjacency matrix W according to the image data

[0118] A weighted undirected graph can be represented as G=(V, E, W). Where V is a vertex set consisting of N vertices, E is an edge set consisting of multiple VxN weighted edges, and W is a weighted adjacency matrix. The weighted adjacency matrix W reflects the similarity between vertices v i and v j , in general, the weighted adjacency matrix W of the weighted undirected graph is non-negative and has equal diagonal elements, i.e. W ij = w ji , W ij ≥ 0.

[0119] Based on this, the threshold Gaussian kernel is used to construct the edge weight matrix W, as follows:

[0120]

[0121] Where, is the Euclidean distance between image vertices v i and v j , σ is a speed control parameter that controls the decay of the weight with distance, and ε is a threshold parameter representing the ε-neighborhood graph.

[0122] (2) Obtain the image represented by the Laplacian matrix L

[0123] The Laplacian matrix L plays a crucial role in describing the characteristics of graph data. L encapsulates the graph structure of the data matrix. In an undirected weighted graph, the calculation of L is determined by W, as follows:

[0124] L=Δ-W (2)

[0125] Where Δ is a diagonal matrix, satisfying the equation Δ ii =∑ j W ij .

[0126] Step (2)b: Establish a group-based graphical model and a combinatorial optimization formula based on kernel specifications.

[0127] (1) Constructing the basic optimization formula

[0128] set up This is the regularization term associated with the Laplacian matrix. Therefore, the basic optimization formula for image denoising is as follows:

[0129]

[0130] Here, x and y are both n×1 vectors representing image patches, L is an n×n Laplacian matrix, and θ is a regularization parameter.

[0131] (2) Construct an optimization formula based on grouped dual graphs

[0132] To construct the dual graph, the traditional method is to divide each data matrix X... m×n Viewed as an n-dimensional column vector X = (x1, ..., x2) n or an m-dimensional row vector X = ((x′1)) T ,…,(x′ m ) T ) T Therefore, each vector in the matrix (each column or each row) is considered as a node in the computation of the weighted adjacency matrix W. The weighted adjacency matrix W is then... m×m The Euclidean distance function of the line is The Euclidean distance function of the column is Therefore, using W m×m and W n×n The row Laplace matrix L can be obtained. r And the Laplace matrix L c .

[0133] Given that each group is a matrix, we construct a dual graph T that includes both row and column graphs. m×n The optimization expression for the dual graph model is defined as follows:

[0134]

[0135] where X and Y are m x n image data matrices, and θ r and θ c are regularization control parameters to determine the influence degree of the regularization term, i.e., row and column

[0136] is a group-based row graph regularization term defined by the similarity of pixel intensity located at the same position of all similar patch images, specifically:

[0137]

[0138] is a group-based column graph regularization term defined by the similarity of pixel intensity located at all positions corresponding to each patch image, specifically:

[0139]

[0140] L r and L c are row Laplacian matrix and column Laplacian matrix, respectively.

[0141] (3) Construction of optimization formula of kernel specification

[0142] Considering the low-rank property of grouped image intensity, a low-rank optimization process is further introduced. The conventional replacement of low-rank data matrix X is called kernel specification or trace specification ||X|| * , which is specifically defined as follows:

[0143] ||X|| * = tr((XX T ) 1 / 2 ) =∑ k σ k (7)

[0144] where σ k is the singular value of X.

[0145] (4) Construction of combined optimization formula based on group-based graph model and kernel specification

[0146] Specifically defined as follows:

[0147]

[0148] where θ n , θ r and θ c are control parameters of kernel norm, row graph and column graph. In equation (7), the regularization term reflects the non-local self-similarity, and the kernel specification reflects the low-rank property that a large amount of information images can be used.

[0149] Step (2)c: optimization processing is solved by using KNN algorithm, the specific steps are as follows:

[0150] (1) Calculate the optimization formula value between the current patch image and all patch images;

[0151] (2) According to the ascending order of optimization value;

[0152] (3) Select the K nearest neighbor patch image of optimization value;

[0153] (4) Statistics K patch image class frequency, the highest frequency of K patch image as the result image after denoising.

[0154] As Figure 5 is a group-based graph model and kernel specification image denoising process.

[0155] Step (3), realize the deep learning model based on non-negative constraint sparse autoencoder (FFSAE), and use it for defect area extraction.

[0156] Autoencoder is a kind of encoding form aiming at encoding input into low-dimensional space, which can be reconstructed by decoder to the original data. This method allows to extract useful features from unlabeled data, and can reproduce the original input data through latent vector expansion. It is usually used in data compression, denoising, anomaly detection, image recovery and other aspects.

[0157] Figure 6 is the structure of autoencoder in image denoising and recovery application.

[0158] Autoencoder network is a kind of unsupervised learning algorithm. Essentially, autoencoder learns function M W，b(X) ≈X, in other words, by learning the estimated value of feature function, we can get In the specific implementation, the autoencoder can guarantee the network to learn the compression features of the input data by forcing the number of hidden units to be less than the input dimension, and the feature structure in the input data can be found by using the compression features. The standard autoencoder is composed of encoder and decoder.

[0159] Figure 7is a standard autoencoder structure, in the present patent implementation, a non-negative constraint sparse autoencoder (FFSAE) is proposed to extract defect regions and identify them correctly. The FFSAE algorithm emphasizes that the neuron weight is non-negative, converts the input encoding into a low-dimensional space encoding form, and reconstructs the original data through decoding. This method allows useful features to be extracted from unlabeled data, and the original input data can be reproduced by extending the latent vector. Based on the non-negative characteristics of the neuron weight, the FFSAE algorithm can improve the interpretability of the recognition network operation, enhance the distinguishability of the learned features, and further improve the reliability and accuracy of electronic circuit defect recognition. The specific implementation process of the FFSAE algorithm is as follows:

[0160] Step (3)a: encode the input data using the encoder

[0161] First, a set of encoders θ E ={W E ,b E} is used to convert the input data into a "compressed" representation of image features. The encoder can be understood as a nonlinear transformation function: E(): R d →R s (d>s), which transforms the input signal X m ∈R d into a hidden layer feature vector h m ∈R s by formula (9):

[0162] h m =E(X m ,θ E )=sigm(W E X E +b E ) (9)

[0163] Where θ E represents the encoder parameters composed of the weight matrix W E and the bias vector b E .

[0164] Step (3)b: define the cost function η AE (W,b)

[0165] The essence of the autoencoder is to learn the image compression characteristics in the hidden layer, and to reconstruct the input using the minimum average error in all training samples. Therefore, the average reconstruction error of all training samples is defined as the cost function η AE (W,b), and a weight decay penalty term α is added to minimize the overfitting risk and improve the generalization ability of the algorithm. The specific definition is as formula (10):

[0166]

[0167] where W = {W E , W D}, b = {b E , b D}, M is the number of training samples, and a is a regularization penalty term that controls the reduction weight.

[0168] Step (3) c: Establishing sparse auto-encoder

[0169] (1) Solve the average activation value of hidden layer units

[0170] Sparse auto-encoder can be constructed by imposing sparsity on the hidden layer units of auto-encoder, and sparse auto-encoder expects the average activation value of each hidden layer unit to be close to zero. Let [h m ] j be the activation value of the jth hidden unit related to X m , then the average activation value of the jth hidden layer unit on the entire training set is calculated as formula (11):

[0171]

[0172] (2) Define the penalty term

[0173] The sparsity constraint of sparse auto-encoder is enforced by . Wherein, e is a predefined sparsity parameter, usually a small value close to 0 (such as 0.05). In order to meet the sparsity constraint, the activation of the hidden layer unit must be mostly close to zero. In order to achieve this, an additional penalty term is added to punish the case of deviating from e, and the penalty term is defined as Kullback-Leibler (KL) divergence, as shown in formula (12):

[0174]

[0175] where, is the average activation vector of hidden units, and s is the number of hidden units. is a standard function for measuring the difference between two distributions.

[0176] It can be seen that when , can reach the minimum value of 0, and when deviates from e upwards, it will cause it to be invalid, so minimizing this penalty term can make close to e.

[0177] (3) Define the sparse cost function η SAE (W, b)

[0178] The training objective optimization function of the sparse autoencoder is to minimize the average reconstruction error η AE (W, b) and the sparsity penalty term in equation (12) Thus, the sparse cost function η SAE (W, b) is defined as equation (13):

[0179]

[0180] where β is the weight to control the sparsity penalty term. It can be seen that since ε represents the average activation of all hidden units, and the activation of hidden units depends on the parameters {W, b}, therefore, the ε term also depends on {W, b}.

[0181] Step (3)d: Establishing the cost function η FFSAE (W, b)

[0182] When the weights of the reinforced neurons are non-negative, the interpretability of the network operation can be improved, and the discriminability of the learned features can also be enhanced. Therefore, we propose a non-negative constrained autoencoder (FFSAE), in order to realize the non-negativity of the weights, the cost function in equation (13) is modified as η FFSAE (W, b), which is specifically as equation (14):

[0183]

[0184] where,

[0185] The goal of FFSAE training is to minimize η FFSAE (W, b) as a function of W and b. According to equation (15), the penalty value assigned to the negative weights is the square value of the corresponding term, while the penalty value assigned to the non-negative weights is 0. Therefore, minimizing the cost function η FFSAE (W, b) can better reduce the number of negative weights. In addition, as the regularization term of the non-negative constrained autoencoder (FFSAE), η FFSAE (W, b), equation (14) will have the characteristics of reducing reconstruction error, encouraging learning sparse features, and also reducing the number of non-negative weights.

[0186] Step (3)e: Update and

[0187] In order to achieve the above optimization goal, first initialize the parameters and To approach zero for random values, then apply gradient descent optimization algorithm for training, updating parameters in each iteration and Specifically as (16), (17) formula:

[0188]

[0189]

[0190] Where, λ>0 is the learning rate.

[0191] Step (3) f: using the decoder to reconstruct the input signal

[0192] Using the decoder θ D ={W D , b D} to recover the hidden features reversely, so as to reconstruct the input signal. In specific implementation, through the decoder D(): R s →R d , the hidden feature vector h m is recovered reversely into the reconstructed vector The implementation formula is as follows:

[0193]

[0194] Where, θ D represents the decoder parameters composed of the weight matrix W D and the bias vector b D .

[0195] Step (4), generate the defect detection map.

[0196] In the proposed method, finally, the defect detection map needs to be generated to obtain the correct defect type. In specific implementation, first, a defect-free output image is generated by using the classification model, and then it is subtracted from the input image to obtain the defect detection map. This image subtraction is a numerical calculation, that is, the numerical value of another image is subtracted from the numerical value of the entire image. Through this method, we can detect the changes between two images and use it for the identification of circuit defects. The specific steps are as follows:

[0197] Step (4) a: predicting the circuit image

[0198] The defect electronic circuit image for detection is sent into the trained self-encoder (FFSAE) model, and a high-quality circuit image is predicted from the defect circuit image by training the model.

[0199] Step (4) b: generating the defect detection map

[0200] The predicted image is subtracted from the original defect input image, i.e. a defect detection map is generated. Finally, the defect position is highlighted by setting a proper threshold for the defect detection map, so as to complete the correct classification of the electronic circuit defect type.

[0201] When the defect type is determined by using the defect detection map, a performance evaluation index needs to be defined. The performance index is used to measure the similarity between the predicted image and the target image. In the present patent, a structural similarity measurement index (SSI) is used as the evaluation index to measure the degradation of the structural information of a certain image relative to the structural information of another image. The SSI is calculated by comparing the brightness, contrast and structure to measure the similarity of two pictures. The specific calculation is as follows:

[0202]

[0203] In the formula, μ and σ respectively represent the average value and variance of each pixel parameter; σ xy is the covariance; c1 and c2 are constants to prevent the divisor from being zero.

[0204] Step 5, based on the above technology, an experimental platform is built to complete the specific implementation of motor fault classification, and the following test experiments are mainly completed:

[0205] (1) Circuit defect detection experiment based on FFSAE

[0206] A subset is randomly sampled from the electronic circuit image dataset, and color features and shape features are randomly selected and input into the non-negative constrained autoencoder (FFSAE). The detection effect is obtained as shown in Figure 8 . It can be seen that the method can accurately detect samples with large color changes, such as broken lines and oxidation defects. In addition, the proposed method can also detect dust pseudo-defects with small color changes.

[0207] (2) Comparison experiment of real defect and pseudo-defect detection

[0208] In order to verify the correctness of the proposed method, FFSAE is compared with conventional algorithms (SVM, BP and RBF). The electronic circuit dataset used includes 500 defect images, of which 300 are real defects and 200 are pseudo-defects. Gray images and color images are used for each method. The brightness value of the gray image and 15 kinds of features corresponding to the RGB of the color image are taken. The test results of real defects and pseudo-defects of each method are shown in Table 1. During the test, since the subsets and features are selected randomly, the method is executed 10 times, and the average value is taken as the final result.

[0209] Table 1 Classification results of different methods

[0210]

[0211] The results in Table 1 show that the present method gives better discrimination between real defects and false defects than the existing methods.

[0212] (3) Comparison experiment of defect detection between color image and gray image

[0213] In the experiment, 200 color defect images and 200 gray defect images are selected, and different methods are used for defect measurement, respectively, to obtain the experimental results in Table 2.

[0214] Table 2: Classification results of different images

[0215]

[0216] Table 2 shows that when using color images alone, the measurement results are better than those in Table 1, while when using gray images alone, the measurement results are worse than those in Table 1. On the other hand, overall, the correct rates of all methods are improved when using color images than when using gray images. That is, the effectiveness of color images is reflected in the combination of features represented by multiple colors, such as ratio, entropy in RGB, correlation between test image and reference image, etc. The combination of colors is better for defect classification, which further illustrates the important role of color images in defect measurement and classification.

[0217] The above embodiments are only for illustrating the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and cannot limit the protection scope of the present application. Any equivalent transformation or modification made according to the spirit and essence of the present application should be covered within the protection scope of the present application.

Claims

1. A method for electronic circuit defect detection based on non-negative constrained sparse autoencoder, characterized in that, Comprising the following steps: (1) Preprocessing of the electronic circuit defect data set; image acquisition and cropping of the electronic circuit, determination of the electronic circuit defect data set and defect type, completion of image data enhancement and feature selection; (2) Complete the group-based graph model and kernel specification method GNN of the electronic circuit image denoising, and use the Laplace graph matrix and kernel specification of the image to encapsulate the graph structure; (3) Construct a deep learning model based on the non-negative constraint sparse autoencoder FFSAE, and use it for defect area extraction; Step (3)a: encode the input data using the encoder First, a set of encoders θ E = {W E ,b E} is used to convert the input data into image features in a "compressed" representation; The input signal X is transformed into a hidden layer feature vector h by the following equation m ∈R d ∈R m ∈R s : h m = E(X m , θ E ) = sigm(W E X E + b E ) where θ E represents the encoder parameters consisting of a weight matrix W E and a bias vector b E , the encoder is a nonlinear transformation function: E(g): R d → R s (d > s); Step (3) b: Define cost function η AE (W,b) The average reconstruction error of all training samples is defined as the cost function η AE (W,b), while increasing the weight decay penalty term a, which is defined as follows: where W = {W E ,W D}, b = {b E ,b D}, M is the number of training samples, and a is a regularization penalty term that controls the reduction weight. Step (3)c: Establish a sparse autoencoder (1) solving for the average activation value of the hidden layer units Sparse autoencoders can be constructed by imposing sparsity on the hidden units of an autoencoder, with the sparse autoencoder desiring the average activation of each hidden unit to be close to zero; let h m ] j The activation of the jth hidden unit related to X m The average activation of the jth hidden unit over the entire training set is computed as follows: (2) Defining the penalty term The sparsity constraint of the sparse autoencoder is enforced by adding an extra penalty term to the reconstruction loss, where ε is a predefined sparsity parameter, to penalize cases where the activations significantly deviate from ε. The penalty term is defined as the Kullback-Leibler (KL) divergence, as shown in the following equation: L = -∑i∑jxijlogxij+ε∑i∑jxijlogxij wherein, is the average activation vector of the hidden units, s is the number of hidden units, is a standard function for measuring the difference between two distributions; It can be seen that when , the minimum value of 0 can be reached, and when deviates upward by ε, it will be invalid, so minimizing this penalty term can make close to ε; (3) define a sparse cost function η SAE (W,b) The training objective optimization function of the sparse autoencoder is to minimize the average reconstruction error η AE (W,b) and a sparsity penalty term Thus, the sparse cost function η SAE (W,b) is defined as: Wherein, beta is the weight used to control the sparsity penalty term; Since epsilon represents the average activation of all hidden units, and the activation of the hidden unit depends on the parameters {W, b}, therefore, the epsilon term also depends on {W, b}; Step (3) d: Establishing the cost function η of the non-negative constraint standard autoencoder FFSAE (W,b) A non-negative constrained autoencoder FFSAE is proposed, and the cost function is modified as η FFSAE (W,b), as follows: wherein Step (3)e: Update using gradient descent method and First, initialize parameters and a random value close to zero, then apply a gradient descent optimization algorithm to train, updating parameters at each iteration and as follows: Wherein, lambda>0 is the learning rate; Step (3)f: Use the decoder to reconstruct the input signal Using the decoder θ D = {W D ,b D} to recover the hidden features in reverse, thereby reconstructing the input signal, in practice, through the decoder D(g): R s → R d , the hidden feature vector h m is recovered in reverse to a reconstructed vector The implementation formula is as follows: where θ D represents the decoder parameters consisting of a weight matrix W D and a bias vector b D . (4) Generate a defect detection map, use the successfully trained autoencoder FFSAE model to predict a high-quality circuit image, and subtract the original defect input image from the predicted image to generate a defect detection map. Finally, by setting a proper threshold for the defect detection map, the defect location is highlighted, thereby completing the correct classification of the electronic circuit defect type.

2. The electronic circuit defect detection method of non-negative constrained sparse autoencoder according to claim 1, wherein, The specific steps of preprocessing the electronic circuit defect data set in step (1) are as follows: Step (1)a: Collect and crop the electronic circuit image to determine the electronic circuit defect data set; Step (1)b: Determine the electronic circuit defect type, set two types of defects, one is true defect, the other is pseudo defect; wherein, the true defect is a defect caused by the change of lead shape, which is specifically set as disconnection defect, connection defect, protrusion defect and crack defect; The characteristic of pseudo defect is only color change, and the shape feature of lead and basic component is unchanged, which is specifically set as oxidation defect and dust defect; Step (1)c: Complete image data enhancement and feature selection Data enhancement: geometric transformation and noise addition are used to complete data enhancement: random rotation is used to overcome the positional deviation of image data; Multiply a random matrix with noise distribution with the original data; Feature selection: determine the feature parameters as color information and shape information, wherein the color information includes 30 types, which are extracted from the RGB and HSV color models as follows: 1) maximum value, 2) minimum value, 3) average value, 4) proportional high value, 5) ratio of lead area to candidate area and lead, 6) ratio of basic component to candidate component, 7) position difference between numerical center of gravity and maximum value, 8) variance, 9) standard deviation, 10) kurtosis, 11) skewness, 12) entropy, 13) difference between maximum value and minimum value, 14) median value, 15) correlation between test image and reference image; The shape information includes 8 types, including 1) area, 2) perimeter, 3) x direction size, 4) y direction size, 5) aspect ratio, 6) diagonal length, 7) complexity, 8) roundness.

3. The method of claim 1, wherein the method is characterized by, The step (2) completes the circuit image denoising based on the group-based graph model and the kernel specification method GNN, and the specific steps are as follows: Step (2)a: using Laplacian matrix L to represent the collected patch image (1) obtaining a weighted adjacent matrix W according to image data The weighted adjacency matrix W of the undirected weighted graph is non-negative and has equal diagonal elements, i.e. W ij = w ji ,W ij ≥ 0, the weight matrix W of the edge is composed of the threshold Gaussian kernel, as follows: wherein, is the Euclidean distance between image vertices v i and v j is a threshold parameter representing the ε-neighborhood graph; (2) obtaining an image represented by a Laplacian matrix L L = Δ-W where Δ is a diagonal matrix satisfying the equation Δ = ∑ ii = ∑ j W ij ; Step (2)b: establishing a combination optimization formula based on a group-based graph model and kernel specification (1) constructing a basic optimization formula Let is the regularization term associated with the Laplacian matrix, then the basic optimization formula based on image denoising is as follows: Wherein, x and y are both n×1 vectors representing image blocks, L is an n×n Laplacian matrix, and θ is a regularization parameter; (2) constructing an optimization formula based on a group dual graph Considering that each group is a matrix, i.e. constructing the dual graph T comprising the row and column graphs m×n The optimization expression of the dual graph model is defined as follows: where X and Y are m x n image row and column data matrices, θ r and θ c are regularization control parameters used to determine the degree of influence of the regularization term, i.e., row and column is a group-based row graph regularization term defined by the similarity of pixel intensities located at the same position of all similar patch images, specifically: is a group-based column graph regularization term defined by the similarity of pixel intensities located at all positions corresponding to each patch image, specifically: L r and L c are the row and column Laplacian matrices, respectively; (3) constructing an optimization formula based on a kernel specification Introducing low-rank optimization, the conventional replacement for low-rank data matrix X is called kernel norm or trace norm ||X||. * , which is defined as follows: ||X| * = tr((XX T ) 1 / 2 )=∑ k σ k where σ k is the singular value of X; (4) constructing a combination optimization formula based on a group-based graph model and kernel specification The specific definitions are as follows: where θ n , θ r and θ c are the control parameters of the nuclear norm, row graph and column graph, respectively. It can be seen that the regularization term reflects the non-local self-similarity, and the nuclear norm reflects the low-rank property of the image that can use a large amount of information.

4. The method for detecting defects of electronic circuits based on a non-negative constraint sparse autoencoder according to claim 3, characterized in that, The combination optimization formula based on the group-based graph model and the kernel specification in step (2) is optimized and solved by using a KNN algorithm, and the specific steps are as follows: (1) calculating the optimization formula value between the current patch image and all patch images; (2) arranging in ascending order according to the optimization value; (3) selecting the K nearest patch images of the optimization value; (4) counting the frequency of the class in which the K patch images appear, and taking the class with the highest frequency among the K patch images as the result image after denoising.

5. The method of claim 1, wherein the method is characterized by, When the defect detection image is used to determine the defect type, the performance evaluation index needs to be defined, and the structural similarity measurement index SSI is used as the evaluation index to measure the degradation degree of the structural information of a certain image relative to the structural information of another image, and the specific calculation is as follows: where μ and σ represent the mean and variance of each pixel parameter, respectively; σ xy is the covariance; and c1, c2 are constants to prevent division by zero.

Images