A printed circuit board electronic component target detection method based on deformable convolution

Abstract

The application discloses a printed circuit board (PCB) electronic component target detection method based on deformable convolution, and belongs to the field of computer vision. In view of the deformation problem of the detection object caused by the diversification of the shooting angle in the PCB target detection process, the recognition and positioning of the PCB electronic component are realized. The detection method is realized through the deformable convolution structure, so that the convolution kernel is adaptively deformed according to the size and shape of the target, the adaptability of the neural network to unknown changes is improved, the generalization ability is enhanced, and the precision of the PCB target detection is improved. The problem that the object space deformation cannot be solved by the current deep neural network target detection is solved. The experimental results show that the method can accurately detect the PCB electronic component and has strong adaptive ability, improves the intelligent degree in the recycling and disassembling process of the waste electronic products, and improves the automation degree of the process.

Description

TECHNICAL FIELD

[0001] The application realizes target detection of printed circuit boards (PCB) in the process of recycling waste household appliances by using a target detection method of PCB electronic components based on deformable convolution. PCB is one of the important components of the electronic industry and is also a core component of household appliances. PCB target detection is an important part of waste household appliance recycling and is an important branch of computer vision, belonging to both the control field and the computer field. BACKGROUND

[0002] With the rapid development of China's economy and the rapid progress of science and technology, people's living standards are constantly improving, and the number of household appliances is growing and the replacement cycle is shortened. The number of waste household appliances is growing year by year, providing sufficient raw materials for the electronic equipment recycling industry. The sufficient supply of raw materials and the full demand of the downstream market will provide strong support for the sustainable and healthy development of the electronic waste disassembly industry. Therefore, the research results of the application have broad application prospects.

[0003] Printed circuit boards (PCB) are one of the important components of the electronic industry and are also a necessary component for electrical interconnection between electronic components. The dual role of conductive circuits and insulating substrates simplifies the assembly and soldering of electronic products; reduces the size of the whole machine, reduces product costs, and improves the quality and reliability of electronic equipment. PCB is usually used in combination with electronic components such as capacitors, transformers, resistors, etc. The identification and recycling of electronic components can reduce manufacturing costs, reduce the consumption of raw materials such as silicon, aluminum, nickel, copper, plastic, etc., and promote the green development of the industry. Therefore, waste household appliance PCB electronic component detection is one of the important sub-processes of waste household appliance recycling and should also be the research focus of appliance target detection.

[0004] A key challenge for object detection is how to make the model adapt to the target scale, pose, perspective and partial geometric transformation. Although in recent years, convolutional neural networks have made great success in image classification, object detection, semantic segmentation and other visual recognition tasks, their ability to model geometric transformation mainly comes from extensive image data augmentation and model size expansion. Industrial PCB electronic component target detection is usually complex, and waste household appliance electronic components are often placed on a moving conveyor belt and cannot be completely arranged in order, which will bring deformation caused by multiple angles and a certain degree of occlusion. At the same time, the scale of most specific industrial datasets cannot be compared with general object detection datasets. Therefore, a new object detection model must be established to meet the needs of industrial PCB electronic component target detection, solve the challenge of target deformation and improve detection accuracy. Accurate target detection can improve the quality of waste household appliance recycling, thereby promoting the development of household appliance recycling industry. In addition, by improving the automation level of the detection process, it can also effectively reduce the high cost of enterprise brought by manual identification.

[0005] The application designs a printed circuit board electronic component target detection method based on deformable convolution, which realizes the identification and positioning of PCB electronic components by constructing a network model and using a model prediction method. SUMMARY

[0006] In order to solve the above practical problems, the application is a printed circuit board electronic component target detection method based on deformable convolution, which adds deformable convolution to the region proposal network of Faster R-CNN. The deformable convolution makes the sampling network adaptively deform to the input features through two-dimensional offset. Compared with the traditional method, the detection accuracy of the detection method for PCB electronic components is improved, and the deformation problem existing in industrial PCB detection is solved.

[0007] A printed circuit board electronic component target detection method based on deformable convolution, characterized in that it comprises the following steps:

[0008] It includes four stages of "data acquisition", "deformable convolution electronic component detection model establishment", "deformable convolution electronic component detection model training" and "deformable convolution electronic component detection model prediction", and the specific steps are as follows:

[0009] (1) Data acquisition stage

[0010] An image data set of printed circuit board electronic components is acquired, including 5 types of electronic component images: resistor, capacitor, chip, transformer, and varistor. The image data set of printed circuit board electronic components is divided into a training set, a validation set, and a test set, with a picture quantity ratio of 4:1:1. The pixels of each image in the image data set of printed circuit board electronic components are normalized to [0, 1], and each picture is subjected to size scaling, with a fixed short side of 600 and a long side less than or equal to 1000.

[0011] (2) Deformable convolution electronic component detection model establishment stage

[0012] The deformable convolution electronic component detection model includes four modules: electronic component feature extraction, deformable region proposal, electronic component proposal box feature map generation, and electronic component detection box generation.

[0013] ① Electronic component feature extraction

[0014] A convolution residual feature extraction is constructed, which includes four stages: the first stage is composed of a convolution layer with a 7×7 convolution kernel, a batch normalization processing layer, an activation function layer, and a 3×3 maximum pooling layer. The remaining three stages are composed of three, four, and six bottleneck structures, respectively. Each bottleneck structure is formulated as follows:

[0015] Y i =H(X i )+F(X i ,W i ) (1)

[0016]

[0017] wherein formula (1) X is the feature input of the i-th bottleneck structure, i = 1, 2, …, 13, Y i is the output of the i-th bottleneck structure, H(X i ) is the direct linear mapping of X i , which includes convolution operation, F(X i ,W i ) is a nonlinear mapping, which includes three convolution operations, three batch normalizations, and two activation functions, W i is the standard convolution kernel parameter, formula (2) is the bottleneck structure output activation function, X i+1 is the feature input of the i+1-th bottleneck structure, and σ(Y i ) is the activation function of the output Y i of the i-th bottleneck structure; the final output of the feature extraction is a feature map with a dimension of 1024.

[0018] ② Deformable region proposal

[0019] The deformable region proposal is constructed, and the structure is that one convolution kernel is a 3*3 deformable convolution layer, and two convolution kernels are 1*1 convolution layers connected in parallel, and the specific structure is that:

[0020] The expression formula of the 3*3 deformable convolution layer is:

[0021]

[0022] P=P0+P n +ΔP n (4)

[0023] c(P)=c(P0+P n +ΔP n ) (5)

[0024] In formula (3), R={(-1,-1),(-1,0),…,(0,1),(1,1)} represents a relative position set of 9 points of the convolution kernel, the center is (0,0), P n is each position of the convolution kernel, h is a deformable convolution kernel parameter, c is an input feature, P0 is a single feature point on the feature map, d is an output feature, {ΔP n |n=1,…,N},N=|R|, corresponds to 9 position offsets in R, such as R[0]=(-1,-1), ΔP n is a two-dimensional quantity, corresponding to horizontal and vertical coordinate offsets, Δm n is a coordinate offset additional weight, formula (4) P is a position after adding the offset, which is a floating point number, and formula (5) c(P) is a position feature value after adding the offset, which is equivalent to being obtained by a bilinear interpolation method, and the expression formula is:

[0025]

[0026] G(Q,P)=g(Q x ,P x )·g(Q y ,P y ) (7)

[0027] g(a,b)=max(0,1-|a-b|) (8)

[0028] Wherein c(P) is a position feature value after adding the offset, which is obtained from formula (6), Q is four point positions on the feature map closest to P and less than 1, G(Q,P) is a distance operation performed on Q and P coordinates, c(Q) is a pixel value corresponding to Q, formula (7) is a G(Q,P) formula, which is a multiplication of two g functions, and the g functions are respectively operated on Q and P horizontal and vertical coordinates, Q x ,Q y ,P x ,Py For Q, P corresponds to the coordinates, x represents the horizontal coordinate, y represents the vertical coordinate, formula (8) is the specific formula of g(a, b), a, b is the formula example input, max represents the maximum value of binary comparison, |a-b| represents the absolute value function formula of a, b difference;

[0029] 1x1 convolution for target score prediction, output 1-dimensional vector, containing the target score of the proposal box generated by each point of the feature extraction output feature map, containing the target score, not containing the target score;

[0030] 1x1 convolution for target proposal box boundary regression, output 1-dimensional vector, containing the position information of each size proposal box generated by each point of the feature extraction output feature map, including horizontal and vertical coordinates, length and width dimensions;

[0031] Variable region proposal final output proposal box vector;

[0032] ③Electronic component proposal box feature map generation

[0033] Constructing electronic component proposal box feature map generation, its input is the 1024 feature map output by feature extraction and the proposal box vector output by variable region proposal, mapping the proposal box to the feature map and processing through average pooling, finally outputting the feature map corresponding to each proposal box, and the feature map size is unified to 14x14 and the dimension is 1024;

[0034] ④Electronic component detection box generation

[0035] This contains 3 bottleneck structures, 1 adaptive average pooling layer, 2 parallel fully connected layers, and 2 fully connected layers with 2048 neurons each. The input is the feature map output by the proposal box feature map generation, which is output by the 3 bottleneck structures with a size of 7x7 and a dimension of 2048. The adaptive average pooling layer outputs a 1-dimensional vector with a dimension of 2048. Finally, the 2 fully connected layers predict the class and detection box respectively;

[0036] (3) Training phase of deformable convolution electronic component detection model

[0037] ① Design the loss function of variable region proposal

[0038] The region proposal loss function formula is:

[0039]

[0040] Where, L1({s j},{b j}) is the target and boundary box loss function, j=1,2,…256, s j represents the probability of the jth proposal box being predicted as the real label, The category true value of the proposal box, 1 for positive samples and 0 for negative samples, b j represents the boundary box regression parameter of the jth proposal box, containing 4 position information, represents the boundary box regression parameter of the artificial annotation true box corresponding to the proposal box in the image, N cls = 256, N reg is the number of screened proposal boxes, set to 2000, and λ1 is set to 0.1, L cls is a binary cross-entropy loss function, and the expression formula is as follows:

[0041]

[0042] The variable naming rule of the formula is the same as formula (9);

[0043] L reg The loss function expression formula is as follows:

[0044]

[0045] The variable naming rule of the formula is the same as formula (9);

[0046]

[0047] Formula (12) is The function, and the variable naming rule of the formula is the same as formula (9);

[0048] 2. Designing an electronic component detection box generation loss function

[0049] The electronic component detection box generation loss function formula is:

[0050] L2(p,u,d u ,v)=L obc (p,u)+λ2L reg (d u ,v) (13)

[0051] Where L2(p,u,d u ,v) is the multi-task loss function of the detection network, u corresponds to the target true category label, u takes a value of 6, which is 5 types of electronic components and background, p is the probability distribution predicted by the classifier p=(p0,…p k ), corresponding to 5 types of electronic components and background, k=1,2,…6, d u corresponds to the regression parameter of the corresponding category u predicted by the detection box predictor regression prediction The parameter meanings are respectively the horizontal coordinate d x of the upper left corner of the detection box, the vertical coordinate d y , the width d w of the detection box, and the height dh , v corresponds to the real target detection frame parameter, λ2 is set to 0.2, the detection frame loss function L reg The target classification loss function L obc The formula is:

[0052] L obc (p, u) = -log(p u ) (14)

[0053] Where p u represents the predicted probability of the corresponding category;

[0054] 3. Use the loss function to train the deformable convolution electronic component detection model

[0055] The region proposal loss function and the detection network loss function are summed to obtain the total loss value, and the model is trained through the total loss value and the back propagation algorithm, and the back propagation algorithm expression formula is:

[0056]

[0057]

[0058] Where w new is the updated parameter of the neuron, w old is the existing neuron parameter, E is the total loss value, is the total loss value, and all neuron parameters of the convolutional neural network are derived.

[0059] 4. Complete the deformable convolution electronic component detection model training

[0060] The number of model training iterations is set to 50, and the training is terminated after the number of iterations is reached, the weight parameters of the last training are saved, and the model training is completed.

[0061] (4) Deformable convolution electronic component detection model prediction stage

[0062] After completing the model training, the complete model weight is loaded, the online printed circuit board test set is loaded, the model is input for prediction, the printed circuit board electronic component detection frame is obtained, the printed circuit board electronic component image with the detection frame is output, and the detection is completed. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 The model structure diagram of the present application

[0064] Figure 2 The deformable convolution schematic diagram

[0065] Figure 3 The actual measurement diagram of the present application DETAILED DESCRIPTION

[0066] The application designs a printed circuit board electronic component target detection method based on deformable convolution. The proposed algorithm model can identify and locate PCB electronic components in the industrial waste home appliance recycling process.

[0067] The experimental data set is made by the inventor and consists of 1200 PCB photos (multi-angle), containing 10567 electronic component targets. The electronic components are divided into 5 categories, namely resistor (resistor), cap (capacitor), transformer (transformer), MOV (varistor), and chip (chip). The data set is divided into training set, validation set, and test set in a ratio of 4:1:1 for model training and model prediction stages.

[0068] A printed circuit board electronic component target detection method based on deformable convolution, characterized by comprising the following steps:

[0069] The method includes four stages: data acquisition, deformable convolution electronic component detection model establishment, deformable convolution electronic component detection model training, and deformable convolution electronic component detection model prediction. The specific steps are as follows:

[0070] (1) Data acquisition stage

[0071] Get the printed circuit board electronic component image data set, which includes 5 types of electronic component images: resistor, capacitor, chip, transformer, and varistor. Divide the printed circuit board electronic component image data set into training set, validation set, and test set in a ratio of 4:1:1. Normalize the pixels of each image in the printed circuit board electronic component image data set to [0, 1]. Resize each image to a fixed short side of 600 and a long side less than or equal to 1000.

[0072] (2) Deformable convolution electronic component detection model establishment stage

[0073] The deformable convolution electronic component detection model includes four modules: electronic component feature extraction, deformable region proposal, electronic component proposal box feature map generation, and electronic component detection box generation.

[0074] ① Electronic component feature extraction

[0075] Construct a convolution residual feature extraction, which includes four stages: the first stage consists of a convolution layer with a 7x7 kernel, a batch normalization processing layer, an activation function layer, and a 3x3 max pooling layer. The remaining three stages consist of 3, 4, and 6 bottleneck structures, respectively. Each bottleneck structure is formulated as follows:

[0076] Y i = H(X i ) + F(X i , W i ) (1)

[0077]

[0078] where formula (1) X is the feature input of the i-th bottleneck structure, i = 1, 2, …, 13, Y i is the output of the i-th bottleneck structure, H(X i ) is the direct linear mapping of X i , which contains convolution operation, F(X i , W i ) is the nonlinear mapping, which contains cubic convolution operation, cubic batch normalization and twice activation function, W i is the standard convolution kernel parameter, formula (2) is the bottleneck structure output activation function, X i+1 is the feature input of the i+1-th bottleneck structure, σ(Y i ) is the activation function of the output Y i of the i-th bottleneck structure; the final output feature map dimension of the feature extraction is 1024;

[0079] ② Deformable region proposal

[0080] A deformable region proposal is constructed, which has a structure of 1 convolution kernel of 3x3 deformable convolution layer, and 2 convolution kernels of 1x1 convolution layer connected in parallel, and the specific structure is:

[0081] The expression formula of the 3x3 deformable convolution layer is:

[0082]

[0083] P = P0+ P n + ΔP n (4)

[0084] c(P) = c(P0+ P n + ΔP n ) (5)

[0085] where formula (3), R = {(-1,-1),(-1,0),…,(0,1),(1,1)} represents the relative position set of 9 points of the convolution kernel, the center is (0,0), P n is each position of the convolution kernel, h is the deformable convolution kernel parameter, c is the input feature, P0 is a single feature point on the feature map, d is the output feature, {ΔP n |n = 1, …, N}, N = |R|, corresponding to the 9 position offsets in R, such as R[0] = (-1,-1), ΔPn is a two-dimensional quantity, corresponding to the horizontal and vertical coordinate offset, Am n is a coordinate offset added weight, formula (4) P is the position after adding the offset, is a floating-point number, formula (5) c(P) is the characteristic value of the position after adding the offset, which is equivalent to the position obtained by the bilinear interpolation method, and its expression formula is:

[0086]

[0087] G(Q,P)=g(Q x ,P x )·g(Q y ,P y ) (7)

[0088] g(a,b)=max(0,1-|a-b|) (8)

[0089] Where c(P) is the characteristic value of the position after adding the offset, obtained from formula (6), Q is the four points on the feature map closest to P and the distance is less than 1, G(Q,P) is the distance operation on Q,P coordinates, c(Q) is the pixel value corresponding to Q, formula (7) is the formula for G(Q,P), which is the multiplication of two g functions, and the g function is executed on the horizontal and vertical coordinates of Q and P respectively, Q x ,Q y ,P x ,P y are the corresponding coordinates of Q and P, x represents the horizontal coordinate, and y represents the vertical coordinate, formula (8) is the specific formula of g(a,b), a and b are formula examples, max represents the maximum value of binary comparison, and |a-b| represents the absolute value function formula of a-b;

[0090] 1x1 convolution for target score prediction, output 1-dimensional vector, containing the target score of the proposal box generated by each point of the feature extraction output feature map, containing the target score, not containing the target score;

[0091] 1x1 convolution for target proposal box boundary regression, output 1-dimensional vector, containing the position information of each size proposal box generated by each point of the feature extraction output feature map, including horizontal and vertical coordinates, length and width dimensions;

[0092] Variable region proposal final output proposal box vector;

[0093] ③Electronic component proposal box feature map generation

[0094] The electronic component proposal box feature map generation maps the proposal box to the feature map and performs average pooling processing, finally outputs the feature map corresponding to each proposal box, and the feature map size is unified to 14×14 and the dimension is 1024;

[0095] ④Electronic component detection box generation

[0096] This contains 3 bottleneck structures, 1 adaptive average pooling layer, 2 parallel fully connected layers, and 2 fully connected layers with 2048 neurons each. The input feature map generated by the proposal box is outputted through the 3 bottleneck structures, and the output size is 7×7 and the dimension is 2048. The adaptive average pooling layer outputs a 1-dimensional vector of 2048 dimensions. Finally, the 2 fully connected layers predict the class and detection box respectively;

[0097] (3) Deformable convolution electronic component detection model training stage

[0098] ① Design deformable region proposal loss function

[0099] The region proposal loss function formula is:

[0100]

[0101] Where, L1({s j},{b j}) is the target and boundary box loss function, j=1,2,…256, s j represents the probability of the jth proposal box being predicted as the real label, is the class true value of the proposal box, is 1 when it is a positive sample, and 0 when it is a negative sample, b j represents the boundary box regression parameter of the jth proposal box prediction, which contains 4 position information, represents the boundary box regression parameter of the artificial annotation real box corresponding to this proposal box in the image, N cls =256, N reg is the number of filtered proposal boxes, set to 2000, λ1 is set to 0.1, L cls is a binary cross-entropy loss function, and the expression formula is as follows:

[0102]

[0103] The formula variable naming rule is the same as formula (9);

[0104] L reg The loss function expression formula is as follows:

[0105]

[0106] The naming rules for formula variables are the same as those for formula (9);

[0107]

[0108] Formula (12) is The naming rules for functions and formula variables are the same as those for formula (9);

[0109] ②Design the loss function for generating the detection frame of electronic components

[0110] The formula for the loss function in generating the electronic component detection frame is:

[0111] L2(p,u,d u ,v)=L obc (p,u)+λ2L reg (d u ,v) (13)

[0112] Where L2(p,u,d) u (v) is the multi-task loss function of the detection network, u corresponds to the true category label of the target, u takes a value of 6, representing 5 categories of electronic components and background, and p is the probability distribution predicted by the classifier p = (p0, ..., p) k ), corresponding to 5 categories of electronic components and background, k=1,2,…6,d u The regression parameters of the corresponding category u predicted by the corresponding bounding box predictor The parameters represent the x-coordinate d of the top left corner of the detection box, respectively. x y-coordinate d y Detection box width d w Detection frame height d h v corresponds to the parameters of the true target detection box, λ2 is set to 0.2, and the detection box loss function L reg The region proposal loss function is the same as that in formula (12), and the target classification loss function L obc The formula is:

[0113] L obc (p, u) = -log(p) u (14)

[0114] Where p u This represents the predicted probability for the corresponding category;

[0115] ③ Training a deformable convolutional electronic component detection model using a loss function

[0116] The total loss value is obtained by summing the region proposal loss function and the detection network loss function. The model is then trained using this total loss value and the backpropagation algorithm. The backpropagation algorithm is expressed as follows:

[0117]

[0118]

[0119] where w new is the updated parameter of the neuron, w old is the existing parameter of the neuron, and E is the total loss value, is the partial derivative of the total loss value with respect to all neuron parameters of the convolutional neural network;

[0120] ④ Complete the training of the deformable convolution electronic component detection model

[0121] The number of iterations of model training is set to 50, and the training is terminated after the number of iterations is reached, the weight parameters of the last training are saved, and the model training is completed.

[0122] (4) Deformable convolution electronic component detection model prediction stage

[0123] After completing the model training, the complete model weight is loaded, the printed circuit board test set is loaded online, the model is input for prediction, the printed circuit board electronic component detection frame is obtained, the printed circuit board electronic component image with the detection frame is output, and the detection is completed.

[0124] Table 1 is a detection result graph of the present application

[0125]

Claims

1. A deformable convolution-based printed circuit board electronic component target detection method, characterized in that, Comprising the following steps: Comprising "data collection", "variable convolution electronic component detection model establishment", "variable convolution electronic component detection model training", "variable convolution electronic component detection model prediction" a total of 4 stages, the specific steps are as follows: (1) Data collection stage Obtain printed circuit board electronic component image data set, a total of 5 types of electronic component images, including: resistor, capacitor, chip, transformer, and varistor; divide the printed circuit board electronic component image data set into training set, validation set, and test set, the picture quantity ratio is 4:1:1; normalize the pixels of each image in the printed circuit board electronic component image data set to [0, 1], and perform size scaling on each picture, with the short side fixed at 600 and the long side less than or equal to 1000; (2) Variable convolution electronic component detection model establishment stage The variable convolution electronic component detection model includes electronic component feature extraction, deformable region proposal, electronic component proposal box feature map generation, and electronic component detection box generation; ① Electronic component feature extraction Construct a convolution residual feature extraction, which includes 4 stages: the first stage consists of a convolution layer with a 7x7 kernel, a batch normalization processing layer, an activation function layer, and a 3x3 max pooling layer, and the remaining 3 stages consist of 3, 4, and 6 bottleneck structures respectively, each bottleneck structure is as follows: Y i = H(X i ) + F(X i , W i ) (1) wherein formula (1) X is the feature input of the i-th bottleneck structure, i = 1, 2, …, 13, Y i is the output of the i-th bottleneck structure, H(X i ) is the direct linear mapping of X i , containing convolution operation, F(X i , W i ) is the nonlinear mapping, containing cubic convolution operation, cubic batch normalization and twice activation function, W i is the standard convolution kernel parameter, formula (2) is the bottleneck structure output activation function, X i+1 is the feature input of the i+1-th bottleneck structure, σ(Y i ) is the activation function of the i-th bottleneck structure output Y i ; the final output feature map dimension of feature extraction is 1024; ② Deformable region proposal Construct a deformable region proposal with the structure of a 3x3 deformable convolution layer with a 3x3 kernel, two 1x1 convolution layers connected in parallel, and the specific structure is as follows: The 3x3 deformable convolution layer expression formula is: P = P0+ P n + ΔP n (4) c(P) = c(P0+ P n + ΔP n ) (5) In formula (3), R = {(-1, -1), (-1, 0), ..., (0, 1), (1, 1)} represents the set of relative positions of the nine points of the convolution kernel, centered at (0, 0). n Let $\mathbf{p}$ be the position of the convolution kernel, $h$ be the deformable convolution kernel parameters, $c$ be the input feature, $P0$ be a single feature point on the feature map, and $d$ be the output feature. Let $\mathbf{p}$ be the input feature, ...P0$ be a single feature point on the feature map, and $d$ be the output feature. n |n=1,…,N},N=|R|,corresponding to 9 offset positions in R, such as R[0]=(-1,-1), ΔP n It is a two-dimensional quantity, corresponding to the offset of the horizontal and vertical coordinates, Δm n Adding weights to the coordinate offset, P in formula (4) is the position after adding the offset, and is a floating-point number. In formula (5), c(P) is the position feature value after adding the offset, which is equivalent to obtaining it through bilinear interpolation. Its expression formula is: G(Q, P) = g(Q x , P x ) · g(Q y , P y ) (7) g(a, b) = max(0, 1 - |a - b|) (8) where c(P) is the position feature value after adding the bias, obtained from formula (6), Q is the four points on the feature map closest to P and having a distance less than 1, G(Q, P) is a distance operation performed on the coordinates of Q and P, c(Q) is the pixel value corresponding to Q, formula (7) is the formula of G(Q, P), which is the multiplication of two g functions, the g functions are respectively performed on the horizontal and vertical coordinates of Q and P, Q x , Q y , P x , P y are the corresponding coordinates of Q and P, x represents the horizontal coordinate, and y represents the vertical coordinate, formula (8) is the specific formula of g(a, b), a and b are example inputs of the formula, max represents a binary comparison to obtain the maximum value, and |a-b| represents an absolute value function formula obtained by subtracting a and b; 1x1 convolution for target score prediction, output 1-dimensional vector, containing the target score of the proposal box generated by each point of the feature extraction output feature map, containing the target score, not containing the target score; 1x1 convolution for target proposal box boundary regression, output 1-dimensional vector, containing the position information of each size proposal box generated by each point of the feature extraction output feature map, including horizontal and vertical coordinates, length and width dimensions; The deformable region proposal finally outputs a proposal box vector; ③ Electronic component proposal box feature map generation Construct an electronic component proposal box feature map generation, which inputs the 1024 feature map output by the feature extraction and the proposal box vector output by the deformable region proposal, maps the proposal box to the feature map, and performs average pooling processing, finally outputs the feature map corresponding to each proposal box, and the feature map size is unified to 14x14 and the dimension is 1024; ④ Electronic component detection box generation This contains 3 bottleneck structures, 1 adaptive average pooling layer, and 2 parallel fully connected layers, with 2048 neurons in each of the 2 fully connected layers, inputting the feature map output by the proposal box feature map generation, outputting a feature map with a size of 7x7 and a dimension of 2048 after passing through 3 bottleneck structures, outputting a 1-dimensional vector with a dimension of 2048 after passing through the adaptive average pooling layer, and finally the 2 fully connected layers predict the class and detection box respectively; (3) Deformable convolution electronic component detection model training stage ① Design of deformable region proposal loss function The region proposal loss function formula is: wherein L1({s j}, {b j}) is a target and bounding box loss function, j = 1, 2, … 256, s j represents the probability that the jth proposal box is predicted as the real label, is the class true value of the proposal box, is 1 when it is a positive sample, and is 0 when it is a negative sample, b j represents the bounding box regression parameters for predicting the jth proposal box, containing 4 position information, represents the bounding box regression parameters of the artificial annotation real box corresponding to the proposal box in the image, N cls = 256, N reg is the number of screened proposal boxes, set to 2000, λ1 is set to 0.1, L cls is a binary cross-entropy loss function, and the expression formula is as follows: The variable naming rule of the formula is the same as formula (9); L reg The loss function is expressed as follows: The variable naming rule of the formula is the same as formula (9); Equation (12) is Function, equation variable naming rules with equation (9); ② Design of electronic component bounding box generation loss function The electronic component bounding box generation loss function formula is: L2(p, u, d u , v) = L obc (p, u) + λ2L reg (d u , v) (13) wherein L2(p, u, d u is a detection network multi-task loss function, u corresponds to a target real class label, u takes a value of 6, which is 5 types of electronic components and background, p is a probability distribution p = (p0, … p k ) predicted by a classifier, corresponding to 5 types of electronic components and background, k = 1, 2, … 6, d u corresponds to a regression parameter of a corresponding class u predicted by a bounding box predictor regression The parameter meanings are respectively a horizontal coordinate d x of a top-left corner of a bounding box, a vertical coordinate d y of the top-left corner of the bounding box, a width d w of the bounding box, and a height d h of the bounding box, v corresponds to a real target bounding box parameter, λ2 is set to 0.2, and a bounding box loss function L reg is the same as formula (12), a target classification loss function L obc is formula: L obc (p, u) = -log(p u ) (14) where p u represent the predicted probabilities for the corresponding class. ③ Training of deformable convolution electronic component detection model using loss function The total loss value is obtained by summing the region proposal loss function and the detection network loss function, and the model is trained through the total loss value and the back propagation algorithm. The expression formula of the back propagation algorithm is: where w new is the updated parameter of the neuron, w old is the existing parameter of the neuron, E is the total loss value, is the partial derivative of the total loss value with respect to all neuron parameters of the convolutional neural network; ④ Completion of deformable convolution electronic component detection model training The number of model training iterations is set to 50. After the training reaches the number of iterations, the training is terminated, the weight parameters of the last training are saved, and the model training is completed. (4) Deformable convolution electronic component detection model prediction stage After the model training is completed, the complete model weight is loaded, the printed circuit board test set is loaded online, the model is input for prediction, the printed circuit board electronic component bounding box is obtained, the printed circuit board electronic component image with the bounding box is output, and the detection is completed.

Images