An anchor-free target detection network for electronic components and a detection method using the network

Abstract

The present invention discloses an anchor-box-free target detection network for electronic components and a detection method using the network. The anchor-box-free target detection network for electronic components comprises an encoder, a decoder and a head part, wherein the encoder part comprises an improved backbone feature extraction network EfficientNet, the EfficientNet comprises an MBConv‑ECA module and a variability convolution module, the decoder part comprises a deconvolution module for realizing an upsampling function, and the head part comprises a classification prediction and a bounding box regression prediction module. After the classification prediction and the bounding box regression prediction, the loss calculation is performed on the result of the forward propagation of the anchor-box-free target detection network and the true value, and the back propagation is performed, so as to update the gradient and complete the network training; the anchor-box-free target detection of electronic components is realized by using the network, the category of the electronic components and their position information can be obtained in real time, and the real-time detection of the electronic components is realized.

Description

Technical Field

[0001] The present invention belongs to the field of electronic component recognition and detection, and specifically relates to an anchor-free target detection network for electronic components and a detection method using the network. Background Art

[0002] The identification and detection of electronic components is an important part of realizing the intelligent assembly of electronic components. Therefore, the identification and detection technology for electronic components has also developed rapidly. In the existing technology, the identification and detection of electronic components are mainly concentrated in the following two ways: one is based on traditional template matching and feature point matching to realize the identification and positioning of electronic components, and the other is based on deep learning target detection algorithm to realize the identification and positioning of electronic components.

[0003] However, the above recognition and positioning technologies still have the following defects: 1. Although the traditional template matching and feature point matching are fast, they are not accurate and robust, and have high requirements for lighting and image shooting angles. 2. The target detection algorithm based on deep learning realizes the detection of electronic components based on anchor frames, and the size setting of the anchor frame will directly affect the final detection effect to a large extent. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes an anchor-free target detection network for electronic components and a detection method using the network. There is no need to design anchor frames of corresponding sizes for various electronic components. At the same time, improving the structure of the network can realize accurate identification of electronic components.

[0005] The technical solution adopted by the present invention is as follows:

[0006] An anchor-free target detection network for electronic components includes three parts: an encoder, a decoder, and a head. The encoder uses an improved backbone feature extraction network EfficientNet, which is composed of an MBConv-ECA module and a deformable convolution module. The improved backbone feature extraction network EfficientNet can complete feature extraction of an input image and generate a final feature map.

[0007] The decoder consists of multiple deconvolution modules, which perform deconvolution operations. The decoder part performs deconvolution operations on the final feature map generated by the improved backbone feature extraction network EfficientNet to generate the final predicted feature map.

[0008] The Head part contains two branches: classification prediction and regression prediction. After the final prediction feature map is obtained through the Decoder part, the final prediction feature map is passed to the Head part to generate the classification prediction result and the bounding box regression prediction result.

[0009] Furthermore, the structure of the improved backbone feature extraction network EfficientNet has 8 layers, among which Layer 1 is a common 3×3 convolution, Layer 2-Layer 6 is composed of MBConv-ECA modules, and Layer 7-Layer 8 is composed of variable convolution modules.

[0010] Furthermore, the MBConv-ECA module replaces the attention mechanism SE module in the MBConv module with an ECA module; the MBConv-ECA module includes a 1×1 convolution, a BN layer and a Swish activation function, a k×k depth-separable convolution, an ECA module and a 1×1 convolution.

[0011] Furthermore, the ECA module performs adaptive global average pooling on the feature map after feature extraction to convert the H×W×C feature map into 1×1×C, and then undergoes a one-dimensional convolution with a convolution kernel of k, where k is generated according to the number of channels, as shown in the following expression:

[0012]

[0013] Among them, H, W, and C represent the height, width, and number of channels of the feature map respectively; b and γ are the hyperparameters in the convolution kernel generating function, b = 1, γ = 2, if k is divisible by 2, then k = k, if k is not divisible by 2, then k = k + 1;

[0014] After one-dimensional convolution, each channel can pay attention to the relationship between its k neighboring channels, and then obtain the weight corresponding to each channel through the Sigmoid activation function. The weight dimension is 1×1×C, and then multiplied by the incoming feature map H×W×C to obtain the feature map after the ECA attention module.

[0015] Furthermore, the k×k depthwise separable convolution in the MBConv-ECA module is replaced with a deformable convolution to obtain a deformable convolution module. The deformable convolution learns a direction vector for each convolution kernel parameter, which allows the convolution kernel to be transformed into any shape, improves the generalization of the network, and enhances the feature extraction capability of the network without being limited to a fixed convolution kernel.

[0016] Furthermore, after the classification prediction and bounding box regression prediction, the loss of the forward propagation result of the anchor-free object detection network and the true value are calculated and back-propagated to update the gradient and complete the network training.

[0017] Furthermore, the loss includes classification loss and bounding box regression loss, which is expressed as:

[0018] L 总 =λ1L cls +λ2L reg

[0019] Among them, λ1 and λ2 represent the proportion of classification loss and bounding box regression loss to the total loss, respectively, and L cls is the classification loss, L reg is the bounding box regression loss.

[0020] Furthermore, the method to obtain the classification loss is:

[0021]

[0022] Among them, μ, are the hyperparameters in the loss function, N is the number of positive samples, y is the dependent variable representing the value on the generated heat map, is the predicted value of the classification prediction branch; when y=1, it corresponds to the center point, that is, the loss value corresponding to the positive sample is calculated; when y is not equal to 1, it is a negative sample or an ignored sample, and the negative sample is taken to calculate the corresponding loss value. Since y is 1 near the center and gradually decreases outward, when calculating the loss of negative samples, multiply it by It can make the weight of negative samples close to the center small, and the weight of negative samples far from the center large, thereby completing the calculation of classification loss.

[0023] Furthermore, the method to obtain the bounding box regression loss is:

[0024] The distance from the center point of each positive sample to the four sides of the true box is the true value. Assuming that the coordinates of the center point of the positive sample are (x′, y′), the coordinates of the upper left corner and the lower right corner of the true box are (x0, y0) and (x1, y1), respectively, the distance from the center point of the positive sample to the four sides of the true box is:

[0025]

[0026]

[0027]

[0028]

[0029] According to the result of the bounding box regression prediction branch, the bounding box regression loss can be obtained. The specific calculation formula is as follows:

[0030]

[0031] Among them, only positive samples are involved in the bounding box regression loss, and GIoU loss is used, so the calculation of the regression box loss is completed.

[0032] A method for anchor-free target detection of electronic components. The specific process is as follows:

[0033] S1, collect RGB images of electronic components and input them into the anchor-free target detection network;

[0034] S2, anchor-free object detection network uniformly scales the RGB images of electronic components to 512×512 size;

[0035] S3, input the RGB image of the electronic components processed by S2 into the improved backbone feature extraction network EfficientNet to extract features and generate a feature map.

[0036] S4. Input the feature map into the Decoder partial deconvolution module, and generate the final predicted feature map based on the above feature map.

[0037] S5. The final predicted feature map is passed through the classification prediction module and the bounding box regression prediction module of the Head part to generate a category prediction result and a bounding box regression prediction result.

[0038] S6. After post-processing operations such as screening and non-maximum suppression based on the prediction results, the final prediction box is restored to the original image to obtain the category and specific location information of the electronic components in the original image, completing the network prediction process.

[0039] Beneficial effects of the present invention:

[0040] (1) In the technical solution of the present invention, the MBConv module in EfficientNet is replaced by the MBConv-ECA module, making the network more lightweight. At the same time, the relationship between channels is considered without reducing the dimension, which is more efficient and can improve the final prediction accuracy of the network.

[0041] (2) The MBConv modules in the last two layers of EfficientNet are replaced with DFConv modules. By learning a direction vector for the parameters of each convolution kernel, the convolution kernel can be transformed into any shape and is not limited to a fixed size of convolution kernel. This can improve the network's ability to extract features, enhance the network's generalization ability, and make the network more adaptable.

[0042] (3) Improving the traditional upsampling into a deconvolution module enables these parameters to be updated during the network training process, making the learning ability stronger.

[0043] (4) The heat map generation module generates a corresponding heat map according to the size of the GTBox. The weight gradually decreases from the center to the periphery. When calculating the loss, the weight of negative samples close to the center is made smaller, and the weight of negative samples close to the periphery is made larger. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is the overall network structure diagram of the present invention.

[0045] Figure 2 This is a specific network structure diagram of the present invention.

[0046] Figure 3 This is a diagram of the MBConv-ECA module involved in the present invention.

[0047] Figure 4 This is a diagram of the variable convolution module involved in the present invention.

[0048] Figure 5 This is a diagram of the deconvolution module involved in the present invention.

[0049] Figure 6 This is a frame regression diagram involved in the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] An anchor-free object detection network for electronic components, such as Figure 1 and 2 As shown, the detection network consists of three parts: Encoder, Decoder and Head;

[0052] The encoder uses an improved backbone feature extraction network EfficientNet, which is composed of an MBConv-ECA module and a deformable convolution module. Figure 2 As shown in the specific network structure diagram, the improved backbone feature extraction network EfficientNet includes 8 layers, among which Layer1 is a common 3×3 convolution, Layer2-Layer6 is composed of MBConv-ECA modules, and Layer7-Layer8 is composed of variable convolution modules. The improved backbone feature extraction network EfficientNet can complete the feature extraction of the input image and generate the final feature map.

[0053] like Figure 3The MBConv-ECA module replaces the attention mechanism SE module in the MBConv module with the ECA module. The feature map input to the MBConv-ECA module first undergoes 1×1 convolution to increase the number of input channels of the feature map by n times (when n=1, a 1×1 convolution layer is not required), then passes through the BN layer and the Swish activation function, and then passes through the k×k depthwise separable convolution, where k is 3 or 5. For details, see the attached Figure 3 As shown in the figure, after extracting features through depthwise separable convolution, the feature map extracted through depthwise separable convolution is passed to the ECA module to generate the weights corresponding to each channel. The ECA module is as follows: first, the feature map after feature extraction is adaptively global average pooled to convert the H×W×C feature map into 1×1×C, and then it is subjected to a one-dimensional convolution with a convolution kernel of k, where k is generated according to the number of channels, as shown in the following expression:

[0054]

[0055] Among them, H, W, and C represent the height, width, and number of channels of the feature map respectively; b and γ are the hyperparameters in the convolution kernel generating function in the above formula, b = 1, γ = 2, if k is divisible by 2, then k = k, if k is not divisible by 2, then k = k + 1.

[0056] After one-dimensional convolution, each channel can pay attention to the relationship between its neighboring k channels, and then the weight corresponding to each channel is obtained through the Sigmoid activation function. The weight dimension is 1×1×C, and then multiplied with the incoming feature map H×W×C to obtain the feature map after the ECA attention module, and then through 1×1 convolution to reduce the dimension to obtain the feature map after the MBConv-ECA module. The shortcut branch in the MBConv-ECA module only exists when the input feature map and the output feature map are of the same size. At this time, the step length stride in the depthwise separable convolution is 1.

[0057] Compared with the MBConv-ECA module, the deformable convolution module replaces the k×k depthwise separable convolution in the MBConv-ECA module with a deformable convolution. Deformable convolution learns a direction vector (the offset in the x direction and the offset in the y direction) for each convolution kernel parameter, which allows the convolution kernel to be transformed into any shape, greatly improving the generalization of the network and enhancing the network's feature extraction capabilities without being limited to a fixed convolution kernel.

[0058] The decoder is composed of multiple deconvolution modules. In this embodiment, three deconvolution modules are designed. The decoder part performs an upsampling operation on the final feature map generated by the improved backbone feature extraction network EfficientNet to generate the final predicted feature map. The upsampling operation of the decoder is completed by the deconvolution module. Figure 5 In the deconvolution module shown in the figure, the feature map of the input decoder first undergoes a transposed convolution operation, i.e., a deconvolution operation, and the width and height of the input feature map are doubled, i.e., from H×W×C1 to 2H×2W×C2, and then the deconvolution module operation is completed after the BN and Relu activation functions. Figure 2 As shown in the figure, the improved backbone feature extraction network EfficientNet generates a 16×16×320 feature map after feature extraction, and its feature map becomes 32×32×256 after a deconvolution module, and its feature map becomes 64×64×128 after another deconvolution module, and finally generates a final prediction feature map of 128×128×64 after a deconvolution module. After three deconvolution module processing, the Decoder part is completed. In general, this part further converts the feature map generated by the backbone feature extraction network into the final prediction feature map.

[0059] The Head part contains two branches: classification prediction and regression prediction. After the final prediction feature map is obtained through the Decoder part, the final prediction feature map is passed to the Head part to generate the classification prediction result and the bounding box regression prediction result. Figure 1 As shown in the figure, the classification prediction branch obtains the category prediction, and its output result is 128×128×num_classes, which represents the predicted category probability size corresponding to each grid. The bounding box regression prediction branch is responsible for predicting the distance size from the center point to the four edges, and its output result is 128×128×4, where 4 represents the distance from the center point to the left, top, right, and bottom four edges (l, t, r, b). For details, see the above attached Figure 6 shown.

[0060] After classification prediction and bounding box regression prediction, the entire forward propagation process of the anchor-free object detection network has ended. Next, it is necessary to calculate the loss of the forward propagation result and the true value of the anchor-free object detection network and perform back propagation to update the gradient and complete the network training. The specific process of loss calculation is as follows:

[0061] In the process of loss calculation, we first need to obtain the true value, and then calculate the deviation between the true value and the predicted value, which involves generating a heat map. The heat map is used for loss calculation of the classification prediction branch. First, the RGB images of the collected electronic components are uniformly scaled to 512×512 size when input into the network. At the same time, the real box (GTBox) is also scaled to a size relative to 512×512, and then the GTBox is projected onto a 128×128 all-0 matrix. The center of the GTBox projection is selected as the positive sample (the corresponding position on the matrix is ​​filled with 1), and the GTBox projection box αw×αh (α is between 0-1) is ignored. Samples, and the remaining samples in the projection box are negative samples. At the same time, a heat map is generated in a linear decreasing manner from the projection center to the periphery of the projection box. The specific generation formula of the heat map is as follows:

[0062] y=βx

[0063] Among them, y is the dependent variable representing the value on the generated heat map, x is the independent variable obtained by discretizing the specific value from the projection center to the outside, y=1 at the projection center point, y=0 in the projection bounding box, and β is the linear decreasing slope. After generating the heat map, the classification loss can be calculated. The classification loss includes the classification loss of positive samples and the classification loss of negative samples. The specific calculation formula is as follows:

[0064]

[0065] The classification loss uses the Focal Loss loss function, where μ, are the hyperparameters in the loss function, μ, Let 2, 4, and N be the number of positive samples respectively, and y be the heat map generated above. When y=1, it corresponds to the center point, that is, the loss value corresponding to the positive sample is calculated. When y is not equal to 1, it is a negative sample or an ignored sample, and the negative sample is taken to calculate the corresponding loss value. Since y is 1 near the center and gradually decreases outward, when calculating the loss of negative samples, multiply it by This can make the weight of negative samples close to the center small and the weight of negative samples far from the center large. This completes the calculation of classification loss.

[0066] In the process of calculating the regression box loss, it is necessary to know the distance from the center point of each positive sample to the four sides of the real box (GTBox). This distance is the true value. Assuming that the coordinates of the center point of the positive sample are (x′, y′), the coordinates of the upper left corner and the lower right corner of the real box are (x0, y0) and (x1, y1), respectively, the distance from the center point of the positive sample to the four sides of the real box is:

[0067]

[0068]

[0069]

[0070]

[0071] According to the result of the bounding box regression prediction branch, the bounding box regression loss can be obtained. The specific calculation formula is as follows:

[0072]

[0073] Among them, only positive samples are involved in the bounding box regression loss, and GIoU loss is used, so the calculation of the regression box loss is completed.

[0074] The total loss of the anchor-free object detection network is a linear weighted combination of the above classification loss and the bounding box regression loss, as shown in the following formula:

[0075] L 总 =λ1L cls +λ2L reg

[0076] Among them, λ1 and λ2 represent the proportion of classification loss and bounding box regression loss to the total loss, respectively, and both are between 0 and 1. After the total loss is calculated, back propagation can be performed to update the weights and complete the network training.

[0077] Based on the above anchor-free target detection network for electronic components, this application also proposes an anchor-free target detection method for electronic components. The specific process is as follows:

[0078] S1, collect RGB images of electronic components and input them into the anchor-free target detection network;

[0079] S2, anchor-free object detection network uniformly scales the RGB images of electronic components to 512×512 size;

[0080] S3, input the RGB image of the electronic components processed by S2 into the improved backbone feature extraction network EfficientNet to extract features and generate a feature map.

[0081] S4. Input the feature map into the Decoder partial deconvolution module, and generate the final predicted feature map based on the above feature map.

[0082] S5. The final predicted feature map is passed through the classification prediction module and the bounding box regression prediction module of the Head part to generate a category prediction result and a bounding box regression prediction result.

[0083] S6. After post-processing operations such as screening and non-maximum suppression based on the prediction results, the final prediction box is restored to the original image to obtain the category and specific location information of the electronic components in the original image, completing the network prediction process.

[0084] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A method for constructing an anchor-free target detection network for electronic components, characterized in that: It includes three parts: Encoder, Decoder and Head; the Encoder adopts the improved backbone feature extraction network EfficientNet, which is composed of MBConv-ECA module and deformable convolution module; the feature extraction of the input image can be completed through the improved backbone feature extraction network EfficientNet to generate the final feature map; the MBConv-ECA module replaces the attention mechanism SE module in the MBConv module with the ECA module; the MBConv-ECA module includes 1×1 convolution, BN layer and Swish activation function, k×k depth separable convolution, ECA module and 1×1 convolution; the ECA module performs adaptive global average pooling on the feature map after feature extraction, converts the H×W×C feature map into 1×1×C, and then undergoes a one-dimensional convolution with a convolution kernel of k, where k is generated according to the number of channels, as shown in the following expression: Among them, H, W, and C represent the height, width, and number of channels of the feature map respectively; b and γ are the hyperparameters in the convolution kernel generating function, b = 1, γ = 2, if k is divisible by 2, then k = k, if k is not divisible by 2, then k = k + 1; After one-dimensional convolution, each channel can pay attention to the relationship between its neighboring k channels, and then the weight corresponding to each channel is obtained through the Sigmoid activation function. The weight dimension is 1×1×C, and then multiplied by the incoming feature map H×W×C to obtain the feature map after the ECA attention module; The decoder consists of multiple deconvolution modules, which perform deconvolution operations. The decoder part performs deconvolution operations on the final feature map generated by the improved backbone feature extraction network EfficientNet to generate the final predicted feature map. The Head part contains two branches: classification prediction and regression prediction. After the final prediction feature map is obtained through the Decoder part, the final prediction feature map is passed to the Head part to generate the classification prediction result and the bounding box regression prediction result.

2. According to claim 1, a method for constructing an anchor-free target detection network for electronic components is characterized in that: The structure of the improved backbone feature extraction network EfficientNet has 8 layers, among which Layer 1 is a common 3×3 convolution, Layer 2-Layer 6 is composed of MBConv-ECA modules, and Layer 7-Layer 8 is composed of deformable convolution modules.

3. The method for constructing an anchor-free target detection network for electronic components according to claim 1, characterized in that: The k×k depthwise separable convolution in the MBConv-ECA module is replaced by a deformable convolution to obtain a deformable convolution module. The deformable convolution learns a direction vector for each convolution kernel parameter, which allows the convolution kernel to become any shape, improves the generalization of the network, and enhances the feature extraction capability of the network without being limited to a fixed convolution kernel.

4. The method for constructing an anchor-free target detection network for electronic components according to claim 1, characterized in that: After the classification prediction and bounding box regression prediction, the loss of the forward propagation result of the anchor-free object detection network and the true value are calculated and back-propagated to update the gradient and complete the network training.

5. The method for constructing an anchor-free target detection network for electronic components according to claim 4, characterized in that: The loss includes classification loss and bounding box regression loss, which is expressed as: L 总 =λ1L cls +λ2L reg Among them, λ1 and λ2 represent the proportion of classification loss and bounding box regression loss to the total loss, respectively, and L cls is the classification loss, L reg is the bounding box regression loss.

6. The method for constructing an anchor-free target detection network for electronic components according to claim 5, characterized in that: The method to obtain the classification loss is: Among them, μ, are the hyperparameters in the loss function, N is the number of positive samples, y is the dependent variable representing the value on the generated heat map, is the predicted value of the classification prediction branch; when y=1, it corresponds to the center point, that is, the loss value corresponding to the positive sample is calculated; when y is not equal to 1, it is a negative sample or an ignored sample, and the negative sample is taken to calculate the corresponding loss value. Since y is 1 near the center and gradually decreases outward, when calculating the loss of negative samples, multiply it by It can make the weight of negative samples close to the center small, and the weight of negative samples far from the center large, thereby completing the calculation of classification loss.

7. The method for constructing an anchor-free target detection network for electronic components according to claim 5, characterized in that: The method to obtain the bounding box regression loss is: The distance from the center point of each positive sample to the four sides of the true box is the true value. Assuming that the coordinates of the center point of the positive sample are (x', y'), the coordinates of the upper left corner and the lower right corner of the true box are (x0, y0) and (x1, y1), respectively, then the distance from the center point of the positive sample to the four sides of the true box is: According to the result of the bounding box regression prediction branch, the bounding box regression loss is obtained. The specific calculation formula is as follows: Among them, only positive samples are involved in the bounding box regression loss, and GIoU loss is used, so the calculation of the regression box loss is completed.

8. A method for detecting electronic components without anchor boxes, characterized in that: Based on the anchor-free target detection network constructed by the anchor-free target detection network construction method for electronic components according to claim 1, the target detection method comprises the following steps: S1, collect RGB images of electronic components and input them into the anchor-free target detection network; S2, anchor-free object detection network uniformly scales the RGB images of electronic components to 512×512 size; S3, input the RGB image of the electronic components processed by S2 into the improved backbone feature extraction network EfficientNet to extract features and generate a feature map; S4, input the feature map into the Decoder partial deconvolution module, and generate the final predicted feature map according to the feature map; S5, the final prediction feature map is passed through the classification prediction module and the bounding box regression prediction module of the Head part to generate a category prediction result and a bounding box regression prediction result; S6. After screening and post-processing of non-maximum suppression based on the prediction results, the final prediction box is restored to the original image to obtain the category and specific location information of the electronic components in the original image, thus completing the network prediction process.

Images