Electronic component target and angle detection method based on deep learning
The deep learning-based electronic component target and angle detection method solves the problem of insufficient full-angle range modeling capability in existing technologies, and achieves high-precision and low-latency electronic component detection, which is suitable for resource-constrained or real-time-critical scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING DAHENG IMAGE VISION CO LTD
- Filing Date
- 2026-03-02
- Publication Date
- 2026-05-19
AI Technical Summary
Existing rotating target detection methods work stably within the range of [0°, 180°], which is insufficient for applications such as electronic component detection that require a complete characterization of the target's orientation. Furthermore, solutions that extend to the full angular range typically increase model size and computational burden, reducing real-time performance and deployability.
A deep learning-based method for detecting electronic components and their angles is adopted. By collecting, cleaning, and labeling PCB board image data, a neural network model is built. Modules such as CBS, Bottleneck, C3k, SPPF, and PSA are used, combined with attention mechanisms and multi-scale feature pyramids. An angle difference coefficient is introduced to correct the cross-union ratio loss, enabling end-to-end inference and reducing redundant computation.
It achieves accurate modeling of the entire angular range of [0°, 360°], improves detection accuracy and robustness, reduces inference latency, and is suitable for real-time applications in embedded or edge scenarios.
Smart Images
Figure CN122066932A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of image recognition and active optical inspection technology, and in particular to a method for detecting electronic components and angles based on deep learning. Background Technology
[0002] Object detection is one of the most important and comprehensively developed research areas in computer vision, and deep learning methods have now become the mainstream approach for this task. Based on the alignment of the bounding boxes and detection boxes, this research area can be divided into the more common horizontal object detection and rotated object detection. In horizontal object detection, all boxes are aligned with both the horizontal and vertical axes, making it more suitable for natural scene images. However, for more accurate object localization, such as scene text detection or remote sensing image analysis, rotated object detection is a more suitable method, as described in existing technology Đ. Nedeljković, “YUDO: YOLO for Uniform Directed Object Detection,” arXiv preprint arXiv:2308.04259, Aug.2023.
[0003] In rotating target detection, traditional methods often represent the detection box using the center point, width, height, and angle (typically within the range of [0°, 180°]). However, in applications such as electronic component inspection, it is often necessary to accurately model the complete orientation of the target, i.e., support detection within the angle range of [0°, 360°) to meet high-precision requirements for modeling, matching, and defect localization. However, existing technologies still have shortcomings in their ability to meet this complete angle range: some methods that can cover [0°, 360°) rely on more complex angle representations or larger models and post-processing strategies, significantly increasing parameter scale and computational burden, and leading to higher inference latency. This reduces the usability and practicality in resource-constrained or real-time-critical electronic component inspection scenarios. Summary of the Invention
[0004] Given that existing rotating target detection methods typically only operate stably within a limited angular range ([0°, 180°]), they are insufficient for applications such as electronic component detection that require a complete characterization of target orientation, and lack the ability to accurately model the full angular range ([0°, 360°). Furthermore, some solutions that extend to the full angular range often rely on more complex angle representations, discretized classification or multi-branch regression designs, and redundant post-processing steps, significantly increasing model size and computational burden, and reducing real-time performance and deployability. Therefore, this invention aims to propose an improved rotating target detection method.
[0005] This invention discloses a method for detecting electronic components and their angles based on deep learning. The method includes the following steps:
[0006] Step 1: Acquire image data of electronic components, process the image data, and generate a dataset;
[0007] Step 2: Build a neural network model for detecting the target angle of electronic components, and train the network model using the dataset;
[0008] Step 3: Load the trained model parameters, obtain the result feature map through the model forward operation, and restore it according to the labeled form to obtain the network model prediction result.
[0009] Step 4: Input the image data of the electronic component to be detected into the trained model to predict the target position and angle of the electronic component.
[0010] Furthermore, step 1 also includes the following steps:
[0011] Step 101: Use an industrial camera to collect sufficient PCB board image data at the production site;
[0012] Step 102: Clean the collected data and retain image data that is clear, properly exposed, and has obvious features;
[0013] Step 103: Crop the PCB board image. During cropping, overlapping areas are allowed to be retained in the images, so that the data size of each cropped image is 640x640 pixels.
[0014] Step 104: Label the electronic components in the PCB board image. The labeling form is a rectangle in any direction. All categories of labels must be defined with a 0-degree standard. For each category, a unique direction is selected as the 0-degree positive direction.
[0015] The label format is [category, rectangle width, rectangle height, rectangle center point coordinates, angle between the rectangle direction vector and the x-axis in the Cartesian coordinate system], where the angle is the rotation angle of clockwise rotation from the standard positive direction as the initial state to the target state, and the range is [0°, 360°).
[0016] Finally, the labeled dataset is split into training, validation, and test sets.
[0017] Furthermore, step 2 also includes the following steps:
[0018] Step 21: Build a neural network model for detecting the target angle of electronic components;
[0019] Specifically, it also includes the following steps:
[0020] Step 201: Combine one convolutional layer, one batch normalization layer, and one SiLU activation function layer into a CBS module;
[0021] Step 202: Combine the two CBS modules and one jump connection module into a Bottleneck module;
[0022] Step 203: After inputting the features into one CBS module, the results are input into one of N Bottleneck modules and another CBS module respectively, and then the features are concatenated. Finally, they are input into one CBS module to form a C3k module.
[0023] Step 204: Combine 1 CBS module, 2 feature separation modules, N C3k modules, 1 feature concatenation module, and 1 CBS module into a C3k2 module. After the features pass through the first CBS module and the first feature separation module, they are concatenated with the features after passing through the C3k module and then output. At this time, the module parameter is C3k=True. When the parameter C3k=False, the C3k module in the module will be replaced by the Bottleneck module.
[0024] Step 205: Combine one CBS module, three consecutive max pooling layers, one feature concatenation layer, and one CBS module into an SPPF module;
[0025] Step 206: Combine the attention module with the two CBS modules to form a PSA module;
[0026] Step 207: Combine the CBS module, N PSA modules, and the feature splicing module into a C2PSA module;
[0027] Step 208: Combine two CBS modules and one convolutional layer to form the localization detection head module D1;
[0028] Step 209: Combine the depthwise convolution, CBS module, and convolutional layer to form the classification detection head module D2;
[0029] Step 210: Combine two CBS modules and one convolutional layer to form the angle detection head module D3;
[0030] Step 211: Combine the positioning, classification, and angle detection head modules D1, D2, and D3 into the overall detection head module D4.
[0031] Furthermore, step 2 also includes the following steps:
[0032] Step 22: Train the network model using the training dataset;
[0033] Step 22 also includes:
[0034] Step 212: Input the images from the training dataset into two concatenated CBS modules, one C3k2 module, one CBS module, and one C3k2 module to obtain feature map n1;
[0035] Step 213: Input feature map n1 into a cascaded CBS module and a C3k2 module to obtain feature map n2;
[0036] Step 214: Input feature map n2 into a concatenated SPPF and C2PSA module to obtain feature map n3;
[0037] Step 215: Upsample feature map n3 to the same size as feature map n2, and concatenate it with feature map n2 in the channel direction. Then input it into one C3k2 module to obtain feature map P1.
[0038] Step 216: Upsample feature map P1 to the same size as feature map n1, and concatenate it with feature map n1 in the channel direction. Then input it into one C3k2 module to obtain feature map P2.
[0039] Step 217: After inputting feature map P2 into one CBS module, it is concatenated with feature map P1 in the channel direction to obtain feature map P3;
[0040] Step 218: After inputting feature map P3 into one C3k2 module and one CBS module, it is concatenated with feature map n3 in the channel direction to obtain feature map P4;
[0041] Step 219: Input feature map P4 into a C3k2 module to obtain feature map P5;
[0042] Step 220: Input feature map P2 as the first feature map into the detection head module D4 to obtain feature vector t1; input the result of feature map P3 through module C3k2 as the second feature map into the detection head module D4 to obtain feature vector t2; input feature map P5 as the third feature map into the detection head module D4 to obtain feature vector t3.
[0043] The first feature map is downsampled by 8×, the second feature map by 16×, and the third feature map by 32×; the feature tensor t1 corresponds to the first feature map, the feature tensor t2 corresponds to the second feature map, and the feature tensor t3 corresponds to the third feature map.
[0044] Step 221: The feature tensors t1, t2, and t3 obtained from the three scales are fused to obtain the position information (x, y, w, h), angle information, and class probability of all boxes at the three scales. Positive samples are obtained by filtering all predicted boxes based on the position and angle information. The three-class losses of the positive sample predicted boxes are weighted and summed according to preset weights to obtain the final loss result of the overall detection head module D4. Backpropagation is performed based on this loss value to complete the parameter iteration of the D4 module and the upstream network layer.
[0045] Step 222: Train the neural network model using the Adam method. Repeat steps 212 to 221 until the model performs well on the validation set and there is no overfitting.
[0046] Furthermore, in step 206, an attention module is added, and the calculation formula is as follows:
[0047]
[0048] The normalized exponential function normalizes the correlation scores into a probability distribution. This represents the query vector, indicating the current focus of information. This represents a value vector, containing the actual information that needs to be output. This represents a set of key vectors used to calculate relevance with the query vector. This represents the dimension of the key vector, used for scaling dot product operations.
[0049] Furthermore, in step 208, in order to improve the angle sensitivity of the detection frame and simultaneously address the impact of angle periodicity and abrupt changes on the results, this invention proposes an angle difference coefficient. The crossover ratio loss is corrected and calculated as follows:
[0050]
[0051]
[0052] in, This represents the processed angle difference, ranging from [- , ), and These represent the actual rotation angle and the predicted rotation angle, respectively; mod is the modulo operation; an angle difference coefficient is used. The complete intersection-union loss is corrected and added to the distribution focus loss (DFL) to form the detection box loss (BboxLoss), which measures the difference between the D1 output and the true sample value.
[0053] The calculation process for the cross-union ratio loss DirIoULoss based on angle difference correction is as follows:
[0054]
[0055]
[0056]
[0057]
[0058] This is the ratio of the intersection area of the predicted bounding box and the ground truth bounding box to the union area of the predicted bounding box and the ground truth bounding box. The distance between the center points of the predicted bounding box and the ground truth bounding box. The minimum diagonal distance between the bounding rectangles of the predicted bounding box and the ground truth bounding box. This represents the aspect ratio consistency coefficient. Indicates the weighting coefficient. Indicates the loss of a complete intersection and union; , )and( , ) represent the width and height of the target bounding box and the predicted bounding box, respectively; The angle difference coefficient;
[0059] The calculation process for the distributed focus loss (DFL) is as follows:
[0060]
[0061] N is the number of samples, and C is the number of classes. It is the true label of the i-th sample. It is the predicted probability that the i-th sample belongs to category c; It is a balancing factor used to adjust the weights between positive and negative samples; It is a focusing parameter used to control the degree of attention given to difficult samples.
[0062] Furthermore, in step 209, the feature output of D2 is compared with the true sample class, and the binary cross-entropy loss (BCE) is used as the class loss to measure the difference between the D2 output and the true sample class; the calculation process of BCE is as follows:
[0063]
[0064] Represents the binary cross-entropy loss. This represents the true label, with a value of 0 or 1 (0 represents the negative class, and 1 represents the positive class). This represents the probability value (range 0 ≤ a ≤ 1) that the model predicts the sample belongs to the positive class.
[0065] Furthermore, in step 210, in order to increase the contribution of angle to the result and thus improve the accuracy of model angle prediction, the feature output of D3 is compared with the true angle of the sample, and an angle loss AngleLoss is added.
[0066] The calculation process for AngleLoss is as follows:
[0067]
[0068]
[0069] Indicates angle loss, This represents the processed angle difference, ranging from [- , ).
[0070] Furthermore, in step 211, the overall detection head module D4 retains independent prediction result output channels for the three branches: the positioning detection head module D1, the classification detection head module D2, and the angle detection head module D3. The positioning detection head module D1 branch outputs the target position parameter prediction result, the classification detection head module D2 branch outputs the target category probability prediction result, and the angle detection head module D3 branch outputs the target rotation angle prediction result. These three prediction results serve as the core basis for the subsequent dynamic label allocation of the three-scale prediction boxes P3, P4, and P5, and are used to select the optimal prediction box that matches the real annotation.
[0071] Meanwhile, the total loss of the total detection head module D4 is defined as the result of the weighted sum of the localization loss corresponding to the localization detection head module D1 branch, the classification loss corresponding to the classification detection head module D2 branch, and the angle loss corresponding to the angle detection head module D3 branch, according to preset weights. This total loss will be used for network parameter correction in the subsequent backpropagation process.
[0072] Furthermore, in step 221, all predicted boxes are filtered based on position and angle information. That is, by calculating the intersection-union ratio (IU) with the label result box, the predicted boxes with IU greater than a set threshold and the predicted boxes with the highest IU with the label result box are used as positive samples to calculate the loss.
[0073] The beneficial effects achieved by this invention are:
[0074] By predicting the orientation vector of the object to be detected, the angle prediction range is extended to [0°, 360°) and the periodic abrupt changes in angle loss are eliminated.
[0075] By supervising the training of shallow texture information in the network and constructing a multi-scale image feature pyramid within the network structure, and fusing shallow neural network features, enhanced extraction of fine texture and edge information is achieved. An attention mechanism and a multi-scale receptive field module are introduced at the feature fusion point to adaptively suppress complex background interference and improve the allocation of useful channel and spatial information. When detecting complex and textured electronic components, this significantly improves feature representation and anti-interference capabilities, thereby enhancing detection accuracy, model generalization, and robustness.
[0076] The neural network part of the algorithm adopts an end-to-end inference approach, generating detection boxes directly from the confidence and regression information of key points on the feature map based on the improved feature head. This avoids the non-maximum suppression (NMS) step that relies on post-processing in traditional methods, thereby reducing redundant computation and complex post-processing logic. At the same time, the lightweight module design and efficient feature fusion enable the model to have lower latency and higher throughput in the inference stage, making it suitable for embedded or edge scenarios. Attached Figure Description
[0077] Figure 1 Here is a schematic diagram of a neural network model structure according to an embodiment of the present invention.
[0078] Figure 2 The diagram shows a basic convolutional module (CBS) according to an embodiment of the present invention.
[0079] Figure 3 The diagram shows a bottleneck module according to an embodiment of the present invention.
[0080] Figure 4 Here is a schematic diagram of a bottleneck-type feature extraction module 1 (C3k) according to an embodiment of the present invention.
[0081] Figure 5 Here is a schematic diagram of the bottleneck feature extraction module 2 (C3k2, c3k=True) according to an embodiment of the present invention.
[0082] Figure 6 The diagram shows a bottleneck-type feature extraction module 3 (C3k2, c3k=False) according to an embodiment of the present invention.
[0083] Figure 7 Here is a schematic diagram of a fast spatial pyramid pooling module (SPPF) according to an embodiment of the present invention.
[0084] Figure 8 Here is a schematic diagram of a local spatial attention module (PSABlock) according to an embodiment of the present invention.
[0085] Figure 9Here is a schematic diagram of the attention feature extraction module (C2PSA) according to an embodiment of the present invention.
[0086] Figure 10 Here is a schematic diagram of a target bounding box detection head (D1) according to an embodiment of the present invention.
[0087] Figure 11 Here is a schematic diagram of a classification detection head (D2) according to an embodiment of the present invention.
[0088] Figure 12 Here is a schematic diagram of an angle detection head (D3) according to an embodiment of the present invention. Detailed Implementation
[0089] The present invention will be further described below with reference to specific embodiments, and the advantages and features of the present invention will become clearer as a result. However, these embodiments are merely exemplary and do not constitute any limitation on the scope of the present invention. Those skilled in the art should understand that modifications or substitutions can be made to the details and form of the technical solutions of the present invention without departing from the spirit and scope of the present invention, but all such modifications and substitutions fall within the protection scope of the present invention.
[0090] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0091] The specific technical solution of this invention provides a method for detecting electronic component targets and angles based on deep learning, comprising the following steps:
[0092] Step 1: Acquire image information of electronic components. For example, PCB board image data, and then clean and label the data.
[0093] Step 101: Use an industrial camera to collect sufficient PCB board image data at the production site;
[0094] Step 102: Clean the collected data and retain image data that is clear, properly exposed, and has obvious features;
[0095] Step 103: Crop the PCB board image. During cropping, overlapping areas are allowed to be retained in the images, so that the data size of each cropped image is 640x640 pixels.
[0096] Step 104: Label the electronic components in the PCB board image. The labeling form is a rectangle in any direction. All categories of labels must have a 0-degree standard. For each category, a unique direction is selected as the 0-degree positive direction.
[0097] Its representation method is [category (starting from 0, and so on, such as 0, 1, 2), rectangle width, rectangle height, rectangle center point coordinates (x, y), the angle between the rectangle direction vector and the x-axis in the Cartesian coordinate system (angle range [0, 360) clockwise is positive)]. In order to accurately model the complete orientation of the electronic component target, unlike the general case where the tilt angle is based on the horizontal direction with a range of [0°, 180°), this angle is the rotation angle of the standard positive direction as the initial state, and rotated clockwise to the target state, with a range of [0°, 360°).
[0098] In one embodiment, the label is represented as [0,50,20,100,200,45].
[0099] The labeled dataset is divided into training, validation, and test sets, with a ratio of 7:2:1.
[0100] Step 2: Build a neural network model for detecting the target angle of electronic components, and train the network model using the data obtained in Step 1.
[0101] Step 21: Build a neural network model for detecting the target angle of electronic components;
[0102] Figure 1 This is a schematic diagram of the neural network structure of the present invention, which specifically includes the following steps:
[0103] Step 201: Concatenate one convolutional layer, one batch normalization layer, and one SiLU activation function layer to form a CBS module, as shown below. Figure 2 As shown.
[0104] Step 202: Connect the two CBS modules and one jump connection module in series to form the Bottleneck module, as follows: Figure 3 As shown.
[0105] Step 203: After inputting the features into one CBS module, the results are input into one route of N Bottleneck modules and another CBS module respectively, then the features are concatenated, and finally input into one CBS module to form a C3k module. For example... Figure 4 As shown.
[0106] Step 204: Combine 1 CBS module, 2 feature separation modules, N C3k modules, 1 feature concatenation module, and 1 CBS module into a C3k2 module. Features, after passing through the first CBS module and the first feature separation module, are then concatenated with the features after passing through the C3k modules using a skip concatenation technique before output. At this point, the module parameter is C3k=True. Figure 5 As shown. When parameter C3k=False, the N C3k modules in the module will be replaced with N Bottleneck modules. Figure 6 As shown.
[0107] Step 205: Combine one CBS module, three consecutive max pooling layers, one feature concatenation layer, and one CBS module into an SPPF module; wherein the features output by the CBS module and the first two max pooling layers are all concatenated with the features output by the third max pooling layer in a skip concatenation manner, such as... Figure 7 As shown.
[0108] Step 206: Combine the attention module with two CBS modules to form a PSA module. The input feature tensor is first weighted by the attention module. If residual connections are enabled, the attention output is added to the input tensor. The result is then fed into the two CBS modules for further feature extraction. Similarly, when residual connections are enabled, the feedforward output is added to the current tensor. Finally, the enhanced feature tensor is output, thereby achieving global feature relationship modeling and local detail supplementation, improving feature representation capabilities. Figure 8 As shown. The formula for calculating attention is:
[0109]
[0110] Step 207: Combine the CBS module, N PSA modules, and the feature concatenation module into a C2PSA module. The input feature map is expanded through the CBS module (1×1 convolution) and split into two branches. One branch enters a sequence composed of n PSABlocks for multi-head attention and feedforward network processing, and residual connections are enabled to optimize the gradient. This is then fused with the features retained in the other branch. Finally, the original number of channels is restored through the CBS module (1×1 convolution), outputting enhanced features that fuse location-sensitive information and local details, balancing computational efficiency and multi-scale feature representation capabilities. Figure 9 As shown.
[0111] Step 208: Combine two CBS modules and one convolutional layer to form a localization detection head module D1, as follows: Figure 10As shown, the feature output of D1 is compared with the ground truth bounding boxes of the samples, and BboxLoss is used to measure the difference between the output of D1 and the ground truth. BboxLoss consists of two loss functions: the intersection-union ratio loss DirIoULoss based on angle difference correction and the distribution focus loss DFL, using the angle difference coefficient. The complete intersection-union loss is corrected and added to the distribution focus loss (DFL) to form the detection box loss (BboxLoss), which measures the difference between the D1 output and the true sample value.
[0112] The calculation process of DirIoULoss is as follows:
[0113]
[0114]
[0115]
[0116]
[0117] This is the ratio of the intersection area of the predicted bounding box and the ground truth bounding box to the union area of the predicted bounding box and the ground truth bounding box. The distance between the center points of the predicted bounding box and the ground truth bounding box. The minimum diagonal distance between the bounding rectangles of the predicted bounding box and the ground truth bounding box. This represents the aspect ratio consistency coefficient. Indicates the weighting coefficient. Indicates the loss of a complete intersection and union; , )and( , ) represent the width and height of the target bounding box and the predicted bounding box, respectively; DirCorr is the angle difference coefficient. The crossover ratio loss is corrected and calculated as follows:
[0118]
[0119]
[0120] in, This represents the processed angle difference, ranging from [- , ), and These represent the actual rotation angle and the predicted rotation angle, respectively, with mod being the modulo operation.
[0121] The calculation process for the distributed focus loss (DFL) is as follows:
[0122]
[0123] in:
[0124] N is the number of samples.
[0125] C represents the number of categories.
[0126] It is the true label of the i-th sample.
[0127] It is the predicted probability that the i-th sample belongs to category c.
[0128] It is a balancing factor used to adjust the weights between positive and negative samples.
[0129] It is a focusing parameter used to control the degree of attention given to difficult samples.
[0130] Step 209: Connect the depthwise convolution, CBS module, and convolutional layer sequentially to form the classification detection head module D2, as shown below. Figure 11 As shown. The feature output of D2 is compared with the true sample class, and the binary cross-entropy loss (BCE) is used as the class loss to measure the difference between the D2 output and the true sample class. The calculation process of BCE is as follows:
[0131]
[0132] Step 210: Combine two CBS modules and one convolutional layer to form the angle detection head module D3, as shown below. Figure 12 As shown. The feature output of D3 is compared with the sample ground truth, and AngleLoss is used to measure the difference between the D3 output and the sample ground truth. The calculation process of AngleLoss is as follows:
[0133]
[0134]
[0135] This represents the processed angle difference, ranging from [- , ).
[0136] Step 211: The positioning detection head module D1, classification detection head module D2, and angle detection head module D3 are integrated to form a total detection head module D4. The D4 module retains the independent prediction result output channels of the three branches D1, D2, and D3. The D1 branch outputs the target position parameter prediction result, the D2 branch outputs the target category probability prediction result, and the D3 branch outputs the target rotation angle prediction result. These three prediction results serve as the core basis for the subsequent dynamic label allocation of the three-scale prediction boxes P3, P4, and P5, and are used to select the optimal prediction box that matches the real label. At the same time, the total loss of the D4 module is defined as the result of the weighted sum of the positioning loss corresponding to the D1 branch, the classification loss corresponding to the D2 branch, and the angle loss corresponding to the D3 branch according to preset weights. This total loss will be used for network parameter correction in the backpropagation process in the subsequent step 221.
[0137] Step 22: Train the network model using the training dataset;
[0138] In the description, ordinary feature maps generated internally by the neural network are labeled with 'n', for example, feature map n1, feature map n2. Key feature maps are represented by 'P', for example, feature map P1, feature map P2.
[0139] Step 212: The image is input into two CBS modules, one C3k2 module, one CBS module, and one C3k2 module in series to obtain feature map n1.
[0140] Step 213: Input feature map n1 into a cascaded CBS module and a C3k2 module to obtain feature map n2.
[0141] Step 214: Input feature map n2 into a concatenated SPPF and C2PSA module to obtain feature map n3. At this point, feature extraction is initially completed.
[0142] Step 215: Upsample feature map n3 to the same size as feature map n2, and concatenate them in the channel direction. Then input them into one C3k2 module to obtain feature map P1.
[0143] Step 216: Upsample feature map P1 to the same size as feature map n1 in width and height, and concatenate it with feature map n1 in the channel direction. Then input it into one C3k2 module to obtain feature map P2.
[0144] Step 217: After inputting feature map P2 into one CBS module, it is concatenated with feature map P1 in the channel direction to obtain feature map P3.
[0145] Step 218: After inputting feature map P3 into one C3k2 module and one CBS module, it is concatenated with feature map n3 in the channel direction to obtain feature map P4.
[0146] Step 219: Input feature map P4 into a C3k2 module to obtain feature map P5.
[0147] Step 220: After the feature map is input into D4, it is processed through branches D1, D2, and D3. Branch D1 performs position prediction mapping on the original image using the corresponding candidate grid positions, obtaining a localization prediction tensor with a size of (number of candidate boxes at the corresponding scale × 4), storing the center coordinates (x, y) and width (W) and height (H) of the bounding box. Branch D2 obtains a class prediction tensor with a size of (number of candidate boxes at the corresponding scale × number of classes). Branch D3 obtains an angle prediction tensor with a size of (number of candidate boxes at the corresponding scale × 1). Therefore, the results from the three branches are fused, and the output feature vector of D4 has a size of (number of candidate boxes at the corresponding scale × (number of classes + 5)). Based on the above process, feature map P2 is input as the first feature map into the detection head module D4 to obtain feature vector t1 (8× downsampling); the result of feature map P3 passing through the C3k2 module is input as the second feature map into the detection head module D4 to obtain feature vector t2 (16× downsampling); feature map P5 is input as the third feature map into the detection head module D4 to obtain feature vector t3 (32× downsampling).
[0148] Step 221 involves fusing the feature tensors t1, t2, and t3 obtained from the three scales to obtain the position information (x, y, w, h), angle information, and class probability of all boxes at the three scales. All predicted boxes are then filtered based on the position and angle information. Specifically, by calculating the intersection-union ratio (IU) with the labeled result boxes, predicted boxes with IU greater than a set threshold and the predicted box with the highest IU with the labeled result boxes are used as positive samples for loss calculation. The three class losses of the positive sample predicted boxes are then weighted and summed according to preset weights to obtain the final loss result of the overall detection head module D4. Backpropagation is performed based on this loss value to complete the parameter iteration of the D4 module and the upstream network layers.
[0149] Step 222: Train the neural network model using the Adam method. Repeat steps 212 to 221 until the model performs well on the validation set and there is no overfitting.
[0150] Step 3: After training is complete, load the trained model parameters, perform forward computation on the model to obtain the result feature map, and restore it according to the labeled format to obtain the network model prediction result.
[0151] Step 301: Based on the characteristics of the result feature map, convert the information on the feature map into the same format as the labeled information to obtain the final network prediction result.
[0152] Step 4: Input the photo of the PCB board to be inspected into the trained model to predict the target position and angle of the electronic components.
[0153] To objectively evaluate the performance of this invention on the [0°, 360°) rotating target detection task, key experimental verifications were conducted:
[0154] Detection accuracy;
[0155] Inference efficiency (adapted to embedded / real-time scenarios);
[0156] Compared with the existing technology, it has comprehensive advantages of "a deep learning-based method for angle detection and correction of industrial product images".
[0157] First, the dataset and data preparation:
[0158] A large number of PCB board photos were collected from the actual production line. After screening, cleaning, labeling, and cutting, 10,000 sample images with a size of 640x640 were obtained.
[0159] The training set, validation set, and test set are divided in a 7:2:1 ratio for use in the present invention and prior art.
[0160] Evaluation indicators:
[0161] 1. mAP@0.5, the higher the better, the calculation process is as follows:
[0162]
[0163]
[0164] in:
[0165] For the i-th distinct recall value, For the precision corresponding to this recall interval
[0166] C is the number of categories. For class c, AP at IoU=0.5
[0167] Latency, the average inference time per image, in milliseconds; the lower the better.
[0168] Experimental Platform
[0169] CPU: Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
[0170] GPU: NVIDIA A100 40G
[0171] RAM: 756GB
[0172] CUDA: 12.1
[0173] PyTorch: 2.2.2
[0174] Python: 3.11.1
[0175] System version: Ubuntu 18.04
[0176] Experimental steps
[0177] On the same training set, the models are trained according to the methods specified in the two inventions. Training is stopped when the loss value of the model on the validation set is good and there is no overfitting.
[0178] Inference is performed on the test set according to the methods specified in the two inventions, and the performance of the model on the test set is evaluated.
[0179] Performance Comparison
[0180] method mAP@0.5 Latency (ms) This invention 0.956 19 Comparison Methods 0.902 22
[0181] In summary, based on the experimental quantitative results, this invention not only effectively solves the problems of angle periodicity and abrupt change in the [0°, 360°) rotating target detection task, but also shows significant advantages over the existing solution "A Deep Learning-Based Method for Angle Detection and Correction of Industrial Product Images" in terms of detection accuracy and inference efficiency. It is suitable for widespread use in practical application scenarios with high precision and low latency, such as electronic component detection.
[0182] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the scope of protection of the present invention; all technical solutions formed by equivalent transformations or equivalent substitutions fall within the scope of protection of the present invention; the parts of the present invention not described in detail are well-known technologies to those skilled in the art.
Claims
1. A method for detecting targets and angles in electronic components based on deep learning, characterized in that, The deep learning-based method for detecting electronic components and their angles includes the following steps: Step 1: Acquire image data of electronic components, process the image data, and generate a dataset; Step 2: Build a neural network model for detecting the target angle of electronic components, and train the network model using the dataset; Step 3: Load the trained model parameters, obtain the result feature map through the model forward operation, and restore it according to the labeled form to obtain the network model prediction result. Step 4: Input the image data of the electronic component to be detected into the trained model to predict the target position and angle of the electronic component.
2. The method for detecting electronic components and angles based on deep learning according to claim 1, characterized in that, Step 1 also includes the following steps: Step 101: Use an industrial camera to collect sufficient PCB board image data at the production site; Step 102: Clean the collected data and retain image data that is clear, properly exposed, and has obvious features; Step 103: Crop the PCB board image. During cropping, overlapping areas are allowed to be retained in the images, so that the data size of each cropped image is 640x640 pixels. Step 104: Label the electronic components in the PCB board image. The labeling form is a rectangle in any direction. All categories of labels must be defined with a 0-degree standard. For each category, a unique direction is selected as the 0-degree positive direction. The label format is [category, rectangle width, rectangle height, rectangle center point coordinates, angle between the rectangle direction vector and the x-axis in the Cartesian coordinate system], where the angle is the rotation angle of clockwise rotation from the standard positive direction as the initial state to the target state, and the range is [0°, 360°). Finally, the labeled dataset is split into training, validation, and test sets.
3. The method for detecting electronic components and angles based on deep learning according to claim 1, characterized in that, Step 2 also includes the following steps: Step 21: Build a neural network model for detecting the target angle of electronic components; Specifically, it also includes the following steps: Step 201: Combine one convolutional layer, one batch normalization layer, and one SiLU activation function layer into a CBS module; Step 202: Combine the two CBS modules and one jump connection module into a Bottleneck module; Step 203: After inputting the features into one CBS module, the results are input into one of N Bottleneck modules and another CBS module respectively, and then the features are concatenated. Finally, they are input into one CBS module to form a C3k module. Step 204: Combine 1 CBS module, 2 feature separation modules, N C3k modules, 1 feature concatenation module, and 1 CBS module into a C3k2 module. After the features pass through the first CBS module and the first feature separation module, they are concatenated with the features after passing through the C3k module and then output. At this time, the module parameter is C3k=True. When the parameter C3k=False, the C3k module in the module will be replaced by the Bottleneck module. Step 205: Combine one CBS module, three consecutive max pooling layers, one feature concatenation layer, and one CBS module into an SPPF module; Step 206: Combine the attention module with the two CBS modules to form a PSA module; Step 207: Combine the CBS module, N PSA modules, and the feature splicing module into a C2PSA module; Step 208: Combine two CBS modules and one convolutional layer to form the localization detection head module D1; Step 209: Combine the depthwise convolution, CBS module, and convolutional layer to form the classification detection head module D2; Step 210: Combine two CBS modules and one convolutional layer to form the angle detection head module D3; Step 211: Combine the positioning, classification, and angle detection head modules D1, D2, and D3 into the overall detection head module D4.
4. The method for detecting electronic components and angles based on deep learning according to claim 3, characterized in that, Step 2 also includes the following steps: Step 22: Train the network model using the training dataset; Step 22 also includes: Step 212: Input the images from the training dataset into two concatenated CBS modules, one C3k2 module, one CBS module, and one C3k2 module to obtain feature map n1; Step 213: Input feature map n1 into a cascaded CBS module and a C3k2 module to obtain feature map n2; Step 214: Input feature map n2 into a concatenated SPPF and C2PSA module to obtain feature map n3; Step 215: Upsample feature map n3 to the same size as feature map n2, and concatenate it with feature map n2 in the channel direction. Then input it into one C3k2 module to obtain feature map P1. Step 216: Upsample feature map P1 to the same size as feature map n1, and concatenate it with feature map n1 in the channel direction. Then input it into one C3k2 module to obtain feature map P2. Step 217: After inputting feature map P2 into one CBS module, it is concatenated with feature map P1 in the channel direction to obtain feature map P3; Step 218: After inputting feature map P3 into one C3k2 module and one CBS module, it is concatenated with feature map n3 in the channel direction to obtain feature map P4; Step 219: Input feature map P4 into a C3k2 module to obtain feature map P5; Step 220: Input feature map P2 as the first feature map into the detection head module D4 to obtain feature vector t1; input the result of feature map P3 through module C3k2 as the second feature map into the detection head module D4 to obtain feature vector t2; input feature map P5 as the third feature map into the detection head module D4 to obtain feature vector t3. The first feature map is downsampled by 8×, the second feature map by 16×, and the third feature map by 32×; the feature tensor t1 corresponds to the first feature map, the feature tensor t2 corresponds to the second feature map, and the feature tensor t3 corresponds to the third feature map. Step 221: The feature tensors t1, t2, and t3 obtained from the three scales are fused to obtain the position information (x, y, w, h), angle information, and class probability of all boxes at the three scales. Positive samples are obtained by filtering all predicted boxes based on the position and angle information. The three-class losses of the positive sample predicted boxes are weighted and summed according to preset weights to obtain the final loss result of the overall detection head module D4. Backpropagation is performed based on this loss value to complete the parameter iteration of the D4 module and the upstream network layer. Step 222: Train the neural network model using the Adam method. Repeat steps 212 to 221 until the model performs well on the validation set and there is no overfitting.
5. The method for detecting electronic component targets and angles based on deep learning according to claim 3, characterized in that, In step 206, an attention module is added, and the calculation formula is as follows: ； The normalized exponential function normalizes the correlation scores into a probability distribution. This represents the query vector, indicating the current focus of information. This represents a value vector, containing the actual information that needs to be output. This represents a set of key vectors used to calculate relevance with the query vector. This represents the dimension of the key vector, used for scaling dot product operations.
6. The method for detecting electronic components and angles based on deep learning according to claim 3, characterized in that, In step 208, in order to improve the angle sensitivity of the detection frame and simultaneously address the impact of angle periodicity and abrupt changes on the results, this invention proposes an angle difference coefficient. The crossover ratio loss is corrected and calculated as follows: ； ； in, This represents the processed angle difference, ranging from [- , ), and These represent the actual rotation angle and the predicted rotation angle, respectively; mod is the modulo operation; an angle difference coefficient is used. The complete intersection-union loss is corrected and added to the distribution focus loss (DFL) to form the detection box loss (BboxLoss), which measures the difference between the D1 output and the true sample value. The calculation process for the cross-union ratio loss DirIoULoss based on angle difference correction is as follows: ； ； ； ； This is the ratio of the intersection area of the predicted bounding box and the ground truth bounding box to the union area of the predicted bounding box and the ground truth bounding box. The distance between the center points of the predicted bounding box and the ground truth bounding box. The minimum diagonal distance between the bounding rectangles of the predicted bounding box and the ground truth bounding box. This represents the aspect ratio consistency coefficient. Indicates the weighting coefficient. Indicates the loss of a complete intersection; , and , These are the width and height of the target bounding box and the predicted bounding box, respectively; The angle difference coefficient; The calculation process for the distributed focus loss (DFL) is as follows: ； N is the number of samples, and C is the number of classes. It is the true label of the i-th sample. It is the predicted probability that the i-th sample belongs to category c; It is a balancing factor used to adjust the weights between positive and negative samples; It is a focusing parameter used to control the degree of attention given to difficult samples.
7. The method for detecting electronic components and angles based on deep learning according to claim 3, characterized in that, In step 209, the feature output of D2 is compared with the true sample class, and the binary cross-entropy loss (BCE) is used as the class loss to measure the difference between the D2 output and the true sample class. The calculation process of BCE is as follows: ； Represents the binary cross-entropy loss. This represents the true label, with a value of 0 or 1, where 0 represents the negative class and 1 represents the positive class. This represents the probability value that the model predicts a sample to belong to the positive class, with a range of 0 ≤ a ≤ 1.
8. The method for detecting electronic components and angles based on deep learning according to claim 3, characterized in that, In step 210, in order to increase the contribution of angle to the result and thus improve the accuracy of model angle prediction, the feature output of D3 is compared with the true angle of the sample, and angle loss AngleLoss is added. The calculation process for AngleLoss is as follows: ； ； Indicates angle loss, This represents the processed angle difference, ranging from [- , ).
9. The method for detecting electronic components and angles based on deep learning according to claim 3, characterized in that, In step 211, the overall detection head module D4 retains independent prediction result output channels for the three branches: the positioning detection head module D1, the classification detection head module D2, and the angle detection head module D3. The positioning detection head module D1 outputs the target position parameter prediction result, the classification detection head module D2 outputs the target category probability prediction result, and the angle detection head module D3 outputs the target rotation angle prediction result. These three prediction results serve as the core basis for the subsequent dynamic label allocation of the three-scale prediction boxes P3, P4, and P5, and are used to select the optimal prediction box that matches the real label. Meanwhile, the total loss of the total detection head module D4 is defined as the result of the weighted sum of the localization loss corresponding to the localization detection head module D1 branch, the classification loss corresponding to the classification detection head module D2 branch, and the angle loss corresponding to the angle detection head module D3 branch, according to preset weights. This total loss will be used for network parameter correction in the subsequent backpropagation process.
10. The method for detecting electronic component targets and angles based on deep learning according to claim 4, characterized in that, In step 221, all predicted boxes are filtered based on position and angle information. Specifically, the cross-union ratio (CUI) with the label result box is calculated, and the predicted boxes with a CUI greater than a set threshold and the predicted boxes with the highest CUI with the label result box are used as positive samples to calculate the loss.