Electronic component classification and positioning method based on convolutional neural network

Abstract

The invention discloses an electronic component classification and positioning method based on a convolutional neural network, belongs to the technical field of optical detection, and particularly relates to the electronic component classification and positioning method based on the convolutional neural network. The invention aims to solve the problem of low element standard registration and modeling efficiency in the existing AOI use process. The method comprises the following steps: S1, acquiring a data set sample picture; s2, performing data enhancement on the data set sample picture to obtain an enhanced data set; s3, constructing a convolutional neural network model; the specific process is that a convolutional neural network model comprises a backbone network, a multi-scale feature fusion network and a detector; s4, based on the enhanced data set obtained in the S2, obtaining a trained convolutional neural network model; and S5, inputting a to-be-tested PCB electronic component image into the trained convolutional neural network model, and outputting the category and position of the to-be-tested PCB electronic component image by the trained convolutional neural network model.

Description

Technical Field

[0001] This invention belongs to the field of optical detection technology, and in particular relates to a method for classifying and locating electronic components based on convolutional neural networks. Background Technology

[0002] PCBA (Printed Circuit Board Assembly) refers to the process of inserting and soldering various components onto a PCB (Printed Circuit Board). With the development of electronic products and the advancement of intelligent manufacturing technology, printed circuit boards are evolving towards higher precision and higher density. Due to manufacturing processes and human factors, various defects are unavoidable in printed circuit boards; therefore, defect detection of PCBAs is an essential step.

[0003] Traditional manual inspection is not only inefficient but also expensive, failing to meet the demands of modern industrial production. In recent years, with the development of computer technology, machine vision-based PCBA defect detection methods have been gradually applied. However, due to the minute and diverse nature of PCBA defects, distinguishing defects in images is difficult, resulting in poor robustness and unsatisfactory detection rates in general PCBA defect detection systems. Therefore, many companies need to customize and develop inspection equipment specifically for the characteristics and inspection needs of PCBAs. Among these technologies, Automated Optical Inspection (AOI) is gaining increasing importance as it extracts effective component data from captured images, processes and analyzes the data to complete the inspection of relevant targets. Summary of the Invention

[0004] The purpose of this invention is to solve the problem of low efficiency in component standard registration and modeling during the use of existing AOI, and to propose a method for electronic component classification and localization based on convolutional neural networks.

[0005] The specific process of a method for classifying and locating electronic components based on convolutional neural networks is as follows:

[0006] S1. Obtain sample images for the dataset;

[0007] S2. Perform data augmentation on the sample images in the dataset to obtain the augmented dataset;

[0008] S3. Construct a convolutional neural network model; the specific process is as follows:

[0009] Convolutional neural network models include a backbone network, a multi-scale feature fusion network, and a detector;

[0010] The backbone network comprises, in sequence: a first convolutional layer, a first batch normalization (BN) layer, a first SiLU activation function, a second convolutional layer, a second batch normalization (BN) layer, a second SiLU activation function, a first GELAN module, a first CoTN module, a second CoTN module, a third CoTN module, and a spatial pyramid pooling block;

[0011] The multi-scale feature fusion network includes: a first MSA attention mechanism, a second MSA attention mechanism, a third MSA attention mechanism, a first CARAFE module, a second CARAFE module, a first C2fHB lightweight module, a second C2fHB lightweight module, a third C2fHB lightweight module, a fourth C2fHB lightweight module, a sixth convolutional layer, and a seventh convolutional layer.

[0012] The processing procedure of the multi-scale feature fusion network is as follows:

[0013] The first CoTN module outputs feature B, which is input into the first MSA attention mechanism module. The first MSA attention mechanism module outputs feature F.

[0014] The second CoTN module outputs feature C, which is input into the second MSA attention mechanism module. The second MSA attention mechanism module outputs feature G.

[0015] The spatial pyramid pooling block outputs feature E, which is input into the first CARAFE module. The first CARAFE module outputs feature H.

[0016] The output feature G of the second MSA attention mechanism and the output feature H of the first CARAFE module are concatenated to obtain feature I;

[0017] Feature I is input into the first C2fHB lightweight module, and the first C2fHB lightweight module outputs feature J;

[0018] Feature J is input to the second CARAFE module, and the second CARAFE module outputs feature k.

[0019] The output feature K of the second CARAFE module and the output feature F of the first MSA attention mechanism module are concatenated to obtain feature L;

[0020] Feature L is input to the second C2fHB lightweight module, and the second C2fHB lightweight module outputs feature M;

[0021] Feature M is input into the sixth convolutional layer, and the sixth convolutional layer outputs feature N;

[0022] The output feature J of the first C2fHB lightweight module and the output feature N of the sixth convolutional layer are concatenated to obtain feature O.

[0023] Feature O is input to the third C2fHB lightweight module, and the third C2fHB lightweight module outputs feature P;

[0024] Feature P is input into the seventh convolutional layer, and the seventh convolutional layer outputs feature q;

[0025] The spatial pyramid pooling block outputs feature E, which is input to the third MSA attention mechanism module. The third MSA attention mechanism module outputs feature R.

[0026] Feature q and the output feature R of the third MSA attention mechanism are concatenated to obtain feature S;

[0027] Feature S is input to the fourth C2fHB lightweight module, and the fourth C2fHB lightweight module outputs feature t;

[0028] The detector includes detection head 1, detection head 2, and detection head 3;

[0029] The second C2fHB lightweight module outputs feature M as the target localization and classification result obtained by the detection head 1;

[0030] The third C2fHB lightweight module outputs feature P as the target localization and classification result obtained by the detection head 2;

[0031] The fourth C2fHB lightweight module outputs feature t as the target localization and classification result obtained by the detection head 3;

[0032] S4. Based on the enhanced dataset obtained in S2, obtain a trained convolutional neural network model;

[0033] S5. Input the image of the electronic component on the PCB board to be tested into the trained convolutional neural network model. The trained convolutional neural network model outputs the category and location of the electronic component image on the PCB board to be tested.

[0034] The beneficial effects of this invention are as follows:

[0035] This invention is primarily applied to AOI (Automated Optical Inspection) instruments used in SMT assembly production of PCBAs. AOI is an inspection device that detects and provides feedback on component soldering defects encountered in SMT assembly production, based on visual image acquisition and analysis, software visualization processing, and automated control. While AOI is currently the mainstream method for PCBA defect detection, most AOI devices lack intelligence, requiring complex and tedious modeling and programming for each component before use, resulting in low efficiency and a high learning curve. Therefore, realizing automatic classification and positioning of circuit board components based on convolutional neural networks is of great significance for the development of AOI instruments.

[0036] This invention enables rapid identification of component categories and accurate acquisition of component bounding boxes directly on the circuit board image during AOI registration and programming, eliminating the tedious process of drawing and registering individual component bounding boxes. This significantly improves component registration and programming efficiency; the time required for standard component programming using this method can be reduced to one-sixth of the original programming time. Subsequent analysis of the components using the component detection window identifies defective products.

[0037] This invention proposes an advanced and efficient real-time PCBA classification and localization method to address the inefficiency of the registration standard process in AOI equipment's printed circuit board component inspection. Specifically designed for PCBA inspection, this invention employs a real-time component classification and localization algorithm based on a convolutional neural network, offering advantages such as high detection accuracy and excellent real-time performance. First, this invention designs a high-efficiency backbone network based on a generalized efficient aggregation network and a context converter. This backbone network enhances the network's ability to extract contextual information, enabling more effective extraction of PCBA feature information and providing a solid foundation for subsequent component identification and localization. Second, this invention designs a C2fHB module based on the HorNet structure in the feature fusion network, enhancing the model's feature extraction capability without increasing computational load. Simultaneously, a CARAFE module replaces the ordinary upsampling operation, connecting surrounding information through a larger receptive field, reducing feature information loss, and making it more suitable for target-dense inspection tasks like PCBAs. This invention ensures the reliability of electronic component identification during AOI inspection, achieving rapid and accurate classification and localization. Comparative experiments show that it outperforms most current detection networks in terms of both detection accuracy and efficiency, and can greatly improve the efficiency of the component standard programming process for AOI equipment.

[0038] The results of comparing this method with existing methods are shown in Table 1.

[0039] Attached Figure Description

[0040] Figure 1 This is a flowchart of the present invention;

[0041] Figure 2 This is a structural diagram of the convolutional neural network model of the present invention. Detailed Implementation

[0042] Specific Implementation Method 1: The specific process of this implementation method for classifying and locating electronic components based on convolutional neural networks is as follows:

[0043] S1. Obtain sample images for the dataset;

[0044] S2. Perform data augmentation on the sample images in the dataset to obtain the augmented dataset;

[0045] S3. Construct a convolutional neural network model; the specific process is as follows:

[0046] The designed network model is a one-stage network model, which has the advantage of high detection efficiency. It consists of three parts: a backbone network, a multi-scale fusion network, and a detector. The network structure is as follows: Figure 1 As shown.

[0047] Convolutional neural network models include a backbone network, a multi-scale feature fusion network, and a detector;

[0048] The backbone network comprises, in sequence: a first convolutional layer, a first batch normalization (BN) layer, a first SiLU activation function, a second convolutional layer, a second batch normalization (BN) layer, a second SiLU activation function, a first GELAN module, a first CoTN module, a second CoTN module, a third CoTN module, and a spatial pyramid pooling (SPPF) block;

[0049] The multi-scale feature fusion network includes: a first MSA attention mechanism, a second MSA attention mechanism, a third MSA attention mechanism, a first CARAFE module, a second CARAFE module, a first C2fHB lightweight module, a second C2fHB lightweight module, a third C2fHB lightweight module, a fourth C2fHB lightweight module, a sixth convolutional layer, and a seventh convolutional layer.

[0050] The processing procedure of the multi-scale feature fusion network is as follows:

[0051] The first CoTN module outputs feature B, which is input into the first MSA attention mechanism module. The first MSA attention mechanism module outputs feature F.

[0052] The second CoTN module outputs feature C, which is input into the second MSA attention mechanism module. The second MSA attention mechanism module outputs feature G.

[0053] The spatial pyramid pooling (SPPF) block outputs feature E as input to the first CARAFE module, and the first CARAFE module outputs feature H.

[0054] The output feature G of the second MSA attention mechanism and the output feature H of the first CARAFE module are concatenated to obtain feature I;

[0055] Feature I is input into the first C2fHB lightweight module, and the first C2fHB lightweight module outputs feature J;

[0056] Feature J is input to the second CARAFE module, and the second CARAFE module outputs feature k.

[0057] The output feature K of the second CARAFE module and the output feature F of the first MSA attention mechanism module are concatenated to obtain feature L;

[0058] Feature L is input to the second C2fHB lightweight module, and the second C2fHB lightweight module outputs feature M;

[0059] Feature M is input into the sixth convolutional layer, and the sixth convolutional layer outputs feature N;

[0060] The output feature J of the first C2fHB lightweight module and the output feature N of the sixth convolutional layer are concatenated to obtain feature O.

[0061] Feature O is input to the third C2fHB lightweight module, and the third C2fHB lightweight module outputs feature P;

[0062] Feature P is input into the seventh convolutional layer, and the seventh convolutional layer outputs feature q;

[0063] The output feature E of the Spatial Pyramid Pooling (SPPF) block is input to the third MSA attention mechanism module, and the output feature R of the third MSA attention mechanism module is output.

[0064] Feature q and the output feature R of the third MSA attention mechanism are concatenated to obtain feature S;

[0065] Feature S is input to the fourth C2fHB lightweight module, and the fourth C2fHB lightweight module outputs feature t;

[0066] The detector includes detection head 1, detection head 2, and detection head 3;

[0067] The second C2fHB lightweight module outputs feature M as the target localization and classification result obtained by the detection head 1;

[0068] The third C2fHB lightweight module outputs feature P as the target localization and classification result obtained by the detection head 2;

[0069] The fourth C2fHB lightweight module outputs feature t as the target localization and classification result obtained by the detection head 3;

[0070] S4. Based on the enhanced dataset obtained in S2, obtain a trained convolutional neural network model;

[0071] S5. Input the image of the electronic component on the PCB board to be tested into the trained convolutional neural network model. The trained convolutional neural network model outputs the category and location of the electronic component image on the PCB board to be tested.

[0072] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the sample images of the dataset are obtained in step S1; the specific process is as follows:

[0073] An AOI machine is used to scan and photograph the PCB board to collect FOV images containing electronic components. Labelimg annotation software is used to classify and label the electronic components in the FOV images according to capacitors, resistors, diodes, transistors, color-coded resistors, SOP integrated circuit package components, QFP square flat package components, and BGA ball grid array components, generating label files.

[0074] The other steps and parameters are the same as in Specific Implementation Method 1.

[0075] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that, in step S2, data augmentation is performed on the sample images of the dataset to obtain the augmented dataset; the specific process is as follows:

[0076] The obtained dataset sample images are enhanced by flipping, translating, randomly cropping, scaling, and contrast transformation to obtain the enhanced dataset.

[0077] Other steps and parameters are the same as in specific implementation method one or two.

[0078] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the backbone network operation process is as follows:

[0079] The sample images in the dataset are sequentially input into the first convolutional layer, the first batch normalization layer, the first SiLU activation function, the second convolutional layer, the second batch normalization layer, the second SiLU activation function, and the first GELAN module. The first GELAN module outputs feature A.

[0080] Feature A is input into the first CoTN module, and the first CoTN module outputs feature B;

[0081] Feature B is input into the second CoTN module, and the second CoTN module outputs feature C;

[0082] Feature C is input into the third CoTN module, and the third CoTN module outputs feature D;

[0083] Feature D is input to a Spatial Pyramid Pooling (SPPF) block, and the Spatial Pyramid Pooling (SPPF) block outputs feature E;

[0084] The kernel size of the first convolutional layer is 3×3;

[0085] The kernel size of the second convolutional layer is 3×3;

[0086] The first CoTN module sequentially includes a third convolutional layer, a first CoT module, and a second GELAN module;

[0087] The second CoTN module includes, in sequence, a fourth convolutional layer, a second CoT module, and a third GELAN module;

[0088] The third CoTN module includes, in sequence, a fifth convolutional layer, a third CoT module, and a fourth GELAN module.

[0089] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0090] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that feature A is input to the first CoTN module, and the first CoTN module outputs feature B; the specific process is as follows:

[0091] Feature A is input into the third convolutional layer, the output feature of the third convolutional layer is input into the first CoT module, the output feature of the first CoT module is input into the second GELAN module, and the output feature of the second GELAN module is B.

[0092] Feature B is input to the second CoTN module, and the second CoTN module outputs feature C; the specific process is as follows:

[0093] Feature B is input into the fourth convolutional layer, the output feature of the fourth convolutional layer is input into the second CoT module, the output feature of the second CoT module is input into the third GELAN module, and the output feature of the third GELAN module is C.

[0094] The feature C is input into the third CoTN module, and the third CoTN module outputs feature D; the specific process is as follows:

[0095] Feature C is input into the fifth convolutional layer, the output feature of the fifth convolutional layer is input into the third CoT module, the output feature of the third CoT module is input into the fourth GELAN module, and the output feature of the fourth GELAN module is D.

[0096] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0097] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the first C2fHB lightweight module includes: an eighth convolutional layer, a first Split module, a first Hornet module, and a ninth convolutional layer;

[0098] The second C2fHB lightweight module includes: a tenth convolutional layer, a second Split module, a second Hornet module, and an eleventh convolutional layer;

[0099] The third C2fHB lightweight module includes: a twelfth convolutional layer, a third Split module, a third Hornet module, and a thirteenth convolutional layer;

[0100] The fourth C2fHB lightweight module includes: a fourteenth convolutional layer, a fourth Split module, a fourth Hornet module, and a fifteenth convolutional layer.

[0101] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0102] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One to Six in that feature I is input to the first C2fHB lightweight module, and the first C2fHB lightweight module outputs feature J; the specific process is as follows:

[0103] Feature I is input into the eighth convolutional layer, the output feature of the eighth convolutional layer is input into the first split module, the output feature of the first split module is input into the first Hornet module; the output features of the first Hornet module and the output features of the first split module are concatenated, and the concatenated features are input into the ninth convolutional layer, and the output feature J of the ninth convolutional layer is output.

[0104] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0105] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that the feature L is input to the second C2fHB lightweight module, and the second C2fHB lightweight module outputs feature M; the specific process is as follows:

[0106] Feature L is input to the tenth convolutional layer, the output feature of the tenth convolutional layer is input to the second split module, the output feature of the second split module is input to the second Hornet module; the output features of the second Hornet module and the output features of the second split module are concatenated, and the concatenated features are input to the eleventh convolutional layer, and the output feature M of the eleventh convolutional layer is output.

[0107] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0108] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that feature O is input to the third C2fHB lightweight module, and the third C2fHB lightweight module outputs feature P; the specific process is as follows:

[0109] Feature O is input to the twelfth convolutional layer, the output feature of the twelfth convolutional layer is input to the third Split module, the output feature of the third Split module is input to the third Hornet module; the output features of the third Hornet module and the output features of the third Split module are concatenated, and the concatenated features are input to the thirteenth convolutional layer, and the output feature P of the thirteenth convolutional layer is output.

[0110] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0111] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that the feature S is input to the fourth C2fHB lightweight module, and the fourth C2fHB lightweight module outputs feature t; the specific process is as follows:

[0112] Feature S is input to the fourteenth convolutional layer, the output feature of the fourteenth convolutional layer is input to the fourth Split module, the output feature of the fourth Split module is input to the fourth Hornet module; the output features of the fourth Hornet module and the output features of the fourth Split module are concatenated, and the concatenated features are input to the fifteenth convolutional layer, and the fifteenth convolutional layer outputs feature t.

[0113] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0114] The backbone network consists of six feature extraction modules. First, it uses two 3×3 conv_bn_silu blocks for downsampling. Each conv_bn_silu block mainly consists of a standard convolutional layer, a batch normalization (BN) layer, and a SiLU activation function. Then, the more efficient and flexible GELAN module is used for feature extraction. The GELAN module consists of conv_bn_silu blocks and stacked RepNCSP modules. Simultaneously, three CoTN modules combining Context Transformer (CoT) and GELAN are applied to the backbone network to better integrate contextual and local information. Finally, Spatial Pyramid Pooling (SPPF) blocks are used for feature extraction. We propose a Context Transformer (CoTN) module based on CoT and GELAN blocks. This module primarily uses self-attention to acquire contextual information. In traditional self-attention mechanisms, all pairs of query key relationships are learned individually through independent query key pairs, failing to fully utilize the relationships between adjacent key points. CoT obtains static context by encoding keywords individually, fully utilizing the contextual information between adjacent keys. CoT employs... Convolution encodes the key and extracts local context. QUERY and Perform concatenation and execute two Convolution calculation. Multiply the obtained attention matrix by... The convolutional values ​​are used to obtain the dynamic context. Finally, and The features are fused to generate a feature map. By employing the designed CoTN module, the extraction of electronic component features can be improved more effectively, thereby enhancing the overall detection accuracy of the network. It is obtained from the following formula:

[0115]

[0116] in Indicates use Local context extracted by convolution. Queries that do not perform convolution calculations Indicates passage The value obtained from convolution. Here, and Indicates two Convolution calculation, This indicates the global context information obtained. Finally, and The data is then fused to generate a feature map.

[0117] A multi-scale feature fusion network employs bottom-up fusion connections in a bidirectional feature pyramid to preserve bottom-level information, fully integrating shallow fine-grained features and deep semantic features. An MSA attention mechanism is embedded between the backbone network and the feature fusion network, effectively improving the network's focus on the target and enhancing the representation of regions of interest. A CARAFE module is added to the feature pyramid network and path aggregation network. The CARAFE structure consists of two modules: upsampling kernel prediction and feature reconstruction. CARAFE designs a matching upsampling kernel based on the input features and then reconstructs features based on the predicted upsampling kernel to obtain the upsampling result map. The CARAFE operator connects surrounding information through a larger receptive field, reducing information loss and making it more suitable for dense target detection tasks such as PCBAs. Simultaneously, a lightweight CfHB module is designed using the Hornet structure to enhance the model's feature extraction capabilities. Hornet's main structure is a lightweight convolutional operation, depthwise separable convolution (DWConv), which achieves the same effect as traditional convolution with only one-third the computational power. When the input features... When entering the Bumblebee, the dimensionality is first increased through convolution, and then it is divided into two sets of feature maps according to a predetermined ratio. and After passing through the DWConv structure, Output multiple sets of sub-feature maps .after, and Perform a dot product operation and add dimensions through convolution to obtain... Repeat this iterative process until the last sub-feature map is reached. Through the aforementioned multi-level information interaction, Hornet connects feature information by generating self-weighted parameters, thereby enabling the network model to obtain higher-order deep features with stronger semantic information, effectively improving the network's resolution. The Hornet processing procedure is as follows:

[0118]

[0119]

[0120]

[0121]

[0122] in, Representing input features, functions Represents convolution operation, function This represents a depth-separable convolution. This indicates the spatial order set during the Hornet procedure.

[0123] The detector network uses three-scale YOLO detector heads to perform multi-scale target localization and classification on the features after feature fusion network.

[0124] The model training is constrained by a loss function, and CIOU rectangular box loss is used for model regression training. To ensure the AOI defect detection model converges to a state where it can accurately identify the presence of defects in a given image, the error between the network's predicted value and the actual reference value is calculated using the loss function and the corresponding reference sample in the data. This error value is then used through backpropagation of the convolutional network to guide the AOI defect detection model in adjusting the parameters of each layer. The specific loss function is as follows:

[0125]

[0126] in, Represents the classification loss function. Represents the confidence loss function. This represents the regression loss function for the predicted bounding box.

[0127]

[0128] Among them, parameters This represents the total number of cells existing at the current network scale, while the parameter... It means that in the first In the nth cell, the nth Does each predicted bounding box contain a ground truth bounding box? Parameter and parameters These represent the actual classification probability value and the predicted classification probability value, respectively.

[0129]

[0130] Among them, parameters Represents the number of candidate boxes in each cell at the current network scale, parameter This represents the confidence level of the prediction, corresponding to This represents the actual confidence level;

[0131]

[0132]

[0133] in, This represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. These represent the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. The parameter represents the diagonal length of the smallest closure region formed. Used to measure aspect ratio consistency. Parameter middle , , , The tables represent the width and height of the ground truth bounding box and the predicted bounding box.

[0134] The improved algorithm model was trained using the dataset. The dataset sample images were scaled to 416×416×3 and input into the aforementioned convolutional neural network model for iterative training. During iterative training, 100 samples were extracted from the training samples for each iteration and batch normalized. The number of iteration cycles was set, and in each iteration, a validation sample set was input into the model to improve recognition accuracy. If the final accuracy did not meet the requirements, the number of iteration cycles was increased, and the parameters were fine-tuned and retrained. The purpose of iterative training was to achieve the highest classification accuracy on the validation set, meaning the model could automatically recognize the labeled category features in the dataset to the highest degree. At this point, the model's weight parameters were optimal, and the model with these optimal weight parameters was the deep learning detection model.

[0135] The system acquires color images of the electronic components on the actual PCB board to be inspected; based on the deep learning detection model described above, it enables online automatic detection of defective electronic components; and it scans the electronic component products using a CCD / CMOS camera to acquire color images of the electronic components.

[0136] The color appearance images obtained in the above steps are input into a deep learning detection model to identify and locate the component image categories; the category and location information of the corresponding components on the PCB board are obtained to prepare for subsequent registration.

[0137] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A method for classifying and locating electronic components based on convolutional neural networks, characterized in that: The specific process of the method is as follows: S1. Obtain sample images for the dataset; S2. Perform data augmentation on the sample images in the dataset to obtain the augmented dataset; S3. Construct a convolutional neural network model; The specific process is as follows: Convolutional neural network models include a backbone network, a multi-scale feature fusion network, and a detector; The backbone network comprises, in sequence: a first convolutional layer, a first batch normalization (BN) layer, a first SiLU activation function, a second convolutional layer, a second batch normalization (BN) layer, a second SiLU activation function, a first GELAN module, a first CoTN module, a second CoTN module, a third CoTN module, and a spatial pyramid pooling block; The multi-scale feature fusion network includes: a first MSA attention mechanism, a second MSA attention mechanism, a third MSA attention mechanism, a first CARAFE module, a second CARAFE module, a first C2fHB lightweight module, a second C2fHB lightweight module, a third C2fHB lightweight module, a fourth C2fHB lightweight module, a sixth convolutional layer, and a seventh convolutional layer. The processing procedure of the multi-scale feature fusion network is as follows: The first CoTN module outputs feature B, which is input into the first MSA attention mechanism module. The first MSA attention mechanism module outputs feature F. The second CoTN module outputs feature C, which is input into the second MSA attention mechanism module. The second MSA attention mechanism module outputs feature G. The spatial pyramid pooling block outputs feature E, which is input into the first CARAFE module. The first CARAFE module outputs feature H. The output feature G of the second MSA attention mechanism and the output feature H of the first CARAFE module are concatenated to obtain feature I; Feature I is input into the first C2fHB lightweight module, and the first C2fHB lightweight module outputs feature J; Feature J is input to the second CARAFE module, and the second CARAFE module outputs feature k. The output feature K of the second CARAFE module and the output feature F of the first MSA attention mechanism module are concatenated to obtain feature L; Feature L is input to the second C2fHB lightweight module, and the second C2fHB lightweight module outputs feature M; Feature M is input into the sixth convolutional layer, and the sixth convolutional layer outputs feature N; The output feature J of the first C2fHB lightweight module and the output feature N of the sixth convolutional layer are concatenated to obtain feature O. Feature O is input to the third C2fHB lightweight module, and the third C2fHB lightweight module outputs feature P; Feature P is input into the seventh convolutional layer, and the seventh convolutional layer outputs feature q; The spatial pyramid pooling block outputs feature E, which is input to the third MSA attention mechanism module. The third MSA attention mechanism module outputs feature R. Feature q and the output feature R of the third MSA attention mechanism are concatenated to obtain feature S; Feature S is input to the fourth C2fHB lightweight module, and the fourth C2fHB lightweight module outputs feature t; The detector includes detection head 1, detection head 2, and detection head 3; The second C2fHB lightweight module outputs feature M as the target localization and classification result obtained by the detection head 1; The third C2fHB lightweight module outputs feature P as the target localization and classification result obtained by the detection head 2; The fourth C2fHB lightweight module outputs feature t as the target localization and classification result obtained by the detection head 3; S4. Based on the enhanced dataset obtained in S2, obtain a trained convolutional neural network model; S5. Input the image of the electronic component on the PCB board to be tested into the trained convolutional neural network model. The trained convolutional neural network model outputs the category and location of the electronic component image on the PCB board to be tested.

2. The electronic component classification and localization method based on convolutional neural networks according to claim 1, characterized in that: The process of obtaining sample images from the dataset in step S1 is as follows: An AOI machine is used to scan and photograph the PCB board to collect images containing electronic components. Labelimg annotation software is used to classify and label the electronic components in the images according to capacitors, resistors, diodes, transistors, color-coded resistors, SOP, QFP, and BGA, and generate label files.

3. The electronic component classification and localization method based on convolutional neural networks according to claim 2, characterized in that: In step S2, data augmentation is performed on the sample images in the dataset to obtain the augmented dataset; the specific process is as follows: The obtained dataset sample images are enhanced by flipping, translating, randomly cropping, scaling, and contrast transformation to obtain the enhanced dataset.

4. The electronic component classification and localization method based on convolutional neural networks according to claim 3, characterized in that: The backbone network operates as follows: The sample images in the dataset are sequentially input into the first convolutional layer, the first batch normalization layer, the first SiLU activation function, the second convolutional layer, the second batch normalization layer, the second SiLU activation function, and the first GELAN module. The first GELAN module outputs feature A. Feature A is input into the first CoTN module, and the first CoTN module outputs feature B; Feature B is input into the second CoTN module, and the second CoTN module outputs feature C; Feature C is input into the third CoTN module, and the third CoTN module outputs feature D; Feature D is input to the spatial pyramid pooling block, and the spatial pyramid pooling block outputs feature E; The kernel size of the first convolutional layer is 3×3; The kernel size of the second convolutional layer is 3×3; The first CoTN module sequentially includes a third convolutional layer, a first CoT module, and a second GELAN module; The second CoTN module includes, in sequence, a fourth convolutional layer, a second CoT module, and a third GELAN module; The third CoTN module includes, in sequence, a fifth convolutional layer, a third CoT module, and a fourth GELAN module.

5. The electronic component classification and localization method based on convolutional neural networks according to claim 4, characterized in that: Feature A is input into the first CoTN module, and the first CoTN module outputs feature B; the specific process is as follows: Feature A is input into the third convolutional layer, the output feature of the third convolutional layer is input into the first CoT module, the output feature of the first CoT module is input into the second GELAN module, and the output feature of the second GELAN module is B. Feature B is input to the second CoTN module, and the second CoTN module outputs feature C. The specific process is as follows: Feature B is input into the fourth convolutional layer, the output feature of the fourth convolutional layer is input into the second CoT module, the output feature of the second CoT module is input into the third GELAN module, and the output feature of the third GELAN module is C. The feature C is input into the third CoTN module, and the third CoTN module outputs feature D. The specific process is as follows: Feature C is input into the fifth convolutional layer, the output feature of the fifth convolutional layer is input into the third CoT module, the output feature of the third CoT module is input into the fourth GELAN module, and the output feature of the fourth GELAN module is D.

6. The electronic component classification and localization method based on convolutional neural networks according to claim 5, characterized in that: The first C2fHB lightweight module includes: an eighth convolutional layer, a first Split module, a first Hornet module, and a ninth convolutional layer; The second C2fHB lightweight module includes: a tenth convolutional layer, a second Split module, a second Hornet module, and an eleventh convolutional layer; The third C2fHB lightweight module includes: a twelfth convolutional layer, a third Split module, a third Hornet module, and a thirteenth convolutional layer; The fourth C2fHB lightweight module includes: a fourteenth convolutional layer, a fourth Split module, a fourth Hornet module, and a fifteenth convolutional layer.

7. The electronic component classification and localization method based on convolutional neural networks according to claim 6, characterized in that: The feature I is input into the first C2fHB lightweight module, and the first C2fHB lightweight module outputs feature J; The specific process is as follows: Feature I is input into the eighth convolutional layer, the output feature of the eighth convolutional layer is input into the first split module, the output feature of the first split module is input into the first Hornet module; the output features of the first Hornet module and the output features of the first split module are concatenated, and the concatenated features are input into the ninth convolutional layer, and the output feature J of the ninth convolutional layer is output.

8. The electronic component classification and localization method based on convolutional neural networks according to claim 7, characterized in that: The feature L is input to the second C2fHB lightweight module, and the second C2fHB lightweight module outputs the feature M; The specific process is as follows: Feature L is input to the tenth convolutional layer, the output feature of the tenth convolutional layer is input to the second split module, the output feature of the second split module is input to the second Hornet module; the output features of the second Hornet module and the output features of the second split module are concatenated, and the concatenated features are input to the eleventh convolutional layer, and the output feature M of the eleventh convolutional layer is output.

9. The electronic component classification and localization method based on convolutional neural networks according to claim 8, characterized in that: The feature O is input to the third C2fHB lightweight module, and the third C2fHB lightweight module outputs feature P; The specific process is as follows: Feature O is input to the twelfth convolutional layer, the output feature of the twelfth convolutional layer is input to the third Split module, the output feature of the third Split module is input to the third Hornet module; the output features of the third Hornet module and the output features of the third Split module are concatenated, and the concatenated features are input to the thirteenth convolutional layer, and the output feature P of the thirteenth convolutional layer is output.

10. The electronic component classification and localization method based on convolutional neural networks according to claim 9, characterized in that: The feature S is input to the fourth C2fHB lightweight module, and the fourth C2fHB lightweight module outputs feature t; the specific process is as follows: Feature S is input to the fourteenth convolutional layer, the output feature of the fourteenth convolutional layer is input to the fourth Split module, the output feature of the fourth Split module is input to the fourth Hornet module; the output features of the fourth Hornet module and the output features of the fourth Split module are concatenated, and the concatenated features are input to the fifteenth convolutional layer, and the fifteenth convolutional layer outputs feature t.

Images