Training method, device, electronic device and medium for PCB defect detection model

Abstract

The present application provides a training method, device, electronic device and medium for a PCB defect detection model. The method includes: obtaining a model training sample, the model training sample includes PCB images of different physical defect types, and each PCB image is annotated with the labeled defect information of the PCB; inputting the model training sample into the PCB defect detection model to be trained; calling the adaptive feature fusion module to process the PCB image with average pooling and maximum pooling strategies to obtain a spliced ​​feature map; calling the feature aggregation module to perform squeezing, excitation and feature partial convolution operations on the spliced ​​feature map to obtain the predicted defect information corresponding to the PCB image; based on the labeled defect information and the predicted defect information, calculating the loss value of the PCB defect detection model to be trained; when the loss value is within a preset range, obtaining the final PCB defect detection model for detecting physical defects of the PCB.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, device, electronic equipment and medium for a PCB defect detection model. Background Art

[0002] In modern electronic devices, printed circuit boards (PCBs) play a vital role in connecting various electronic components. However, due to the complex manufacturing process and numerous links of PCBs, various defects are prone to occur during the production process, such as open circuits, short circuits, cracks, scratches, etc. These problems will not only seriously affect the functional performance of the PCB, but may also shorten its service life. Therefore, for PCBs, high-quality defect detection is not only a necessary means to ensure product reliability, but also a key step to improve the overall production line yield.

[0003] For a long time, traditional methods such as manual visual inspection, optical inspection, and electrical testing, which are widely used in the industry, can identify defects on PCBs to a certain extent. However, manual visual inspection is inefficient and easily affected by subjective factors, resulting in inconsistent inspection results. Although optical inspection can increase the inspection speed, its ability to identify subtle defects is limited. Although electrical testing has high accuracy, its equipment cost is expensive, which increases the financial burden of manufacturers.

[0004] In recent years, with the development of deep learning, researchers have introduced deep learning-based target detection technology in PCB defect detection. Although deep learning methods have shown great potential in improving recognition accuracy, these algorithms usually require a large number of parameters and computing resources, making them difficult to process in real time and widely deployed, especially in resource-constrained industrial environments. In this context, how to further lightweight the algorithm model while improving detection speed and accuracy has become an urgent problem to be solved in this field. Summary of the invention

[0005] The technical problem to be solved by the embodiments of the present application is to provide a training method, device, electronic device and medium for a PCB defect detection model, so as to improve the detection accuracy and efficiency of industrial PCB defects while reducing the model size.

[0006] In a first aspect, an embodiment of the present application provides a method for training a PCB defect detection model, the method comprising:

[0007] Acquire a model training sample, wherein the model training sample includes PCB images of different physical defect types, and each of the PCB images is marked with marked defect information of the PCB;

[0008] Inputting the model training samples into a PCB defect detection model to be trained, wherein the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on SENetV2 architecture and partial convolution;

[0009] Calling the adaptive feature fusion module to process the PCB image using average pooling and maximum pooling strategies to obtain a spliced ​​feature map;

[0010] Calling the feature aggregation module to perform extrusion, excitation and feature partial convolution operations on the spliced ​​feature map to obtain predicted defect information corresponding to the PCB image;

[0011] Based on the marked defect information and the predicted defect information, a loss value of the PCB defect detection model to be trained is calculated;

[0012] When the loss value is within a preset range, the trained PCB defect detection model to be trained is used as the final PCB defect detection model for detecting physical defects of the PCB.

[0013] Optionally, calling the adaptive feature fusion module to process the PCB image using average pooling and maximum pooling strategies to obtain a spliced ​​feature map includes:

[0014] Calling the adaptive feature fusion module to extract image features of the PCB image and obtain an input feature map;

[0015] Performing average pooling processing on the input feature map to obtain a pooled feature map with reduced spatial resolution;

[0016] The pooled feature map is equally divided to obtain a first feature map and a second feature map;

[0017] Down-sampling the first feature map using a convolution kernel of a first size to obtain a first sampled feature map;

[0018] Performing maximum pooling processing on the second feature map, and down-sampling the maximum pooled feature map using a convolution kernel of a second size to obtain a second sampled feature map;

[0019] The first sampling feature map and the second sampling feature map are spliced ​​at the channel level to obtain the spliced ​​feature map.

[0020] Optionally, the feature aggregation module includes: a SENetV2 network and a partial convolutional network,

[0021] The calling of the feature aggregation module to perform squeezing, excitation and feature partial convolution operations on the splicing feature map to obtain predicted defect information corresponding to the PCB image includes:

[0022] Calling the SENetV2 network to perform squeezing and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibrated feature map;

[0023] Calling the partial convolution network to perform a feature partial convolution operation on the calibration feature map in a mask processing manner to obtain a convolution feature map;

[0024] The convolution feature map is processed to obtain predicted defect information corresponding to the PCB image.

[0025] Optionally, calling the SENetV2 network to perform squeezing and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibrated feature map includes:

[0026] Calling the SENetV2 network to transform the concatenated feature map to obtain a transformed feature map;

[0027] Performing average pooling compression processing on the transformed feature map to obtain multiple channel descriptors;

[0028] The multiple channel descriptors are processed respectively by two fully connected layers to obtain multiple channel weights;

[0029] The transformed feature map is weighted based on the channel weight to obtain the calibration feature map.

[0030] Optionally, the processing the multiple channel descriptors respectively through two fully connected layers to obtain multiple channel weights includes:

[0031] Calling the first fully connected layer and the ReLU activation function to perform dimensionality reduction processing on the multiple channel descriptors to obtain dimensionality reduction descriptors of the multiple channels;

[0032] The second fully connected layer and the sigmoid activation function are called to increase the dimension of the reduced-dimensional descriptors of the multiple channels to obtain multiple channel weights.

[0033] In a second aspect, an embodiment of the present application provides a training device for a PCB defect detection model, the device comprising:

[0034] A model sample acquisition module is used to acquire model training samples, wherein the model training samples include PCB images of different physical defect types, and each of the PCB images is annotated with the annotated defect information of the PCB;

[0035] A model sample input module, used for inputting the model training sample into a PCB defect detection model to be trained, wherein the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on SENetV2 architecture and partial convolution;

[0036] A splicing feature map acquisition module is used to call the adaptive feature fusion module to process the PCB image using average pooling and maximum pooling strategies to obtain a splicing feature map;

[0037] A predicted defect information acquisition module is used to call the feature aggregation module to perform extrusion, excitation and feature partial convolution operations on the splicing feature map to obtain predicted defect information corresponding to the PCB image;

[0038] A loss value calculation module, used for calculating the loss value of the PCB defect detection model to be trained based on the marked defect information and the predicted defect information;

[0039] The defect detection model acquisition module is used to use the trained PCB defect detection model to be trained as the final PCB defect detection model for detecting physical defects of the PCB when the loss value is within a preset range.

[0040] Optionally, the splicing feature map acquisition module includes:

[0041] An input feature map acquisition unit is used to call the adaptive feature fusion module to extract the image features of the PCB image to obtain an input feature map;

[0042] A pooling feature map acquisition unit, used to perform average pooling processing on the input feature map to obtain a pooling feature map with reduced spatial resolution;

[0043] A feature map division unit, used for equally dividing the pooled feature map to obtain a first feature map and a second feature map;

[0044] A first feature map acquisition unit, configured to downsample the first feature map using a convolution kernel of a first size to obtain a first sampled feature map;

[0045] A second feature map acquisition unit, used to perform maximum pooling processing on the second feature map, and downsample the maximum pooled feature map using a convolution kernel of a second size to obtain a second sampled feature map;

[0046] A splicing feature map acquisition unit is used to perform channel-level splicing processing on the first sampling feature map and the second sampling feature map to obtain the splicing feature map.

[0047] Optionally, the feature aggregation module includes: a SENetV2 network and a partial convolutional network,

[0048] The predicted defect information acquisition module includes:

[0049] A calibration feature map acquisition unit, used for calling the SENetV2 network to perform squeezing and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibration feature map;

[0050] A convolution feature map acquisition unit, used for calling the partial convolution network to perform a feature partial convolution operation on the calibration feature map in a mask processing manner to obtain a convolution feature map;

[0051] The predicted defect information acquisition unit is used to process the convolution feature map to obtain the predicted defect information corresponding to the PCB image.

[0052] Optionally, the calibration characteristic map acquisition unit includes:

[0053] A transformation feature acquisition subunit is used to call the SENetV2 network to transform the spliced ​​feature map to obtain a transformation feature map;

[0054] A channel descriptor acquisition subunit is used to perform average pooling compression processing on the transformed feature map to obtain multiple channel descriptors;

[0055] A channel weight acquisition subunit, used for processing the multiple channel descriptors respectively through two fully connected layers to obtain multiple channel weights;

[0056] The calibration feature map acquisition subunit is used to perform weighted processing on the transformation feature map based on the channel weight to obtain the calibration feature map.

[0057] Optionally, the channel weight acquisition subunit includes:

[0058] A dimension reduction descriptor acquisition subunit is used to call the first fully connected layer and the ReLU activation function to perform dimension reduction processing on the multiple channel descriptors to obtain dimension reduction descriptors of multiple channels;

[0059] The multiple channel weight acquisition subunits are used to call the second fully connected layer and the sigmoid activation function to increase the dimension of the reduced-dimensional descriptors of the multiple channels to obtain multiple channel weights.

[0060] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0061] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods for training a PCB defect detection model when executing the program.

[0062] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned methods for training a PCB defect detection model.

[0063] Compared with the prior art, the embodiments of the present application have the following advantages:

[0064] In an embodiment of the present application, by obtaining a model training sample, the model training sample includes PCB images of different physical defect types, and each PCB image is annotated with the annotated defect information of the PCB. The model training sample is input into the PCB defect detection model to be trained, and the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on the SENetV2 architecture and partial convolution. The adaptive feature fusion module is called to process the PCB image with average pooling and maximum pooling strategies to obtain a spliced ​​feature map. The feature aggregation module is called to perform squeezing, excitation and feature partial convolution operations on the spliced ​​feature map to obtain the predicted defect information corresponding to the PCB image. Based on the annotated defect information and the predicted defect information, the loss value of the PCB defect detection model to be trained is calculated. When the loss value is within a preset range, the trained PCB defect detection model to be trained is used as the final PCB defect detection model for detecting physical defects of the PCB. By introducing an adaptive feature fusion module in the target detection model, the embodiment of the present application can retain key information while reducing the dimension of the feature space, thereby overcoming the possibility of reducing the model detection performance while reducing the number of model parameters. The overall performance of the model is effectively optimized, and the detection accuracy and efficiency of industrial PCB defects can be improved on the basis of reducing the model size. At the same time, the use of partial feature convolution not only improves the detection accuracy of the model, but also reduces the number of model parameters.

[0065] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flowchart of a method for training a PCB defect detection model provided in an embodiment of the present application;

[0067] Figure 2 A schematic diagram of an adaptive feature fusion module provided in an embodiment of the present application;

[0068] Figure 3 A schematic diagram of an introduced SaE block provided in an embodiment of the present application;

[0069] Figure 4 A schematic diagram of a Partial Convolution provided in an embodiment of the present application;

[0070] Figure 5 A schematic diagram of an improved C2f_P structure provided in an embodiment of the present application;

[0071] Figure 6 A schematic diagram of a model training and PCB defect detection process provided in an embodiment of the present application;

[0072] Figure 7 A schematic diagram of the structure of a training device for a PCB defect detection model provided in an embodiment of the present application;

[0073] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0075] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0076] Reference Figure 1 , shows a flowchart of the steps of a training method for a PCB defect detection model provided in an embodiment of the present application, such as Figure 1 As shown, the training method of the PCB defect detection model may include: step 101, step 102, step 103, step 104, step 105 and step 106.

[0077] Step 101: Obtain model training samples, where the model training samples include PCB images with different physical defect types, and each of the PCB images is annotated with PCB defect information.

[0078] The embodiments of the present application can be applied to the scenario of training an improved PCB defect detection model.

[0079] PCB (printed circuit board) is the provider of electrical connections for electronic components. It is mainly composed of an insulating base plate, connecting wires and pads for assembling and soldering electronic components.

[0080] When training a PCB defect detection model, model training samples for model training may be obtained, wherein the model training samples may include PCB images of different physical defect types, and each PCB image is annotated with annotated defect information of the PCB.

[0081] In the specific implementation, you can search for existing open source data sets, such as the PCB defect data sets provided by platforms such as Roboflow. These data sets are usually pre-processed and annotated, and are suitable for direct use in model training.

[0082] Of course, if you cannot find an open source dataset that meets your needs, you can consider building your own dataset. This includes collecting PCB images of different types of physical defects and annotating them. Specifically, you can use a high-precision industrial camera and appropriate light source settings to scan and photograph PCB boards with physical defects to obtain clear and stable images (pay attention to the choice of light source to reduce the impact of reflections and shadows and improve image quality). Then, you can annotate the defects of the collected images, which can be done manually or using professional annotation tools. The annotation information can include detailed information such as the type, location, and size of the defect.

[0083] It can be understood that the model training samples also include PCB images without physical defects, and the sample data set for model training is formed by the PCB images with physical defects and the PCB images without physical defects.

[0084] After obtaining the model training samples, the PCB images in the model training samples can also be preprocessed. Specifically, the noise in the image can be removed and the signal-to-noise ratio of the image can be improved through filtering, smoothing and other algorithms. The image contrast, brightness and other parameters can also be adjusted to make the defect features more prominent.

[0085] Of course, in practical applications, other preprocessing methods may be used to preprocess the PCB image, and this embodiment does not limit the preprocessing method.

[0086] After obtaining the model training samples, the model training samples can be divided into a training set and a validation set according to a set ratio (such as 8:2, etc.). The training set can be used to train the PCB defect detection model to be trained, and the validation set can be used to verify the model accuracy of the trained model.

[0087] After the model training samples are obtained, step 102 is executed.

[0088] Step 102: input the model training samples into the PCB defect detection model to be trained, wherein the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on SENetV2 architecture and partial convolution.

[0089] The PCB defect detection model to be trained may be an improved YOLOv8 model. The PCB defect detection model to be trained may include: an adaptive feature fusion module, a feature aggregation module based on SENetV2 architecture and partial convolution.

[0090] In this embodiment, an adaptive feature fusion module (ADown) can be used to improve the original feature extraction Conv module, the SENetV2 architecture can be introduced to improve the C2f (CSP Bottleneck with 2Convolutions) module in the backbone network of the YOLOv8 model, and ParticalConv (PConv) can be used to replace the convolution in the C2f module to improve and optimize the original YOLOv8 target detection algorithm.

[0091] After the model training samples are obtained, the model training samples can be input into the PCB defect detection model to be trained to perform model training.

[0092] After the model training samples are input into the PCB defect detection model to be trained, step 103 is performed.

[0093] Step 103: Call the adaptive feature fusion module to process the PCB image using average pooling and maximum pooling strategies to obtain a spliced ​​feature map.

[0094] After the model training samples are input into the PCB defect detection model to be trained, the adaptive feature fusion module can be called to process the PCB image using the average pooling and maximum pooling strategies to obtain a spliced ​​feature map. Specifically, the image features of the PCB image can be first extracted, and the extracted features can be processed using the average pooling strategy, and then the features can be equally divided and processed using the maximum pooling strategy, and the processed feature maps can be spliced ​​to obtain a spliced ​​feature map. The implementation process can be described in detail in conjunction with the following specific implementation method, and this embodiment does not limit this.

[0095] In a specific implementation of the present application, the above step 103 may include:

[0096] Sub-step A1: calling the adaptive feature fusion module to extract image features of the PCB image and obtain an input feature map.

[0097] In this embodiment, after the model training sample is input into the PCB defect detection model to be trained, the adaptive feature fusion module can be called to extract the image features of the PCB image to obtain the input feature map. In a specific implementation, the adaptive feature fusion module (ADown) can include some pre-trained convolutional layers or other feature extractors for extracting useful feature information from the image. The output is a feature map (usually a multidimensional array) containing image features.

[0098] After calling the adaptive feature fusion module to extract the image features of the PCB image and obtain the input feature map, sub-step A2 is performed.

[0099] Sub-step A2: performing average pooling processing on the input feature map to obtain a pooled feature map with reduced spatial resolution.

[0100] After calling the adaptive feature fusion module to extract the image features of the PCB image to obtain the input feature map, the input feature map can be averaged and pooled to obtain a pooled feature map with reduced spatial resolution. Specifically, average pooling is a commonly used pooling operation that is used to reduce the spatial resolution of the feature map (i.e., reduce the width and height of the feature map) while retaining important feature information, which helps to reduce the amount of calculation and extract more representative features.

[0101] After performing average pooling processing on the input feature map to obtain a pooled feature map with reduced spatial resolution, sub-step A3 is executed.

[0102] Sub-step A3: equally divide the pooled feature map to obtain a first feature map and a second feature map.

[0103] After the input feature map is averaged and pooled to obtain a pooled feature map with reduced spatial resolution, the pooled feature map can be equally divided to obtain a first feature map and a second feature map. Specifically, the pooled feature map can be equally divided into two parts along a certain dimension (such as a channel dimension), which are used as the first feature map and the second feature map, respectively, to facilitate subsequent different processing of the features.

[0104] After equally dividing the pooled feature map to obtain the first feature map and the second feature map, sub-step A4 and sub-step A5 are executed.

[0105] Sub-step A4: down-sample the first feature map using a convolution kernel of a first size to obtain a first sampled feature map.

[0106] After obtaining the first feature map, the first feature map may be downsampled using a convolution kernel of a first size to obtain a first sampled feature map. Specifically, downsampling is usually implemented using a convolution operation, and the spatial resolution of the feature map is reduced by selecting an appropriate convolution kernel size and step size. In this step, a convolution kernel of a first size may be used to downsample the first feature map.

[0107] Sub-step A5: performing maximum pooling processing on the second feature map, and down-sampling the maximum pooled feature map using a convolution kernel of a second size to obtain a second sampled feature map.

[0108] After obtaining the second feature map, the second feature map may be subjected to maximum pooling, and the maximum pooled feature map may be downsampled using a convolution kernel of a second size to obtain a second sampled feature map. Specifically, maximum pooling is a pooling method different from average pooling, which selects the maximum value in each pooling window as output. In this step, the second feature map may be subjected to maximum pooling, and then downsampled using a convolution kernel of a second size.

[0109] Sub-step A6: performing channel-level splicing processing on the first sampling feature map and the second sampling feature map to obtain the spliced ​​feature map.

[0110] After obtaining the first sampling feature map and the second sampling feature map, the first sampling feature map and the second sampling feature map may be spliced ​​at the channel level to obtain a spliced ​​feature map. Specifically, the first sampling feature map and the second sampling feature map may be spliced ​​in the channel dimension. This means merging the channels of the two sampling feature maps into a larger feature map that contains feature information extracted from two different processing paths.

[0111] The structure of the adaptive feature fusion module can be as follows Figure 2 As shown. The input feature map of the adaptive feature fusion module (ADown) is X. The adaptive feature fusion module first performs average pooling on the input feature map X to obtain a feature map P with reduced spatial resolution. Then, the adaptive feature fusion module can equally divide the pooled feature map into two parts, P1 and P2, corresponding to the left and right branches of the module respectively. The left branch downsamples P1 through a 3x3 convolution kernel to obtain C1, thereby enhancing the ability to capture local details of the image. The right branch first uses the maximum pooling process on P2 to emphasize the most significant features, and then uses 1x1 convolution to achieve feature compression and fusion to obtain C2. This process can be shown as follows:

[0112] P1,P2=Split(AvgPool(X))

[0113] C1=Conv3×3 (P1)

[0114] C2=Conv 1×1 (MaxPool(P2))

[0115] Finally, the two processed features are spliced ​​at the channel level to obtain the output feature map Y. This not only ensures that the information from different processing branches can be effectively combined, but also enriches the feature expression ability of the model. This improves the recognition ability of the model in PCB detection. The output expression can be shown as follows:

[0116] Y=Concat(C1,C2).

[0117] The embodiment of the present application reduces the size of the feature map by selecting the maximum value in the local area, thereby reducing the dimension of the feature space. While reducing the dimension, since the maximum value is selected, some key feature information in the image can often be retained, which is very important for the model to accurately identify PCB defects. By retaining key information while reducing the dimension of the feature space, the key information is retained while reducing the dimension of the feature space. In addition, the feature map is divided and differentiated, which effectively reduces the amount of calculation and the amount of parameters, thereby improving the overall calculation efficiency of the system.

[0118] After calling the adaptive feature fusion module to process the PCB image using the average pooling and maximum pooling strategies to obtain a spliced ​​feature map, step 104 is executed.

[0119] Step 104: calling the feature aggregation module to perform extrusion, excitation and feature partial convolution operations on the spliced ​​feature map to obtain predicted defect information corresponding to the PCB image.

[0120] After calling the adaptive feature fusion module to process the PCB image using the average pooling and maximum pooling strategies to obtain the spliced ​​feature map, the feature aggregation module can be called to perform extrusion, excitation and feature partial convolution operations on the spliced ​​feature map to obtain the predicted defect information corresponding to the PCB image. Specifically, the SENetV2 network can be called to perform extrusion and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibration feature map. Furthermore, a partial convolution network can be called to perform feature partial convolution operations on the calibration feature map using a mask processing method, and the convolution feature map can be processed to obtain the predicted defect information corresponding to the PCB image. This implementation process can be described in detail in conjunction with the following specific implementation method.

[0121] In a specific implementation of the present application, the above step 104 may include:

[0122] Sub-step B1: Call the SENetV2 network to perform squeezing and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibrated feature map.

[0123] In this embodiment, SENetV2 is an image classification model that combines a Squeeze-and-Excitation (SE) module and a dense layer. The model aims to improve image classification performance by enhancing feature representation. Specifically, it recalibrates and activates channel features and global features to make the network more focused on key features, thereby improving classification accuracy.

[0124] The SENetv2 architecture is an improvement on SENet. It combines the characteristics of ResNeXt and SENet, and uses a multi-branch fully connected layer (FC) to squeeze and excite features to enhance the learning ability of the network. This paper lightweights the yolov8 model, which reduces the number of parameters and reduces the model detection performance. This embodiment adopts this architecture to maintain accuracy.

[0125] After obtaining the spliced ​​feature map, the SENetV2 network can be called to squeeze and excite the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibrated feature map. The implementation process can be described in detail in conjunction with the following specific implementation method.

[0126] In another specific implementation of the present application, the above sub-step B1 may include:

[0127] Sub-step C1: calling the SENetV2 network to transform the concatenated feature map to obtain a transformed feature map.

[0128] In this embodiment, after obtaining the spliced ​​feature map, the SENetV2 network can be called to transform the spliced ​​feature map to obtain a transformed feature map. Its main function is to transform the input feature map into another feature map. This transformation can be achieved through a convolution operation, the purpose of which is to extract and transform the spatial features of the input feature map before performing the Squeeze operation, so that the subsequent modules can effectively capture and utilize the correlation between channels.

[0129] After calling the SENetV2 network to transform the concatenated feature map and obtaining the transformed feature map, execute sub-step C2.

[0130] Sub-step C2: performing average pooling compression processing on the transformed feature map to obtain multiple channel descriptors.

[0131] After calling the SENetV2 network to transform the concatenated feature map and obtain the transformed feature map, the transformed feature map can be average pooled and compressed to obtain multiple channel descriptors. Specifically, this step is the "Squeeze" operation, which compresses the feature map of each channel into a single value (i.e., channel descriptor) by performing global average pooling on the transformed feature map, thereby obtaining a vector equal to the number of channels. This vector represents the importance or response strength of different channels.

[0132] After performing average pooling compression processing on the transformed feature map to obtain multiple channel descriptors, sub-step C3 is executed.

[0133] Sub-step C3: Processing the multiple channel descriptors respectively through two fully connected layers to obtain multiple channel weights.

[0134] After the transformed feature map is average-pooled and compressed to obtain multiple channel descriptors, the multiple channel descriptors can be processed separately through two fully-connected layers to obtain multiple channel weights. The specific implementation process can be: call the first fully-connected layer and the ReLU activation function to reduce the dimensions of multiple channel descriptors to obtain reduced-dimensional descriptors of multiple channels. Call the second fully-connected layer and the sigmoid activation function to increase the dimensions of the reduced-dimensional descriptors of multiple channels to obtain multiple channel weights. This step is the core of the "Excitation" operation, which uses two fully-connected layers (usually with nonlinear activation functions, such as ReLU, etc.) to learn the dependencies between channels. The first fully-connected layer usually has a smaller output dimension (for example, 1 / 16 of the number of channels) to reduce the amount of calculation and introduce nonlinearity; the second fully-connected layer restores the output dimension to the same as the number of input channels to generate the weight of each channel.

[0135] like Figure 3 The structure of the SaE block is shown. The input feature map X undergoes a transformation to produce a feature map U. Then, the feature map U is compressed into multiple channel descriptors through average pooling. These descriptors pass through two fully connected layers, the first one is dimensionality reduction and the second one is dimensionality increase, and are activated by activation functions ReLU and Sigmoid. Finally, the original feature map U is multiplied by the learned channel weights to obtain the recalibrated feature map. Squeeze operation: First, the spatial information of each channel is compressed into a single value through global average pooling. In this way, a feature map U with a size of H×W×C is converted into a vector of size 1×1×C, which contains the global information of each channel.

[0136] Excitation operation: Next, a set of fully connected layers (FullyConnected layers), that is, two consecutive linear transformations, are applied to this 1×1×C vector. The first fully connected layer reduces the C-dimensional vector to C / r dimensions (where r is a reduction ratio, usually 16), and then passes it through a ReLU activation function. The second fully connected layer then increases it back to C dimensions. In this process, the parameter W is learned to explicitly model the correlation between feature channels.

[0137] Weight generation: Through the last fully connected layer and sigmoid activation function, a 1×1×C channel attention vector s is obtained. This vector s represents the weight of each channel, and it obtains the importance of each channel through learning.

[0138] Sub-step C3: performing weighted processing on the transformed feature map based on the channel weight to obtain the calibration feature map.

[0139] Then, the transformed feature map can be weighted based on the channel weights to obtain the calibrated feature map. Specifically, this channel attention vector (i.e., channel weight) can be weighted to the original feature map channel by channel to complete the recalibration of the original features in the channel dimension. This weighted operation enables the network to focus on the feature channels that are useful for the current task and suppress those feature channels that are less useful.

[0140] After calling the SENetV2 network to transform the concatenated feature map to obtain the transformed feature map, execute sub-step B2.

[0141] Sub-step B2: calling the partial convolutional network to perform a feature partial convolution operation on the calibration feature map using a mask processing method to obtain a convolution feature map.

[0142] After calling the SENetV2 network to transform the concatenated feature map to obtain the transformed feature map, the partial convolution network can be called to perform a feature partial convolution operation on the calibration feature map using a mask processing method to obtain a convolution feature map. In this example, the partial convolution network can be PConv. The core idea of ​​PConv partial convolution is indeed its flexibility and adaptability to data missing. Compared with traditional convolution operations, partial convolution does not mechanically apply the same convolution kernel to all parts of the input data. Instead, it dynamically determines the scope of the convolution kernel based on the validity of the data, that is, whether the data points are missing or damaged.

[0143] The structure of Partial Convolution can be as follows Figure 4As shown. Among them, Input is the starting point of partial convolution, usually an image containing irregular areas or missing data. In image restoration tasks, these irregular areas may be caused by occlusion, damage, or deletion. In partial convolution, mask is a key component, which is used to indicate which pixels are valid (that is, they should participate in the convolution calculation) and which pixels are invalid (that is, they should not participate in the calculation, usually because they are missing or belong to irregular areas). Partial Convolution Operation is the core step of partial convolution, which combines traditional convolution operations and masks to process the input image. Output is the result of partial convolution, usually an updated image in which irregular areas or missing data have been repaired or filled to some extent.

[0144] Specifically, when partial convolution is processing a convolution window, it first checks the data points within the window. For those valid, non-missing data points, PConv applies the convolution kernel like a regular convolution operation. However, for those missing or invalid data points, PConv chooses to ignore them and not include them in the convolution calculation. This flexibility means that the actual area of ​​action of the convolution kernel may be different in each convolution window. This depends entirely on the completeness and distribution of the data in the window. In this way, partial convolution not only improves the robustness to missing data, but also more effectively extracts and utilizes the remaining valid information.

[0145] A key step in implementing PConv is to define a binary mask that distinguishes the validity of data points. Specifically, for each data point, if it is valid, it is marked as 1 at the corresponding mask position. If the data point is missing, it is marked as 0. When performing a convolution operation, the original data and the mask need to be convolved at the same time. The convolution kernel will only be applied to the positions where the mask value is 1.

[0146] In neural networks, convolution operations are an important means of extracting features. The convolution in the C2f module is responsible for feature extraction in the original model, but there are some problems, such as the possibility of causing feature map redundancy, which in turn increases computational complexity and memory access overhead. PConv replaces the convolution in the C2f module to overcome these problems and improve the performance of the model from the root.

[0147] After calling the partial convolution network to perform a feature partial convolution operation on the calibration feature map using a mask processing method to obtain a convolution feature map, sub-step B3 is executed.

[0148] Sub-step B3: Process the convolution feature map to obtain predicted defect information corresponding to the PCB image.

[0149] After calling a partial convolutional network to perform a feature partial convolution operation on the calibration feature map using a mask processing method to obtain a convolution feature map, the convolution feature map can be processed to obtain the predicted defect information corresponding to the PCB image. Specifically, one or more fully connected layers, classifiers, regressors, or upsampling layers can be used, depending on the nature of the task (for example, whether it is a classification task or a regression task, and the specific form of the defect information). In a classification task, the model output can be a probability distribution representing the likelihood of each defect category. In a regression task, the model output can be one or more continuous values ​​representing the location, size, or other attributes of the defect.

[0150] The processing of the feature aggregation module can be combined with Figure 3 This is described in detail as follows.

[0151] like Figure 3 As shown, in order to avoid the decrease in model detection performance while the number of parameters decreases. The present invention improves the C2f module in the backbone network by introducing the SENetV2 architecture. The SENetv2 architecture is an improvement on SENet, combining the characteristics of ResNeXt and SENet. It squeezes and excites features through a multi-branch fully connected layer (FC), enhances the global representation learning ability of the network, and thus obtains better feature representation. The input feature map X is transformed to generate a feature map U. Then, the feature map U is compressed into multiple channel descriptors by average pooling. These descriptors pass through two fully connected layers, the first one is dimensionality reduction and the second one is dimensionality increase, and are activated by activation functions ReLU and Sigmoid. Finally, the original feature map U is multiplied by the learned channel weights to obtain a recalibrated feature map.

[0152] The embodiment of the present application avoids feature map redundancy by introducing ParticalConv (PConv) to replace the convolution in the C2f module, thereby achieving the purpose of reducing computational complexity and memory access, and facilitating deployment on devices with limited computing resources.

[0153] The improved C2f_P structure can be Figure 5 shown.

[0154] The input of this structure can be: H×W×C (H represents the height of the image, W represents the width of the image, and C represents the number of channels of the image). ConvModule is a convolution module that can include components such as convolution layers, activation functions (such as ReLU), and batch normalization to extract and transform the input feature map. Split (split operation) splits the input feature map into two parts along a certain dimension (usually the channel dimension). PConv (partial convolution) is used to process images containing missing or irregular areas. Unlike standard convolution, PConv uses a mask to indicate which pixels are valid, thereby avoiding the use of invalid pixels during the convolution process. In the improved C2f_P, PConv is used twice and can be used to gradually repair or enhance the information in the feature map. The Concat (concatenation) operation can merge multiple feature maps into one feature map along a certain dimension (usually the channel dimension) so that subsequent processing can utilize information from different paths. Conv (convolution layer) is used to further extract and transform the output of Concat to obtain the final output.

[0155] When training the model, you can use the forward propagation training method, that is, in each iteration of model training, the image data in the training set is forward propagated through the neural network. That is, the image data is passed to the model, and passes through a series of convolution and activation layers. In each layer, the input data is weighted, activated, and transformed, and higher-level features are gradually extracted.

[0156] Step 105: Based on the marked defect information and the predicted defect information, a loss value of the PCB defect detection model to be trained is calculated.

[0157] After obtaining the predicted defect information, the loss value of the PCB defect detection model to be trained can be calculated based on the labeled defect information and the predicted defect information.

[0158] In this example, the loss calculation can be used to calculate the loss value between the model's predicted box and the true box, including target classification error, target position error, and target confidence error. Among them, the classification loss uses BCE Loss (binary cross entropy loss), and the regression loss includes DF Loss (distribution focal loss) and CIOULoss (complete intersection over union loss).

[0159] The calculation formula of BCE loss is as follows:

[0160] BCE loss=-[ylog(p)+(1-y)log(1-p)]

[0161] In the above formula, p is the predicted value, y is the true label, the cross entropy loss between each predicted value and the true label is calculated, and then the loss of all predicted values ​​is summarized and averaged to obtain the final classification loss.

[0162] DFL combines the ideas of focal loss and distribution loss to alleviate the problem of unbalanced distribution of positive and negative samples in target detection. The calculation formula of DFL can be shown as follows:

[0163] DFL(s i ,s i+1 )=-[(y i+1 -y)log(s i )+(yy i )log(s i+1 )]

[0164] In the above formula, s i and i+1 are the two limits of the label distribution, y represents the position of the positive sample, and y i and i+1 The integer part representing the label distribution.

[0165] The bounds of the label distribution refer to a range set for the label of each category or position when designing the loss function. Usually, in the target detection task, the position of each target can be represented by a bounding box, and the coordinate values ​​of these bounding boxes are continuous. In order to be able to use discrete neural network outputs to approximate continuous target positions, the DFL method maps continuous coordinate values ​​to a discrete distribution. The "bounds of the label distribution" here refers to the boundary of this discrete distribution, that is, the range of actual continuous coordinates corresponding to each discrete value.

[0166] The role of CIoU is to measure the degree of overlap between the predicted box and the true target box, which is similar to the traditional IoU indicator, but CIoU is more accurate and comprehensive, and performs well in dealing with highly overlapping or incomplete target boxes. Its formula can be shown as follows:

[0167]

[0168] In the above formula, b, b gt denote the center points of the predicted box and the real box respectively, ρ denotes the Euclidean distance between the two center points, c denotes the diagonal length of the minimum circumscribed rectangle of the predicted box and the real box, α denotes the balance coefficient, v is a parameter to measure the consistency of the aspect ratio, w, h and w gt 、h gtRepresents the width and height of the predicted box and the width and height of the real box respectively. Intersection represents intersection and Union represents union. IOU is used to measure the degree of overlap between the predicted bounding box and the real bounding box. The value of IOU ranges from 0 to 1, where 1 represents complete overlap and 0 represents no overlap.

[0169] After the loss value of the PCB defect detection model to be trained is calculated based on the marked defect information and the predicted defect information, step 106 is performed.

[0170] Step 106: When the loss value is within a preset range, the trained PCB defect detection model to be trained is used as a final PCB defect detection model for detecting physical defects of the PCB.

[0171] After the loss value of the PCB defect detection model to be trained is calculated based on the marked defect information and the predicted defect information, it can be determined whether the loss value is within a preset range. If the loss value is within the preset range, the trained PCB defect detection model to be trained can be used as the final PCB defect detection model for detecting physical defects of the PCB.

[0172] In this embodiment, in order to facilitate the evaluation of the performance of the improved model, accuracy (P), recall (R), average mean accuracy (mAP), frames per second (FPS), and weight file (MB) size are used as evaluation indicators in this article.

[0173] Next, combine Figure 6 The improved lightweight PCB defect detection process based on YOLOv8 is described in detail, which may include:

[0174] Step 1: Collect and annotate PCB defect image data, and preprocess the images to generate a PCB defect detection dataset.

[0175] Step 2: Divide the processed data set into training set and test set and use them as the input of the model.

[0176] Step 3: Improve and optimize the original YOLOv8 target detection algorithm by introducing the Adown module, SENetV2 architecture and PConv convolution.

[0177] Step 4: Use the improved YOLOv8 model to train the training set and validation set to obtain the optimal PCB defect detection model.

[0178] Step 5: Deploy the trained lightweight PCB defect detection model to the mobile terminal for detection and obtain the detection results.

[0179] This embodiment verifies the beneficial effects of the method provided in this embodiment through specific experiments.

[0180] This embodiment conducts a series of ablation experiments to verify the effectiveness of the proposed algorithm. The ablation experiment results are shown in the following table. Precision represents the accuracy, mAP represents the average accuracy, Params represents the number of parameters of the model, and FLOPs represents the amount of floating-point calculations of the model.

[0181] Table 1 Ablation experiment

[0182]

[0183]

[0184] From the final results of the ablation experiment, the algorithm provided in this embodiment has the highest accuracy and mAP, and the Params (number of parameters) and GFLOPs (Giga Floating Point Operations Per Second) are only 72% and 77% of the initial model. Therefore, these improvements are effective.

[0185] In order to fully verify the various parameter indicators of this embodiment, six other common target detection algorithms (YOLOv3, YOLO5, YOLOv7, YOLOv8n, YOLOv8s) are also compared with the algorithm proposed in this embodiment. As shown in Table 2 below:

[0186] Table 2 Comparative experiment

[0187]

[0188] From the comparison results, it can be seen that the Params and GFLOPs of this embodiment are the smallest among all models, but it has the highest Precision and mAP second only to YOLOv8s. The algorithm proposed in this embodiment can indeed improve the accuracy of PCB defect detection under the premise of lightweight.

[0189] The training method of the PCB defect detection model provided in the embodiment of the present application obtains a model training sample, and the model training sample includes PCB images of different physical defect types, and each PCB image is annotated with the annotated defect information of the PCB. The model training sample is input into the PCB defect detection model to be trained, and the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on the SENetV2 architecture and partial convolution. The adaptive feature fusion module is called to process the PCB image with average pooling and maximum pooling strategies to obtain a spliced ​​feature map. The feature aggregation module is called to perform squeezing, excitation and feature partial convolution operations on the spliced ​​feature map to obtain the predicted defect information corresponding to the PCB image. Based on the annotated defect information and the predicted defect information, the loss value of the PCB defect detection model to be trained is calculated. When the loss value is within a preset range, the trained PCB defect detection model to be trained is used as the final PCB defect detection model for detecting physical defects of the PCB. By introducing an adaptive feature fusion module in the target detection model, the embodiment of the present application can retain key information while reducing the dimension of the feature space, and overcomes the possibility of reducing the model detection performance while reducing the model parameter amount. The overall performance of the model is effectively optimized, and the detection accuracy and efficiency of industrial PCB defects can be improved on the basis of reducing the model size. At the same time, the use of partial feature convolution not only improves the detection accuracy of the model, but also reduces the number of model parameters.

[0190] Reference Figure 7 , shows a schematic diagram of the structure of a training device for a PCB defect detection model provided in an embodiment of the present application, such as Figure 7 As shown, the training device 700 of the PCB defect detection model may include the following modules:

[0191] The model sample acquisition module 710 is used to acquire model training samples, wherein the model training samples include PCB images of different physical defect types, and each of the PCB images is annotated with the annotated defect information of the PCB;

[0192] A model sample input module 720, used to input the model training sample into a PCB defect detection model to be trained, wherein the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on SENetV2 architecture and partial convolution;

[0193] The splicing feature map acquisition module 730 is used to call the adaptive feature fusion module to process the PCB image using average pooling and maximum pooling strategies to obtain a splicing feature map;

[0194] The predicted defect information acquisition module 740 is used to call the feature aggregation module to perform extrusion, excitation and feature partial convolution operations on the spliced ​​feature map to obtain the predicted defect information corresponding to the PCB image;

[0195] A loss value calculation module 750, configured to calculate a loss value of the PCB defect detection model to be trained based on the marked defect information and the predicted defect information;

[0196] The defect detection model acquisition module 760 is used to use the trained PCB defect detection model to be trained as the final PCB defect detection model for detecting physical defects of the PCB when the loss value is within a preset range.

[0197] Optionally, the splicing feature map acquisition module includes:

[0198] An input feature map acquisition unit is used to call the adaptive feature fusion module to extract the image features of the PCB image to obtain an input feature map;

[0199] A pooling feature map acquisition unit, used to perform average pooling processing on the input feature map to obtain a pooling feature map with reduced spatial resolution;

[0200] A feature map division unit, used for equally dividing the pooled feature map to obtain a first feature map and a second feature map;

[0201] A first feature map acquisition unit, configured to downsample the first feature map using a convolution kernel of a first size to obtain a first sampled feature map;

[0202] A second feature map acquisition unit, used to perform maximum pooling processing on the second feature map, and downsample the maximum pooled feature map using a convolution kernel of a second size to obtain a second sampled feature map;

[0203] A splicing feature map acquisition unit is used to perform channel-level splicing processing on the first sampling feature map and the second sampling feature map to obtain the splicing feature map.

[0204] Optionally, the feature aggregation module includes: a SENetV2 network and a partial convolutional network,

[0205] The predicted defect information acquisition module includes:

[0206] A calibration feature map acquisition unit, used for calling the SENetV2 network to perform squeezing and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibration feature map;

[0207] A convolution feature map acquisition unit, used for calling the partial convolution network to perform a feature partial convolution operation on the calibration feature map in a mask processing manner to obtain a convolution feature map;

[0208] The predicted defect information acquisition unit is used to process the convolution feature map to obtain the predicted defect information corresponding to the PCB image.

[0209] Optionally, the calibration characteristic map acquisition unit includes:

[0210] A transformation feature acquisition subunit is used to call the SENetV2 network to transform the spliced ​​feature map to obtain a transformation feature map;

[0211] A channel descriptor acquisition subunit is used to perform average pooling compression processing on the transformed feature map to obtain multiple channel descriptors;

[0212] A channel weight acquisition subunit, used for processing the multiple channel descriptors respectively through two fully connected layers to obtain multiple channel weights;

[0213] The calibration feature map acquisition subunit is used to perform weighted processing on the transformation feature map based on the channel weight to obtain the calibration feature map.

[0214] Optionally, the channel weight acquisition subunit includes:

[0215] A dimension reduction descriptor acquisition subunit is used to call the first fully connected layer and the ReLU activation function to perform dimension reduction processing on the multiple channel descriptors to obtain dimension reduction descriptors of multiple channels;

[0216] The multiple channel weight acquisition subunits are used to call the second fully connected layer and the sigmoid activation function to increase the dimension of the reduced-dimensional descriptors of the multiple channels to obtain multiple channel weights.

[0217] The training device of the PCB defect detection model provided in the embodiment of the present application obtains model training samples, and the model training samples include PCB images of different physical defect types, and each PCB image is annotated with the annotated defect information of the PCB. The model training samples are input into the PCB defect detection model to be trained, and the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on the SENetV2 architecture and partial convolution. The adaptive feature fusion module is called to process the PCB image with average pooling and maximum pooling strategies to obtain a spliced ​​feature map. The feature aggregation module is called to perform squeezing, excitation and feature partial convolution operations on the spliced ​​feature map to obtain the predicted defect information corresponding to the PCB image. Based on the annotated defect information and the predicted defect information, the loss value of the PCB defect detection model to be trained is calculated. When the loss value is within a preset range, the trained PCB defect detection model to be trained is used as the final PCB defect detection model for detecting physical defects of the PCB. By introducing an adaptive feature fusion module in the target detection model, the embodiment of the present application can retain key information while reducing the dimension of the feature space, thereby overcoming the possibility of reducing the model detection performance while reducing the model parameter amount. The overall performance of the model is effectively optimized, and the detection accuracy and efficiency of industrial PCB defects can be improved on the basis of reducing the model size. At the same time, the use of partial feature convolution not only improves the detection accuracy of the model, but also reduces the number of model parameters.

[0218] An embodiment of the present application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the training method of the above-mentioned PCB defect detection model when executed by the processor.

[0219] Figure 8 FIG. 8 is a schematic diagram showing the structure of an electronic device 800 according to an embodiment of the present invention. Figure 8 As shown, the electronic device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 802 or computer program instructions loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0220] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, a microphone, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0221] The various processes and processing described above may be performed by the processing unit 801. For example, the method of any of the above embodiments may be implemented as a computer software program, which is tangibly contained in a computer-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the CPU 801, one or more actions in the method described above may be performed.

[0222] In addition, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the training method of the above-mentioned PCB defect detection model is implemented.

[0223] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0224] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0225] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminals (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0226] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0227] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal so that a series of operating steps are executed on the computer or other programmable terminal to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0228] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0229] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or terminal including the elements.

[0230] The above is a detailed introduction to a training method for a PCB defect detection model, a training device for a PCB defect detection model, an electronic device and a computer-readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A training method for a PCB defect detection model, characterized in that: The method comprises: Acquire a model training sample, wherein the model training sample includes PCB images of different physical defect types, and each of the PCB images is marked with marked defect information of the PCB; Inputting the model training samples into a PCB defect detection model to be trained, wherein the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on SENetV2 architecture and partial convolution; Calling the adaptive feature fusion module to process the PCB image using average pooling and maximum pooling strategies to obtain a spliced ​​feature map; Calling the feature aggregation module to perform extrusion, excitation and feature partial convolution operations on the spliced ​​feature map to obtain predicted defect information corresponding to the PCB image; Based on the marked defect information and the predicted defect information, a loss value of the PCB defect detection model to be trained is calculated; When the loss value is within a preset range, the trained PCB defect detection model to be trained is used as the final PCB defect detection model for detecting physical defects of the PCB.

2. The method according to claim 1, characterized in that The calling of the adaptive feature fusion module to process the PCB image using the average pooling and maximum pooling strategies to obtain a spliced ​​feature map includes: Calling the adaptive feature fusion module to extract the image features of the PCB image and obtain an input feature map; Performing average pooling processing on the input feature map to obtain a pooled feature map with reduced spatial resolution; The pooled feature map is equally divided to obtain a first feature map and a second feature map; Down-sampling the first feature map using a convolution kernel of a first size to obtain a first sampled feature map; Performing maximum pooling processing on the second feature map, and down-sampling the maximum pooled feature map using a convolution kernel of a second size to obtain a second sampled feature map; The first sampling feature map and the second sampling feature map are spliced ​​at the channel level to obtain the spliced ​​feature map.

3. The method according to claim 1, characterized in that The feature aggregation module includes: SENetV2 network and part of the convolutional network, The calling of the feature aggregation module to perform squeezing, excitation and feature partial convolution operations on the splicing feature map to obtain predicted defect information corresponding to the PCB image includes: Calling the SENetV2 network to perform squeezing and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibrated feature map; Calling the partial convolution network to perform a feature partial convolution operation on the calibration feature map in a mask processing manner to obtain a convolution feature map; The convolution feature map is processed to obtain predicted defect information corresponding to the PCB image.

4. The method according to claim 3, characterized in that The calling of the SENetV2 network to perform squeezing and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibration feature map includes: Calling the SENetV2 network to transform the concatenated feature map to obtain a transformed feature map; Performing average pooling compression processing on the transformed feature map to obtain multiple channel descriptors; The multiple channel descriptors are processed respectively by two fully connected layers to obtain multiple channel weights; The transformed feature map is weighted based on the channel weight to obtain the calibration feature map.

5. The method according to claim 4, characterized in that The multiple channel descriptors are processed respectively by two fully connected layers to obtain multiple channel weights, including: Calling the first fully connected layer and the ReLU activation function to perform dimensionality reduction processing on the multiple channel descriptors to obtain dimensionality reduction descriptors of the multiple channels; The second fully connected layer and the sigmoid activation function are called to increase the dimension of the reduced-dimensional descriptors of the multiple channels to obtain multiple channel weights.

6. A training device for a PCB defect detection model, characterized in that: The device comprises: A model sample acquisition module is used to acquire model training samples, wherein the model training samples include PCB images of different physical defect types, and each of the PCB images is annotated with the annotated defect information of the PCB; A model sample input module, used for inputting the model training sample into a PCB defect detection model to be trained, wherein the PCB defect detection model to be trained includes: an adaptive feature fusion module, a feature aggregation module based on SENetV2 architecture and partial convolution; A splicing feature map acquisition module is used to call the adaptive feature fusion module to process the PCB image using average pooling and maximum pooling strategies to obtain a splicing feature map; A predicted defect information acquisition module is used to call the feature aggregation module to perform extrusion, excitation and feature partial convolution operations on the splicing feature map to obtain predicted defect information corresponding to the PCB image; A loss value calculation module, used for calculating the loss value of the PCB defect detection model to be trained based on the marked defect information and the predicted defect information; The defect detection model acquisition module is used to use the trained PCB defect detection model to be trained as the final PCB defect detection model for detecting physical defects of the PCB when the loss value is within a preset range.

7. The device according to claim 6, characterized in that The splicing feature map acquisition module includes: An input feature map acquisition unit is used to call the adaptive feature fusion module to extract the image features of the PCB image to obtain an input feature map; A pooling feature map acquisition unit, used to perform average pooling processing on the input feature map to obtain a pooling feature map with reduced spatial resolution; A feature map division unit, used for equally dividing the pooled feature map to obtain a first feature map and a second feature map; A first feature map acquisition unit, configured to downsample the first feature map using a convolution kernel of a first size to obtain a first sampled feature map; A second feature map acquisition unit, used to perform maximum pooling processing on the second feature map, and downsample the maximum pooled feature map using a convolution kernel of a second size to obtain a second sampled feature map; A splicing feature map acquisition unit is used to perform channel-level splicing processing on the first sampling feature map and the second sampling feature map to obtain the splicing feature map.

8. The device according to claim 6, characterized in that The feature aggregation module includes: SENetV2 network and part of the convolutional network, The predicted defect information acquisition module includes: A calibration feature map acquisition unit, used for calling the SENetV2 network to perform squeezing and excitation operations on the spliced ​​feature map through a multi-branch fully connected layer to obtain a calibration feature map; A convolution feature map acquisition unit, used for calling the partial convolution network to perform a feature partial convolution operation on the calibration feature map in a mask processing manner to obtain a convolution feature map; The predicted defect information acquisition unit is used to process the convolution feature map to obtain the predicted defect information corresponding to the PCB image.

9. The device according to claim 8, characterized in that The calibration feature map acquisition unit comprises: A transformation feature acquisition subunit is used to call the SENetV2 network to transform the spliced ​​feature map to obtain a transformation feature map; A channel descriptor acquisition subunit is used to perform average pooling compression processing on the transformed feature map to obtain multiple channel descriptors; A channel weight acquisition subunit, used for processing the multiple channel descriptors respectively through two fully connected layers to obtain multiple channel weights; The calibration feature map acquisition subunit is used to perform weighted processing on the transformation feature map based on the channel weight to obtain the calibration feature map.

10. The device according to claim 9, characterized in that The channel weight acquisition subunit comprises: A dimension reduction descriptor acquisition subunit is used to call the first fully connected layer and the ReLU activation function to perform dimension reduction processing on the multiple channel descriptors to obtain dimension reduction descriptors of multiple channels; The multiple channel weight acquisition subunits are used to call the second fully connected layer and the sigmoid activation function to increase the dimension of the reduced-dimensional descriptors of the multiple channels to obtain multiple channel weights.

11. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the training method for the PCB defect detection model according to any one of claims 1 to 5 when executing the program.

12. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method for a PCB defect detection model according to any one of claims 1 to 5.

Images