A PCB image defect assessment method based on multi-scale feature fusion

Abstract

The present invention discloses a PCB image defect assessment method based on multi-scale feature fusion, comprising the following steps: collecting and storing PCB image data, screening and preprocessing the PCB images; performing data enhancement on the pre-processed data, then dividing all images obtained after data enhancement into a training set and a validation set, adding a configuration file to the data set to form a data set; building a PCB image defect assessment model based on multi-scale feature fusion; inputting the training set and label file from the PCB image data set into the PCB image defect assessment model based on multi-scale feature fusion for full training; and completing PCB image defect assessment using the trained assessment model. The present invention fully considers the low accuracy of existing PCB image micro-defect assessments, optimizes and improves the PCB image defect assessment model, and designs the C2f_DLKA, Dysample, and FFM-Head modules to address the low accuracy of existing micro-defect assessments in PCB images.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and relates to PCB image defect assessment technology, and specifically to a PCB image defect assessment method based on multi-scale feature fusion. Background Art

[0002] Driven by the dual forces of global digitalization and intelligent transformation, emerging technologies such as 5G, artificial intelligence, and the Internet of Things are accelerating, reshaping modern industrial systems and social life. As the core hub of electronic products, printed circuit boards (PCBs) face diverse application scenarios, placing stringent demands on PCB image defect assessment technology, making it a key area of ​​urgent technological advancement.

[0003] Breakthroughs in deep learning for object detection have provided a new path to solving the challenge of PCB image defect assessment. However, existing deep learning models generally suffer from large parameter counts, computational complexity, and insufficient ability to extract the features of tiny target defects in PCB images.

[0004] Therefore, it is crucial to study and integrate advanced feature fusion strategy design, break through the performance bottleneck of traditional small defect target evaluation algorithm, enhance the evaluation accuracy of small defect targets, and provide a reusable technical paradigm for industrial quality inspection. Summary of the Invention

[0005] Purpose of the invention: In order to overcome the shortcomings of the existing technology, a PCB image defect assessment method based on multi-scale feature fusion is provided. Taking into full consideration the low accuracy of current PCB image defect assessment based on deep learning, the YOLOv8n model is optimized and improved, and the C2f_DLKA, Dysample, and FFM-Head modules are introduced to solve the problem of low accuracy in the assessment of minor defects in the existing PCB image defect assessment model.

[0006] Technical solution: To achieve the above objectives, the present invention provides a PCB image defect assessment method based on multi-scale feature fusion, comprising the following steps:

[0007] S1: Collect and save the image data of the PCB board, filter and pre-process the image of the PCB board;

[0008] S2: Perform data augmentation on the preprocessed data, then divide all the images obtained after data augmentation into a training set and a validation set, and add a configuration file to the dataset to form a dataset;

[0009] S3: Build a PCB image defect assessment model based on multi-scale feature fusion;

[0010] S4: Input the training set and label file in the PCB image dataset into the PCB board image defect assessment model based on multi-scale feature fusion for full training;

[0011] S5: Complete the evaluation of PCB board image defects through the trained PCB board image defect evaluation model based on multi-scale feature fusion, and output the evaluation results.

[0012] Furthermore, the PCB image defect assessment model based on multi-scale feature fusion in step S3 adopts the improved YOLOv8n, and the specific improvements include:

[0013] The dynamic large kernel convolutional attention mechanism is introduced into the C2f module of the backbone network in YOLOv8n, and the C2f_DLKA module is constructed;

[0014] Replace the normal upsampling in the neck network of YOLOv8n with a dynamic upsampling structure;

[0015] A new PCR composite detection head structure is proposed, which includes a hierarchical interleaved feature fusion module and a multi-layer detection head, and combines fine-grained bounding box optimization and feature enhancement modules.

[0016] Furthermore, the operation of the DLKA module in the C2f_DLKA module includes:

[0017] B1: Perform convolution operations on each channel of the input feature map through depthwise convolution to independently extract features within each channel;

[0018] B2: Use point-by-point convolution to fuse the channel features extracted by depthwise convolution, reducing the amount of computation while achieving feature extraction and inter-channel information integration;

[0019] B3: Based on the depthwise separable convolution, the receptive field of the convolution kernel is expanded through depthwise dilated convolution;

[0020] B4: Adjust the number of channels of the feature map; compress or expand the channels through linear transformation so that the output feature map meets the input requirements of the subsequent network structure, and further integrate the information between channels.

[0021] Furthermore, the operation of dynamic upsampling in the improved YOLOv8n includes:

[0022] C1: Using PyTorch’s built-in functions, the input feature map is converted into a continuous feature map through bilinear interpolation, and then content-aware sampling points are generated.

[0023] C2: Determine the sampling set based on the generated sampling points:

[0024] For the static range factor, the feature map is linearly transformed and the offset is added to the original sampling grid to obtain the sampling set;

[0025] For the dynamic range factor, the point-level dynamic range factor is generated by linearly projecting the input features, and the offset is further adjusted to determine the sampling set;

[0026] C3: After obtaining the sampling set, the grid sampling function is used to perform sampling operations on the input feature map based on the sampling set to implement the upsampling process and output the upsampled feature map.

[0027] Furthermore, the improved PCR composite detection head structure in YOLOv8n is used to integrate multi-scale feature maps and ultimately output the bounding box and evaluation results of the PCB image defect target, specifically including:

[0028] During the initialization phase, the module constructs two core branches based on the number of PCB image defect categories and feature channels. One branch is responsible for bounding box prediction, determining the location and size of PCB image defects through in-depth analysis of feature maps. The other branch focuses on category distribution prediction, determining the specific category to which the target belongs.

[0029] The hierarchical interleaved feature fusion module (FFM) is used to optimize the fusion of input feature maps. At the same time, the distributed focal loss module is used to refine the prediction accuracy of the bounding box.

[0030] During the forward propagation process, the input multi-scale feature map will first undergo FFM fusion processing to fully combine the advantages of features at different scales, and then be sent to the detection head to predict the bounding box and category distribution respectively; in the training phase, the processed feature map is output for parameter adjustment and optimization of the model; in the inference phase, the final evaluation prediction result is output.

[0031] Furthermore, the improved YOLOv8n mid-level interleaved feature fusion module FFM achieves deep fusion of multi-scale feature maps through upsampling and downsampling operations. The specific operation includes:

[0032] D1: The input high-resolution feature map is upsampled to the corresponding scale; the low-resolution feature map is downsampled to the corresponding scale;

[0033] D2: The high-resolution feature map, the low-resolution feature map, and the medium-resolution feature map are spliced ​​together. The spliced ​​feature map is then processed by the convolution layer to further extract the fused feature information and generate a fused feature map.

[0034] D3: The fused feature map is decomposed through the activation function;

[0035] D4: Combined with the PCRC module for processing, the PCRC module optimizes the fused features from different angles and further generates three sub-feature maps, representing features of different resolutions;

[0036] D5: The sub-feature map undergoes upsampling or downsampling operations and adjusts its scale according to actual needs to form an enhanced multi-scale feature map.

[0037] Furthermore, the PCRC module in step D4 focuses on feature optimization and adopts two parallel processing paths. One path (MCRC), namely the maximum pooling and convolution path, highlights the local maximum in the feature map through the maximum pooling operation and captures the most significant feature information. Then, through the convolution operation, these features are further extracted and refined to enhance their discriminability. The other path (ACRC), namely the average pooling and convolution path, uses the average pooling operation to smooth the feature map and extract the overall statistical features. The convolution operation is also used for feature refinement.

[0038] The outputs of these two paths represent feature information from different angles respectively. The outputs of the two paths are adjusted to the same scale through upsampling and then added together, so that different types of features complement each other and generate optimized feature maps, thereby further enhancing the feature expression ability and providing better feature input for detection tasks.

[0039] Furthermore, the step S4 specifically includes:

[0040] E1: Use the improved network model for training. After the input image enters the backbone network, downsampling and feature extraction are performed to produce feature maps of different scales.

[0041] E2: The features extracted by the backbone network are upsampled, fused, and processed through the neck network to generate detection results at different scales;

[0042] E3: The feature map output by the neck network is input into the detection layer to predict the category, bounding box and confidence of the PCB image defect, and finally complete the defect assessment task.

[0043] Furthermore, the step E1 specifically includes:

[0044] Initial downsampling: Use a 3x3 convolution kernel with a stride of 2 to reduce the input image size from [640, 640, 3] to [320, 320, 64] and [160, 160, 128] respectively;

[0045] Feature extraction and downsampling: Features are extracted through the C2f_DLKA module, and subsequent convolutional layers continue to downsample with a stride of ×2, reducing the image size to [80,80,256], [40,40,512], and [20,20,1024];

[0046] Feature enhancement: The SPPF module is used to enhance the feature expression of the last layer without changing the size.

[0047] Furthermore, the step E2 specifically includes:

[0048] Upsampling and feature fusion: Upsampling twice and concatenating with the corresponding layer features in the backbone network respectively, while keeping the feature map size unchanged;

[0049] Downsampling and feature fusion: Downsample twice and concatenate with the corresponding features to obtain feature maps with sizes of [80, 80, 256] [40, 40, 512] and [20, 20, 1024].

[0050] Beneficial effects: Compared with the existing technology, the present invention fully considers the low accuracy of existing PCB image defect assessment based on deep learning, optimizes and improves the YOLOv8n model, and introduces the C2f_DLKA, Dysample, and FFM-Head modules, which solves the problem of low accuracy in the assessment of minor defects in the existing PCB image defect assessment model and greatly improves the assessment effect of minor defects on PCB boards. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the overall process of the method of the present invention;

[0052] Figure 2 This is a network structure diagram of the PCB board image defect assessment model based on multi-scale feature fusion in the present invention;

[0053] Figure 3 This is the structural diagram of the dynamic large kernel convolutional attention mechanism (DLKA) in this invention;

[0054] Figure 4 It is the structure and calculation flow chart of the dynamic sampler (Dysample) in the present invention;

[0055] Figure 5 It is a structural diagram of the PCR composite detection head (FFM-Head) of the present invention;

[0056] Figure 6 This is a comparison chart of the defect assessment results of the present invention and the original YOLOv8 model in the PCB dataset. DETAILED DESCRIPTION

[0057] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0058] Example 1:

[0059] like Figure 1 As shown, this embodiment provides a PCB image defect assessment method based on multi-scale feature fusion, including the following steps:

[0060] S1: Collect and save the image data of the PCB board, filter and pre-process the image of the PCB board;

[0061] S2: Perform data augmentation on the preprocessed data, then divide all the images obtained after data augmentation into a training set and a validation set. Add a configuration file to the dataset to form a dataset that can be used for PCB image defect assessment.

[0062] S3: Build a PCB image defect assessment model based on multi-scale feature fusion;

[0063] S4: Input the training set and label file in the PCB image dataset into the PCB board image defect assessment model based on multi-scale feature fusion for full training;

[0064] S5: Complete the evaluation of PCB board image defects through the trained PCB board image defect evaluation model based on multi-scale feature fusion, and output the evaluation results.

[0065] In step S1 of this embodiment:

[0066] Collect image data of PCB defects, including six categories: mouse bite, hole, burr, open circuit, short circuit, fake copper, etc.

[0067] Use the LabelImg annotation tool to annotate the defects in the image and retain the annotation information to obtain the PCB defect dataset and its corresponding annotation file.

[0068] Step S2 of this embodiment specifically includes:

[0069] A1: We use RoboFlow to perform data augmentation, rotating, cropping, blurring, and adjusting brightness to increase the diversity of the dataset and improve the generalization ability of the model.

[0070] A2: Divide the dataset into training and validation sets in a ratio of 8:2;

[0071] A3: Add a configuration file to the dataset, specify the number of dataset categories, training set path, validation set path, and other information to form a complete PCB defect dataset that can be used for algorithm training.

[0072] In step S3 of this embodiment, the PCB board image defect assessment model based on multi-scale feature fusion adopts the improved YOLOv8n, referring to Figure 2 Specific improvements include:

[0073] The structure of the PCB image defect assessment model based on multi-scale feature fusion is different from the original YOLOv8 model in the following ways:

[0074] In the backbone network of the model, in order to enhance its feature extraction capability, a DLKA attention mechanism is proposed. The DLKA attention mechanism is introduced into the C2f module of the backbone network of YOLOv8n to construct the C2f_DLKA module.

[0075] In the neck network of YOLOv8n, the existing upsampling method is prone to losing feature information in the process of restoring resolution, which affects the detection accuracy. Therefore, the Dysample dynamic upsampler is introduced to preserve feature information while improving upsampling efficiency.

[0076] A new PCR composite detection head structure is proposed, which includes a hierarchical interleaved feature fusion module and a multi-layer detection head, and combines fine-grained bounding box optimization and feature enhancement modules.

[0077] like Figure 3 As shown, the operation of the DLKA module in the C2f_DLKA module includes:

[0078] B1: Perform convolution operations on each channel of the input feature map through depthwise convolution to independently extract features within each channel;

[0079] B2: Use point-by-point convolution (1×1 convolution) to fuse the channel features extracted by depthwise convolution, reducing the amount of computation while achieving feature extraction and inter-channel information integration;

[0080] B3: Based on depthwise separable convolution, the receptive field of the convolution kernel is expanded through depthwise dilated convolution. Through special convolution kernel settings, the convolution operation can capture contextual information in a wider area, allowing the model to have a more comprehensive and in-depth understanding of the image content. The dilated convolution method is used, that is, the convolution kernel has interval sampling during the convolution process. This increases the receptive field of the convolution kernel without increasing the number of parameters, thereby obtaining richer global feature information.

[0081] The special features of the convolution kernel here are:

[0082] First, the structural design of the convolution kernel introduces holes;

[0083] 2. Sampling method of convolution kernel (interval sampling).

[0084] B4: 1×1 convolution is mainly used to adjust the number of channels of the feature map; through linear transformation, the channels are compressed or expanded so that the output feature map meets the input requirements of the subsequent network structure, while further fusing the information between channels.

[0085] like Figure 4 The figure shows the structure of the dynamic sampler and its calculation process for both static and dynamic range factor versions. The sample set is generated by the sampling point generator, and the input features are resampled by the grid sampling function. The upper part of the figure shows the dynamic sampler based on dynamic upsampling, namely the sampling point generator. The sample set is the sum of the generated offset and the original grid position. The middle part of the figure shows the version using the "static range factor", in which the offset is generated by the linear layer. The lower part of the figure describes the implementation using the "dynamic range factor", which first generates a range factor and then uses it to adjust the offset. "o" represents the sigmoid function. In dynamic upsampling, the static range factor and the dynamic range factor are calculated simultaneously, and their results are finally combined to form the subsequent grid sampling and other operations to complete the upsampling process.

[0086] The improved dynamic sampler (Dysample) in YOLOv8n performs dynamic upsampling operations including:

[0087] C1: The feature map first enters the sampling point generator. Here, content-sensitive sampling points are generated based on the content of the feature map. This process uses PyTorch's built-in functions to convert the input feature map into a continuous feature map through bilinear interpolation, and then generate content-aware sampling points.

[0088] C2: Determine the sampling set based on the generated sampling points:

[0089] For the static range factor, the feature map is linearly transformed, and the offset is added to the original sampling grid to obtain the sampling set through operations such as pixel reorganization;

[0090] For the dynamic range factor, the point-level dynamic range factor is generated by linearly projecting the input features, the offset is further adjusted, and the sampling set is determined through operations such as pixel reassembly. Through these operations, the offset range of the sampling position is constrained to avoid the overlap of local sampling position offsets affecting boundary prediction.

[0091] C3: After obtaining the sampling set, the grid sampling function is used to upsample the input feature map based on the sampling set. This process results in an upsampled feature map of size sH × sW × C (where s is the upsampling scale factor, H and W are the height and width of the original feature map, and C is the number of channels). The upsampled feature map retains more detailed information and better processes the details and semantic information of the features. It is then used for subsequent operations such as splicing and fusion with other feature maps.

[0092] like Figure 5 As shown in the figure, the improved PCR composite detection head structure (FFM-Head) in YOLOv8n is used to integrate multi-scale feature maps and ultimately output the bounding box and evaluation results of the PCB image defect target, including:

[0093] During the initialization phase, the module constructs two crucial core branches based on the number of target categories and feature channels. One branch is responsible for bounding box prediction, accurately determining the location and size of the target in the image through in-depth analysis of the feature map. The other branch focuses on class distribution prediction, determining the specific category to which the target belongs.

[0094] The hierarchical interleaved feature fusion module is used to optimize the fusion of input feature maps. At the same time, the Distribution Focal Loss (DFL) module is used to refine the prediction accuracy of the bounding box.

[0095] During the forward propagation process, the input multi-scale feature map will first undergo FFM fusion processing to fully combine the advantages of features at different scales. It will then be sent to the detection head to predict the bounding box and category distribution respectively. During the training phase, the processed feature map is output for parameter adjustment and optimization of the model. During the inference phase, the final prediction result is output to provide accurate defect assessment information for practical applications.

[0096] The improved YOLOv8n mid-level interleaved feature fusion module (FFM) achieves deep fusion of multi-scale feature maps through carefully designed upsampling and downsampling operations, thereby significantly enhancing the feature representation capability. Specific operations include:

[0097] D1: The input high-resolution feature map is raised to the corresponding scale through upsampling methods such as interpolation; the low-resolution feature map is reduced to the corresponding scale through downsampling operations such as convolution;

[0098] D2: The high-resolution feature map, the low-resolution feature map, and the medium-resolution feature map are spliced ​​together. The spliced ​​feature map is then processed by the convolution layer to further extract the fused feature information and generate a fused feature map.

[0099] D3: The fused feature map is decomposed through the activation function, which can highlight the key information in the feature and suppress unnecessary noise;

[0100] D4: Combined with the PCRC (Pooling and Convolutional Refinement Component) module for processing, the PCRC module optimizes the fused features from different angles and further generates three sub-feature maps, representing features of different resolutions;

[0101] The PCRC module focuses on feature optimization and uses two parallel processing paths. One path (MCRC), namely the maximum pooling and convolution path, uses the maximum pooling operation to highlight the local maximum in the feature map and capture the most significant feature information. Then, through the convolution operation, these features are further extracted and refined to enhance their discriminability. The other path (ACRC), namely the average pooling and convolution path, uses the average pooling operation to smooth the feature map and extract the overall statistical features. It also uses the convolution operation to refine the features.

[0102] The outputs of these two paths represent feature information from different angles. By upsampling the outputs of the two paths to the same scale and then adding them together, different types of features complement each other and generate an optimized feature map, thereby further enhancing the feature expression capability and providing higher-quality feature input for detection tasks.

[0103] This process also involves bias initialization, specifically setting the biases of the regression and classification branches to help the model converge faster. During inference, anchors and strides are dynamically generated, and relevant tool functions and auxiliary modules are used to decode the predicted bounding boxes and restore them to the original image scale. This allows the object's position and category to be accurately determined in the original image, completing the object detection task.

[0104] D5: The sub-feature map undergoes upsampling or downsampling operations and adjusts its scale according to actual needs, ultimately forming an enhanced multi-scale feature map, providing richer and more representative feature information for subsequent detection tasks.

[0105] Step S4 of this embodiment specifically includes:

[0106] E1: Use the improved network model for training. After the input image enters the backbone network, it passes through the Conv, C2f_DLKA and other modules for downsampling and feature extraction, and produces feature maps of different scales;

[0107] E2: The features extracted by the backbone network are upsampled, fused, and processed through the neck network to generate detection results at different scales;

[0108] E3: The feature map output by the neck network is input into the detection layer to predict the category, bounding box and confidence of the PCB image defect, and finally complete the defect assessment task.

[0109] In this embodiment, step E1 specifically includes:

[0110] (1) Initial convolution layer: The first convolution layer uses a 3x3 convolution kernel with a stride of 2 and 64 output channels. The image height and width are reduced to half, and the output feature map size becomes [320, 320, 64], denoted as P1 / 2. The second convolution layer uses a 3x3 convolution kernel again on the P1 / 2 feature map with a stride of 2 and 128 output channels. The image size is further reduced to a quarter of the original size, and the output feature map size becomes [160, 160, 128], denoted as P2 / 4.

[0111] (2) Features are extracted through three C2f_DLKA modules, with 128 input and output channels. This module extracts rich features through multiple convolutional layers and residual connections. The image size remains unchanged at this stage [160, 160, 128]. Subsequent convolutional layers continue to downsample with a stride of ×2, reducing the image size to [80, 80, 256], [40, 40, 512], and [20, 20, 1024].

[0112] (3) Feature enhancement: The SPPF module is used to process the output of the last layer, and the feature expression capability is enhanced through multiple maximum pooling layers and splicing operations. The image size is maintained at [20, 20, 1024].

[0113] In this embodiment, step E2 specifically includes:

[0114] (1) Upsampling and feature fusion: The first upsampling operation is to upsample the output of the last layer of the backbone network [20, 20, 1024], and the image size is doubled to [40, 40, 512]. It is then concatenated with the P4 feature map in the backbone network in the channel dimension to obtain the [40, 40, 512] feature map. The upsampling operation is repeated again, and the image size is doubled to [80, 80, 256]. It is then concatenated with the P3 feature map in the backbone network to obtain the P3 / 8 feature map.

[0115] (2) Downsampling and feature fusion: Downsample the P3 / 8 feature map, reduce the image size to half of the original size, to [40, 40, 512], and obtain the P4 / 16 feature map; repeat the downsampling operation, reduce the image size to half of the original size, to [20, 20, 1024], and obtain the P5 / 32 feature map;

[0116] In this embodiment, step E3 specifically includes:

[0117] The three feature maps of P3 / 8 [80, 80, 256], P4 / 16 [40, 40, 512] and P5 / 32 [20, 20, 1024] are input into the detection head. This layer predicts the PCB defect category, bounding box and confidence to complete the PCB defect assessment task.

[0118] In this embodiment, step S5 specifically includes:

[0119] The PCB images are respectively input into the trained PCB board image defect assessment model based on multi-scale feature fusion for defect assessment;

[0120] Defect assessment is performed in the PCB board image defect assessment model based on multi-scale feature fusion, and the location, category and number of defect targets in the PCB image are output.

[0121] Example 2:

[0122] In order to verify the actual effect of the method of the present invention, this embodiment uses the PCB data set to conduct a comparative experiment and obtain Figure 6 The PCB defect assessment results comparison chart shown is analyzed as follows:

[0123] Figure 6 The labels mh, sr, mb, and ot represent the four types of PCB defects: holes, burrs, rat bites, and open circuits. The values ​​behind the labels represent the evaluation accuracy (accuracy ranges from 0 to 1, 1 represents the highest accuracy, and 0 represents the lowest accuracy). For example, mh 0.88 represents a hole defect, and the evaluation accuracy is 0.88; Figure 6 It can be seen that the original YOLOv8 model not only has missed detections, but also has false detections, and its evaluation accuracy is low. The method of the present invention not only does not have these two problems, but also has better defect assessment accuracy than the original model.

Claims

1. A PCB image defect assessment method based on multi-scale feature fusion, characterized in that: The steps include: S1: Collect and save the image data of the PCB board, filter and pre-process the image of the PCB board; S2: Perform data augmentation on the preprocessed data, then divide all the images obtained after data augmentation into a training set and a validation set, and add a configuration file to the dataset to form a dataset; S3: Build a PCB image defect assessment model based on multi-scale feature fusion; S4: Input the training set and label file in the PCB image dataset into the PCB board image defect assessment model based on multi-scale feature fusion for full training; S5: Complete the PCB image defect assessment using the trained multi-scale feature fusion-based PCB image defect assessment model and output the assessment results. In step S3, the PCB image defect assessment model based on multi-scale feature fusion uses the improved YOLOv8n. The specific improvements include: The dynamic large kernel convolutional attention mechanism is introduced into the C2f module of the backbone network in YOLOv8n, and the C2f_DLKA module is constructed; Replace the normal upsampling in the neck network of YOLOv8n with a dynamic upsampling structure; A PCR composite detection head structure is proposed, which includes a hierarchical interleaved feature fusion module and a multi-layer detection head, and combines fine-grained bounding box optimization and feature enhancement modules. The operation of the DLKA module in the C2f_DLKA module includes: B1: Perform convolution operations on each channel of the input feature map through depthwise convolution to independently extract features within each channel; B2: Use point-by-point convolution to fuse the channel features extracted by depthwise convolution, reducing the amount of computation while achieving feature extraction and inter-channel information integration; B3: Based on the depthwise separable convolution, the receptive field of the convolution kernel is expanded through depthwise dilated convolution; B4: Adjust the number of channels of the feature map; compress or expand the channels through linear transformation so that the output feature map meets the input requirements of the subsequent network structure, and further integrate the information between channels; The improved dynamic upsampling operation in YOLOv8n includes: C1: Using PyTorch’s built-in functions, the input feature map is converted into a continuous feature map through bilinear interpolation, and then content-aware sampling points are generated. C2: Determine the sampling set based on the generated sampling points: For the static range factor, the feature map is linearly transformed and the offset is added to the original sampling grid to obtain the sampling set; For the dynamic range factor, the point-level dynamic range factor is generated by linearly projecting the input features, and the offset is further adjusted to determine the sampling set; C3: After obtaining the sampling set, the grid sampling function is used to sample the input feature map based on the sampling set to implement the upsampling process and output the upsampled feature map; The improved PCR composite detection head structure in YOLOv8n is used to integrate multi-scale feature maps and ultimately output the bounding box and evaluation results of PCB image defects, including: During the initialization phase, the module constructs two core branches based on the number of PCB image defect categories and feature channels. One branch is responsible for bounding box prediction, determining the location and size of PCB image defects through in-depth analysis of feature maps. The other branch focuses on category distribution prediction, determining the specific category to which the target belongs. The hierarchical interleaved feature fusion module (FFM) is used to optimize the fusion of input feature maps. At the same time, the distributed focal loss module is used to refine the prediction accuracy of the bounding box. During the forward propagation process, the input multi-scale feature map will first undergo FFM fusion processing to fully combine the advantages of features at different scales, and then be sent to the detection head to predict the bounding box and category distribution respectively; in the training phase, the processed feature map is output for parameter adjustment and optimization of the model; in the inference phase, the final evaluation prediction result is output.

2. The PCB image defect assessment method based on multi-scale feature fusion according to claim 1, characterized in that: The improved YOLOv8n mid-level interleaved feature fusion module FFM achieves deep fusion of multi-scale feature maps through upsampling and downsampling operations. The specific operation includes: D1: The input high-resolution feature map is upsampled to the corresponding scale; the low-resolution feature map is downsampled to the corresponding scale; D2: The high-resolution feature map, the low-resolution feature map, and the medium-resolution feature map are spliced ​​together. The spliced ​​feature map is then processed by the convolution layer to further extract the fused feature information and generate a fused feature map. D3: The fused feature map is decomposed through the activation function; D4: Combined with the PCRC module for processing, the PCRC module optimizes the fused features from different angles and further generates three sub-feature maps, representing features of different resolutions; D5: The sub-feature map undergoes upsampling or downsampling operations and adjusts its scale according to actual needs to form an enhanced multi-scale feature map.

3. The PCB image defect assessment method based on multi-scale feature fusion according to claim 2, characterized in that: In step D4, the PCRC module uses two parallel processing paths; one path, namely the maximum pooling and convolution path, highlights the local maximum in the feature map through the maximum pooling operation, captures the most significant feature information, and then further extracts and refines these features through the convolution operation to enhance their discriminability; the other path, namely the average pooling and convolution path, uses the average pooling operation to smooth the feature map and extract the overall statistical features, and also performs feature refinement through the convolution operation; The outputs of the two paths are adjusted to the same scale through upsampling and then added to generate the optimized feature map.

4. The PCB image defect assessment method based on multi-scale feature fusion according to claim 1, characterized in that: The step S4 specifically includes: E1: Use the improved network model for training. After the input image enters the backbone network, downsampling and feature extraction are performed to produce feature maps of different scales. E2: The features extracted by the backbone network are upsampled, fused, and processed through the neck network to generate detection results at different scales; E3: The feature map output by the neck network is input into the detection layer to predict the category, bounding box and confidence of the PCB image defect, and finally complete the defect assessment task.

5. The PCB image defect assessment method based on multi-scale feature fusion according to claim 4 is characterized in that: The step E1 specifically includes: Initial downsampling: Use a 3x3 convolution kernel with a stride of 2 to reduce the input image size from [640, 640, 3] to [320, 320, 64] and [160, 160, 128] respectively; Feature extraction and downsampling: Features are extracted through the C2f_DLKA module, and subsequent convolutional layers continue to downsample with a stride of ×2, reducing the image size to [80,80,256], [40,40,512], and [20,20,1024]; Feature enhancement: The SPPF module is used to enhance the feature expression of the last layer without changing the size.

6. The PCB image defect assessment method based on multi-scale feature fusion according to claim 5, characterized in that: The step E2 specifically includes: Upsampling and feature fusion: Upsampling twice and concatenating with the corresponding layer features in the backbone network respectively, while keeping the feature map size unchanged; Downsampling and feature fusion: Downsample twice and concatenate with the corresponding features to obtain feature maps with sizes of [80, 80, 256] [40, 40, 512] and [20, 20, 1024].

Images