Lightweight transformer defect detection method and device for high-density PCBs

Abstract

The application provides a lightweight Transformer defect detection method and device for high-density PCBs, wherein the method comprises data preprocessing on a high-density PCB image, deep separable convolution on the preprocessed image to obtain multiple local features, dynamic sparse attention calculation on the multiple local features to obtain the features of key regions, fusion of the features of different key regions by using a MobileViT lightweight convolution block, defect classification and positioning prediction on the fused features to obtain a PCB defect detection result. The deep separable convolution replaces the traditional convolution operation, the dynamic sparse attention mechanism is introduced, and the mask matrix is dynamically generated in the feature importance evaluation mode, so that the feature correlation modeling of the key regions can be realized, the redundant calculation of the non-key regions is reduced, the calculation amount is reduced, and most of the attention is allocated to the micro defects. The above PCB defect detection method reduces the calculation complexity and the model parameter amount while ensuring high detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of PCB detection, in particular to a lightweight Transformer defect detection method and device for high-density PCBs. BACKGROUND

[0002] With the increasing requirements of electronic products on performance and size, the size and component density of printed circuit boards (PCB) are continuously increasing, leading to small defects on high-density PCBs, such as burrs, excess copper, etc., which have a serious impact on the performance and reliability of the circuit. Traditional PCB defect detection methods mostly rely on models such as deep convolutional neural networks (CNN), but due to the small size of defects in high-density PCBs and the complexity of the background of defects, traditional methods often have difficulty in efficiently and accurately detecting small defects.

[0003] Especially in industrial applications, the detection system is required to run under low power consumption and high speed conditions to meet the needs of real-time detection and embedded device deployment. Although existing defect detection methods based on Transformer have made breakthroughs in performance, they are difficult to apply to industrial-grade real-time detection and embedded devices due to their high computational complexity and large number of model parameters.

[0004] Therefore, there is a need for a lightweight Transformer defect detection method for high-density PCBs that can meet the needs of small defect detection on high-density PCBs, not only has the characteristics of lightweight and low power consumption, but also has real-time inference capability and can be used for industrial-grade embedded devices. SUMMARY

[0005] To overcome the problems in the related art, the purpose of the present application is to provide a lightweight Transformer defect detection method and device for high-density PCBs, which can meet the needs of small defect detection on high-density PCBs, not only has the characteristics of lightweight and low power consumption, but also has real-time inference capability and can be used for industrial-grade embedded devices.

[0006] A lightweight Transformer defect detection method for high-density PCBs, comprising:

[0007] obtaining a high-density PCB image;

[0008] performing data preprocessing on the high-density PCB image to obtain a preprocessed image, the data preprocessing including normalization and size adjustment;

[0009] performing depth separable convolution on the preprocessed image to obtain a plurality of local features;

[0010] performing dynamic sparse attention calculation on the plurality of local features to obtain features of key regions;

[0011] fusing features of different key regions by using a MobileViT lightweight convolution block to obtain fused features;

[0012] performing defect classification and positioning prediction on the fused features to obtain a PCB defect detection result; wherein the PCB defect detection result includes a defect type and a defect position of the high-density PCB image.

[0013] In the preferred technical solution of the present application, the depth separable convolution performed on the preprocessed image to obtain a plurality of local features comprises:

[0014] performing channel-by-channel convolution on the preprocessed image to obtain k first feature maps; wherein k is the same as the number of channels of the preprocessed image, and k≥2;

[0015] grouping the k first feature maps into an input feature map sequence, sequentially inputting the input feature map sequence into q point-by-point convolution kernels, and performing q times of point-by-point convolution to obtain q local features, q≥2.

[0016] In the preferred technical solution of the present application, the dynamic sparse attention calculation performed on the plurality of local features to obtain features of key regions comprises:

[0017] The features of the key regions are extracted by using the following formula:

[0018] ;

[0019] wherein SparseAttn is dynamic sparse attention, Q is a query matrix, K is a key matrix, and V is a value matrix; Softmax is a normalized exponential function, K T is a transposed matrix of the key matrix, is an element-by-element multiplication operation, M is a mask matrix, the mask matrix is generated by using a dynamic screening method based on feature response or a multi-scale feature extraction method, d length is the vector length of the key matrix.

[0020] In the preferred technical solution of the present application, the MobileViT lightweight convolution block is used to fuse features of different key regions to obtain fused features, which comprises:

[0021] performing convolution operation on different key regions to extract local features of different key regions;

[0022] The global features of all key regions are extracted by using a Transformer model.

[0023] The local features and the global features are spliced to obtain fused features.

[0024] In the preferred technical solution of the present application, the channel-by-channel convolution of the preprocessed image to obtain k first feature maps comprises:

[0025] The preprocessed image is subjected to channel-by-channel convolution by using the following formula:

[0026] ;

[0027] Wherein, X is the preprocessed image, W std is a channel-by-channel convolution kernel, Y i,j,c is the value of the i-th row and the j-th column of the c-th first feature map, i is the row number of the first feature map, j is the column number of the first feature map, row is the total number of rows of the first feature map, col is the total number of columns of the first feature map, c is the channel index, k is the total number of channels, m is the row adjustment amount, n is the column adjustment amount, is a multiplication operation.

[0028] In the preferred technical solution of the present application, before the features of different key regions are fused by using the MobileViT lightweight convolution block, it further comprises:

[0029] Pruning the MobileViT lightweight convolution block to remove redundant network connections;

[0030] Converting the weights of the MobileViT lightweight convolution block from floating-point numbers to fixed-point numbers.

[0031] In the preferred technical solution of the present application, after the MobileViT lightweight convolution block is pruned, it further comprises:

[0032] The model parameter amount after pruning is calculated according to the following formula:

[0033] ;

[0034] Wherein, P pruned is the model parameter amount after pruning, P original is the original parameter amount, r is the pruning rate, is a multiplication operation.

[0035] In the preferred technical solution of the present application, the global features of all key regions are extracted by using a Transformer model, which comprises:

[0036] The output features after nonlinear transformation are calculated by using the following formula:

[0037] ;

[0038] wherein LN is layer normalization, SparseAttn is dynamic sparse attention, MLP is multi-layer perceptron, F out is the output feature after nonlinear transformation, F in is the matrix of key regions.

[0039] All the output features after nonlinear transformation are spliced to obtain global features of all key regions.

[0040] In the preferred technical solution of the present application, the fusion feature is classified and positioned to obtain a PCB defect detection result, which comprises:

[0041] The fusion feature is input into a YOLOv7 model for defect detection to generate a plurality of detection frames; each detection frame corresponds to a defect type and a rectangular starting point coordinate of the high-density PCB image.

[0042] A lightweight Transformer defect detection device for high-density PCBs comprises:

[0043] An image input module is configured to acquire a high-density PCB image.

[0044] A data preprocessing module is configured to perform data preprocessing on the high-density PCB image to obtain a preprocessed image, wherein the data preprocessing comprises normalization and size adjustment.

[0045] A depth separable convolution module is configured to perform depth separable convolution on the preprocessed image to obtain a plurality of local features.

[0046] A dynamic sparse attention calculation module is configured to perform dynamic sparse attention calculation on the plurality of local features to obtain features of key regions.

[0047] A feature fusion module is configured to fuse features of different key regions by using a MobileViT lightweight convolution block to obtain a fusion feature.

[0048] A PCB defect detection module is configured to classify and position defects of the fusion feature to obtain a PCB defect detection result, wherein the PCB defect detection result comprises a defect type and a defect position of the high-density PCB image.

[0049] The present application has the following advantages:

[0050] The lightweight Transformer defect detection method for high-density PCBs provided by this invention includes: acquiring a high-density PCB image; performing data preprocessing on the high-density PCB image to obtain a preprocessed image, the data preprocessing including normalization and resizing; performing depthwise separable convolution on the preprocessed image to obtain multiple local features; performing dynamic sparse attention calculation on the multiple local features to obtain features of key regions; fusing the features of different key regions using MobileViT lightweight convolutional blocks to obtain fused features; and performing defect classification and localization prediction on the fused features to obtain PCB defect detection results. The PCB defect detection results include the defect type and defect location of the high-density PCB image. By performing normalization and resizing preprocessing on the high-density PCB image, the accuracy of subsequent PCB defect detection is significantly improved. Using depthwise separable convolution instead of traditional convolution operations, decomposing depthwise separable convolution into two steps—channel-wise convolution and pointwise convolution—significantly reduces the number of parameters and computational complexity. Building upon this foundation, a dynamic sparse attention mechanism is introduced. A mask matrix is ​​dynamically generated using feature importance evaluation. This mask matrix covers only a portion of all local features, with each element being either 0 or 1. If an element is 1, the feature at that location is retained; if it is 0, the feature at that location is removed. This enables feature association modeling of critical regions while reducing redundant computation in non-critical regions, thus allocating most attention to minor defects while minimizing computational load. Finally, MobileViT lightweight convolutional blocks are used to prune and quantize the model, further optimizing its speed and data storage capacity. This PCB defect detection method achieves high detection accuracy while reducing computational complexity and model parameter count, making it suitable for industrial applications and meeting the demands for high-precision and high-efficiency PCB defect detection. Attached Figure Description

[0051] Figure 1 This is a flowchart of the lightweight Transformer defect detection method for high-density PCBs of the present invention;

[0052] Figure 2 This is a flowchart of the present invention, which uses MobileViT lightweight convolutional blocks to fuse features of different key regions;

[0053] Figure 3 This is a diagram showing the PCB defect detection results of the present invention. Detailed Implementation

[0054] Preferred embodiments of the present application will be described in greater detail below, with reference to the accompanying drawings. While preferred embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms without being limited by the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and fully convey the scope of the application to those skilled in the art.

[0055] Embodiment 1

[0056] As Figure 1 shown, the embodiment provides a lightweight Transformer defect detection method for high-density PCB, including:

[0057] S1: obtaining a high-density PCB image;

[0058] S2: performing data preprocessing on the high-density PCB image to obtain a preprocessed image, the data preprocessing including normalization and size adjustment;

[0059] S3: performing deep separable convolution on the preprocessed image to obtain a plurality of local features;

[0060] S4: performing dynamic sparse attention calculation on the plurality of local features to obtain features of key regions;

[0061] S5: fusing features of different key regions using MobileViT lightweight convolution block to obtain fused features;

[0062] S6: performing defect classification and positioning prediction on the fused features to obtain a PCB defect detection result; wherein the PCB defect detection result includes a defect type and a defect position of the high-density PCB image.

[0063] High-density PCB (High-Density Printed Circuit Board) refers to a printed circuit board that realizes extremely high wiring density and component integration by micro-wiring, micro-hole diameter, thin dielectric layer, multi-layer stacking, etc. on a limited board surface. High-density PCB image refers to two-dimensional visual data with extremely compact arrangement, tiny spacing, and extremely rich details of features such as components, traces, and pads on the high-density PCB.

[0064] Normalization processing and size adjustment of high-density PCB images are key steps to improve the accuracy of high-density PCB defect detection. The normalization processing includes image input normalization and image input normalization. Before obtaining the high-density PCB image, the normalization processing usually includes geometric normalization and color / grey level normalization. After geometric correction, the effects of image translation, rotation or scaling are eliminated. After color / grey level normalization, the color or grey level distribution of the image can be standardized. After obtaining the high-density PCB image, the normalization processing usually includes pixel value normalization and channel-by-channel normalization. Pixel value normalization refers to uniform numerical range, which scales the pixel value to the range suitable for model adaptation. Channel-by-channel normalization calculates the mean and variance of the RGB channel respectively to avoid color distortion.

[0065] The size adjustment processing of high-density PCB images includes the following key steps. First, the image resolution is unified to ensure that all images have the same DPI (dots per inch), avoiding measurement errors caused by resolution differences. Then, image scaling is performed to scale the image to the target size to meet the image input requirements. Then, the physical size is aligned to accurately correspond the image pixel size to the actual physical size of the high-density PCB. Finally, the image after physical size alignment is scaled again. For high-density PCB edges or large-size boards that may have lens distortion, lens correction is performed before scaling to avoid losing high-frequency signals during scaling.

[0066] The pre-processed image is subjected to deep separable convolution to obtain a plurality of local features, including:

[0067] The pre-processed image is subjected to channel-by-channel convolution to obtain k first feature maps; wherein k is the same as the number of channels of the pre-processed image, and k≥2;

[0068] The k first feature maps are combined to form an input feature map sequence, and the input feature map sequence is sequentially input into q point-by-point convolution kernels for q times of point-by-point convolution to obtain q local features, q≥2.

[0069] The pre-processed image is subjected to channel-by-channel convolution to obtain k first feature maps, including:

[0070] The pre-processed image is subjected to channel-by-channel convolution using the following formula:

[0071] ;

[0072] wherein X is the pre-processed image, W std is a channel-by-channel convolution kernel, Y i,j,cis the value of the i-th row and the j-th column of the c-th first feature map, i is the row index of the first feature map, j is the column index of the first feature map, row is the total number of rows of the first feature map, col is the total number of columns of the first feature map, c is a channel index, k is the total number of channels, m is a row adjustment amount, and n is a column adjustment amount, is a multiplication operation.

[0073] The depth separable convolution is a kind of efficient convolution operation, including two steps of channel-wise convolution and point-wise convolution, which can greatly reduce the calculation amount and parameter amount while maintaining good feature extraction capability. The channel-wise convolution is the first stage of the depth separable convolution, and the core is that each input channel is independently convolved without cross-channel mixing. First, each channel of the pre-processed input image is independently convolved with one convolution kernel, and then a first feature map is output, wherein the number of the first feature map is the same as the number of channels of the pre-processed input image, that is, when the number of channels of the pre-processed input image is k, the number of the first feature map output after the channel-wise convolution is also k.

[0074] The point-wise convolution is the second stage of the depth separable convolution, and the role is to recombine the features across channels without expanding the receptive field. First, the k first feature maps are input into the first point-wise convolution kernel, then the k first feature maps are input into the second point-wise convolution kernel, and the above process is repeated until the k first feature maps are input into the q-th point-wise convolution kernel. Each point-wise convolution kernel performs linear weighted summation on all channels at the corresponding spatial position, and then outputs q local features. The point-wise convolution uses a 1x1 kernel to cross-channel linearly recombine, which can arbitrarily increase or decrease the number of channels without increasing the calculation amount, and does not expand the receptive field, thereby greatly compressing the model while maintaining the accuracy.

[0075] MobileViT is a lightweight and versatile visual Transformer architecture, aiming to combine the spatial inductive bias of CNN with the global modeling capability of ViT, to achieve better performance than traditional lightweight CNN and ViT under the premise of low parameter amount and low delay.

[0076] Figure 3 is the detection result image of the PCB defect of the present application, Figure 3 in which open_circuit represents an open circuit, missing_hole represents a missing hole defect, and short represents a short circuit. The confidence corresponding to the open circuit defect is 0.81, the confidence corresponding to the missing hole defect is 0.84, and the confidence corresponding to the short circuit defect is 0.78.

[0077] The lightweight Transformer defect detection method for high-density PCB provided in this embodiment comprises: acquiring a high-density PCB image; performing data preprocessing on the high-density PCB image to obtain a preprocessed image, wherein the data preprocessing comprises normalization and size adjustment; performing deep separable convolution on the preprocessed image to obtain a plurality of local features; performing dynamic sparse attention calculation on the plurality of local features to obtain features of a key region; fusing features of different key regions by using a MobileViT lightweight convolution block to obtain fused features; performing defect classification and positioning prediction on the fused features to obtain a PCB defect detection result; and wherein the PCB defect detection result comprises a defect type and a defect position of the high-density PCB image. After the high-density PCB image is preprocessed by normalization and size adjustment, the accuracy of subsequent PCB defect detection is significantly improved. The deep separable convolution is used instead of the traditional convolution operation, and the deep separable convolution is decomposed into two steps of channel-by-channel convolution and point-by-point convolution, thereby greatly reducing the parameter quantity and computational complexity. On this basis, the dynamic sparse attention mechanism is introduced, the mask matrix is dynamically generated in the manner of feature importance evaluation, the mask matrix covers only part of the region of all local features, each element in the mask matrix is 0 or 1, if the value of the element is 1, the feature at the position of the element is retained, if the value of the element is 0, the feature at the position of the element is removed, the feature correlation modeling of the key region can be realized, and the redundant calculation of the non-key region is reduced, thereby reducing the amount of calculation and allocating most of the attention to the tiny defects. Finally, the MobileViT lightweight convolution block is used to perform pruning and quantization operations on the model, thereby further optimizing the running speed and data storage capacity of the model. The PCB defect detection method of this embodiment can reduce the computational complexity and model parameter quantity while ensuring high detection accuracy, is suitable for industrial applications, and meets the requirements of high-precision and high-efficiency PCB defect detection.

[0078] Embodiment 2

[0079] As shown in Figure 1 , the embodiment provides a lightweight Transformer defect detection method for high-density PCB, comprising:

[0080] S1: acquiring a high-density PCB image;

[0081] S2: performing data preprocessing on the high-density PCB image to obtain a preprocessed image, wherein the data preprocessing comprises normalization and size adjustment;

[0082] S3: performing deep separable convolution on the preprocessed image to obtain a plurality of local features;

[0083] S4: performing dynamic sparse attention calculation on the plurality of local features to obtain features of a key region;

[0084] S5: fusing features of different key regions by using MobileViT light weight convolutional block to obtain fused features;

[0085] S6: performing defect classification and positioning prediction on the fused features to obtain a PCB defect detection result; wherein the PCB defect detection result comprises a defect type and a defect position of the high-density PCB image.

[0086] Dynamic sparse attention is an attention mechanism that reduces computational complexity by dynamically selecting the most relevant attention regions. The core idea is to dynamically determine which position attention weight needs to be calculated according to the input data, rather than calculating all global attention positions as in traditional attention.

[0087] The dynamic sparse attention calculation on the plurality of local features to obtain the features of the key regions comprises:

[0088] The features of the key regions are extracted by using the following formula:

[0089] ;

[0090] wherein SparseAttn is dynamic sparse attention, Q is a query matrix, K is a key matrix, and V is a value matrix; Softmax is a normalized exponential function, K T is the transpose matrix of the key matrix, is an element-wise multiplication operation, M is a mask matrix, the mask matrix is generated by using a dynamic screening method based on feature response or a multi-scale feature extraction method, d length is the vector length of the key matrix. Dynamic sparse attention is a matrix, the query matrix Q comes from the local features of the first ser1 key region, the key matrix K and the value matrix V come from the local features of the first ser2 key region, wherein ser1≠ser2.

[0091] The core of the dynamic sparse attention mechanism of the embodiment is to calculate only the associated key regions by using the mask matrix M. The size of the mask matrix M is N×N, each element in the mask matrix M is 0 or 1, if the value of the element is 1, the feature at the position of the element is retained; if the value of the element is 0, the feature at the position of the element is removed. It can be inferred that the local features extracted by the depth separable convolution will be input to the dynamic sparse attention, the mask M is generated by the internal mechanism of the module, and then the M is substituted into the dynamic sparse attention formula to calculate the attention vector, thereby realizing the feature correlation modeling of the key regions and reducing the redundant calculation of the non-key regions.

[0092] The attention mask M is dynamically generated through an internal mechanism such as feature importance evaluation, and then M is substituted into the sparse attention formula to calculate the dynamic sparse attention, so as to realize the feature correlation modeling of the key area and reduce the redundant calculation of the non-key area. The mask matrix M is generated in any one of the following two ways: (1) a dynamic screening method based on feature response, which scores the importance of the input local features through a lightweight convolution or a small MLP, that is, the feature response value of the defect area is higher. Select the top 10% of the area as the non-zero element position, and set the rest to 0, so as to ensure that the sparsity S of the mask matrix M is 0.1. (2) A multi-scale feature extraction method is adopted to decompose the local features to different scales, set adaptive thresholds at different scales, filter low-response background areas, and only keep high-response micro-defect candidate areas to form a dynamic mask M.

[0093] The area, ellipticity, linearity, rectangularity and / or perimeter of the defect on the surface of the high-density PCB are different from those of the normal electronic components. The geometric features of the defect are detected by using a feature detection method such as Hough transform. The closer the geometric features are to the importance score of the image area of the PCB defect, the larger the corresponding feature response value is. The RGB three-channel mean and / or gray value of the defect on the surface of the high-density PCB are different from those of the normal electronic components. The larger the variance of the area or the greater the entropy of the area, the higher the importance score of the image area, and the larger the corresponding feature response value.

[0094] One of the above two ways is used to generate the mask matrix M, which can not only ensure that the mask matrix M changes dynamically with the input features, that is, it dynamically adapts to the defect distribution of different PCB images, but also can control the calculation amount by fixing the sparsity, for example, 10%, so as to greatly reduce the operation amount while allocating most of the attention to the tiny defects.

[0095] The calculation formula for generating the mask matrix M based on the dynamic screening method of feature response is:

[0096] ;

[0097] Wherein, M is an N*N mask matrix, M(i,j) is an element of the i-th row and the j-th column of the mask matrix, is an indicator function, when the condition in the parentheses is true, 1, otherwise 0. S is an importance score matrix with a size of N*N; S(i,j)=MLP(Conv(F(i,j))), F(i,j) is the local feature of the i-th row and the j-th column extracted by the depth separable convolution, Conv is a lightweight convolution, and MLP is a small multi-layer perceptron; T is a threshold, T=min({S(i,j)∣S(i,j)∈S top}) Stop Let T be the set of the top 10% of elements in the importance rating matrix S, and let T be the sum of the values ​​of S. top The minimum value in.

[0098] The formula for calculating the mask matrix M using a multi-scale feature extraction method is as follows:

[0099] ;

[0100] Where M is an N×N mask matrix, and M(i,j) is the element in the i-th row and j-th column of the mask matrix. This is to perform an element-wise logical OR operation on Ktotal single-scale masks, meaning that when any single-scale mask is 1 at that position, the final result is 1; when all single-scale masks are 0 at that position, the final result is 0. For the single-scale mask at the k-th scale, The calculation formula is: , This is an indicator function; when the condition inside the parentheses is true, The value is 1, otherwise The value is 0. F k The feature matrix at the k-th scale is the decomposition of F to scale S. k As a result, T k The adaptive threshold at the k-th scale is based on the feature matrix F at the k-th scale. k Calculation of scale characteristic distribution T k Ktotal represents the total number of scales, and Ktotal is a positive integer. For F(i,j) at the k-th scale, 30% or 40% of the feature points are randomly selected within a fixed window centered on the feature point (i,j) to obtain the neighboring feature point set. The fixed window size is 5×5 or 7×7, and rounding is used. For example, if the fixed window size is 5×5, then 30% of the feature points are 7, and 40% of the feature points are 10; if the fixed window size is 7×7, then 30% of the feature points are 14, and 40% of the feature points are 19. The similarity between each feature point in the neighboring feature point set and the central feature point (i,j) is calculated. The average of the 50% of feature points with the middle similarity is calculated. For example, if the neighboring feature point set contains 10 neighboring feature points, the 10 neighboring feature points are sorted in ascending order according to the similarity between each neighboring feature point and the central feature point (i,j). The average of the 3rd to 7th neighboring feature points after ascending order is used as the adaptive threshold T for the k-th scale. k Alternatively, the average of the 4th to 8th adjacent feature points after ascending sorting can be used as the adaptive threshold T for the k-th scale. kIf the adjacent feature point set contains 11 adjacent feature points, the average value of the 4th adjacent feature point to the 8th adjacent feature point in ascending order is taken as the adaptive threshold T of the kth scale k The average value of the plurality of adjacent feature points with a similar degree in ascending order is taken as the adaptive threshold T of the kth scale k Based on the spatial position relationship between different feature points, the data complementarity of the feature points with a similar degree is fully utilized, the judgment standard of the indicator function is moderate, the single-scale mask generated by the indicator function has a suitable range, thereby avoiding missing the feature positions corresponding to the micro defects, and preventing the calculation amount from being too large due to the large mask matrix. Randomly selecting part of the feature points in the fixed window makes the calculation of the adaptive threshold robust and can adapt to different application scenarios.

[0101] The dynamic sparse attention calculation module can be dynamically adjusted according to the input local feature to generate an attention mask M, thereby accurately screening out the key region containing the micro defect. The first way to generate the attention mask M is to score the importance of the input feature map through a lightweight convolution or a small MLP, select the top 10% of the region as the non-zero element position, and set the remaining region to zero, so as to ensure that the sparsity S of M is 0.1, thereby generating a dynamic mask M. The second way to generate the attention mask M is multi-scale thresholding, which combines the multi-scale feature extraction capability of the dynamic sparse attention calculation module to set adaptive thresholds at different scales, filter the low-response background region, and only retain the high-response micro defect candidate region to form a dynamic mask M. The above ways of generating the attention mask M can not only ensure that M changes dynamically with the input feature to adapt to the defect distribution of different high-density PCB images, but also control the calculation amount by fixing the sparsity, so that the calculation amount is reduced by 90%.

[0102] The embodiment is after the local feature is extracted by the depth separable convolution, the dynamic sparse attention calculation module is introduced to significantly reduce the calculation complexity while retaining the global context information. The module can effectively capture the micro defect features at different scales through multi-scale feature extraction, and at the same time avoid excessive attention to large-size images in calculation, thereby improving the detection accuracy and reducing the calculation burden.

[0103] Embodiment 3

[0104] As shown in Figure 1 The embodiment provides a lightweight Transformer defect detection method for high-density PCBs, which includes:

[0105] S1: Obtain a high-density PCB image;

[0106] S2: performing data preprocessing on the high-density PCB image to obtain a preprocessed image, the data preprocessing comprising normalization and size adjustment;

[0107] S3: performing deep separable convolution on the preprocessed image to obtain a plurality of local features;

[0108] S4: performing dynamic sparse attention calculation on the plurality of local features to obtain features of key regions;

[0109] S5: fusing features of different key regions by using a MobileViT lightweight convolution block to obtain fused features;

[0110] S6: performing defect classification and positioning prediction on the fused features to obtain a PCB defect detection result; wherein the PCB defect detection result comprises a defect type and a defect position of the high-density PCB image.

[0111] As shown in the description, Figure 2 fusing features of different key regions by using a MobileViT lightweight convolution block to obtain fused features comprises:

[0112] S51: performing convolution operation on different key regions to extract local features of different key regions;

[0113] S52: extracting global features of all key regions by using a Transformer model;

[0114] S53: performing feature splicing on the local features and the global features to obtain fused features.

[0115] MobileViT is a lightweight visual model that combines the local modeling capability of CNN and the global modeling capability of ViT, achieving high performance while maintaining low computational load. One of the core modules of MobileViT is a lightweight convolution block (Light Convolutional Block), such as deep separable convolution, which is mainly used for local feature extraction and is complementary to the subsequent global Transformer module. By replacing the traditional Transformer layer with a lightweight convolution block, the computational complexity and parameter quantity of the model are greatly reduced.

[0116] Lightweight convolution efficiently captures local details such as edges and textures at shallow layers, and enhances feature expression capability through point-by-point convolution and channel attention. The Transformer module models global context relationships on low-resolution feature maps at deep layers using sparse attention or window mechanism, reducing computational complexity. The two are connected by residual connection or adaptive weighting fusion, forming a collaborative architecture of local perception and global reasoning.

[0117] Before fusing features of different key regions by the MobileViT lightweight convolution block, the method further comprises:

[0118] S41' prunes the MobileViT lightweight convolution block to remove redundant network connections.

[0119] S42' converts the weights of the MobileViT lightweight convolution block from floating-point numbers to fixed-point numbers.

[0120] Pruning is a model compression technique that identifies and removes weights, neurons, and channels that have little or no contribution to the output in a trained neural network, thereby reducing the number of parameters and computations while maintaining the detection accuracy through fine-tuning. By pruning the MobileViT lightweight convolution block, redundant network connections are removed, significantly reducing the amount of computation.

[0121] Quantization is a method of reducing model storage and computational overhead by reducing the precision of numerical representation in neural networks. The core idea is to convert high-precision floating-point operations into low-precision integer operations, thereby maintaining model performance while significantly improving efficiency. By quantizing the weights of the MobileViT lightweight convolution block from floating-point numbers to fixed-point numbers, the model's running speed is further improved, and the data storage requirement is reduced.

[0122] After pruning the MobileViT lightweight convolution block, the method further comprises:

[0123] The number of parameters of the pruned model is calculated according to the following formula:

[0124] ;

[0125] where P pruned is the number of parameters of the pruned model, P original is the original number of parameters, r is the pruning rate, is the multiplication operation.

[0126] The Transformer model is used to extract global features of all key regions, including:

[0127] The output features after nonlinear transformation are calculated using the following formula:

[0128] ;

[0129] where LN is layer normalization, SparseAttn is dynamic sparse attention, MLP is multi-layer perceptron, F out is the output feature after nonlinear transformation, F in is the matrix of key regions.

[0130] The output features after all nonlinear transformations are spliced to obtain global features of all key regions.

[0131] Each time F is calculated out , F is the matrix of two different key regions, for example, the query matrix Q of the first ser1 key region and the key matrix K and the value matrix V of the first ser2 key region are taken as the F in for extracting global features for the 10th time. in , the F in of this time is layer normalized, and then the dynamic sparse attention of the layer normalized result is calculated. When F out is calculated next time, the query matrix Q of the first ser1+1 key region and the key matrix K and the value matrix V of the first ser2+1 key region are taken as the F in for extracting global features for the 11th time. The above steps are repeated until all F out are extracted. out All F out are spliced to obtain global features of all key regions.

[0132] The standard VIT model divides the entire image into non-overlapping image blocks and converts them into a sequence input into a Transformer-based encoder, which has the problems of high computational complexity and less applicable scenarios. MobileViT combines local representation and global representation, retains the local features of key regions at each position, uses the Transformer module to extract and enhance global features, and fuses local features and global features to generate high-quality visual representations. MobileViT performs convolution on different key regions to extract context information of different key regions. Then, normalization and nonlinear activation are used to improve the expression ability of local features, and the nonlinear activation uses the ReLU activation function. The Transformer model is used to divide all key regions into multiple image blocks, each image block is expanded into a one-dimensional vector, and long-distance dependencies are captured through multiple Transformer Blocks. The local features and the global features are spliced in the channel dimension, and the fused features are generated through a lightweight convolution operation.

[0133] The fusion features are classified and positioned to obtain a PCB defect detection result, including:

[0134] The fusion feature is input into a YOLOv7 model for defect detection to generate a plurality of detection boxes; each detection box corresponds to a defect type and a rectangular starting point coordinate of the high-density PCB image, one detection box corresponds to one defect type and one rectangular starting point coordinate, for example, detection box A corresponds to a short circuit defect, and the rectangular starting point coordinate corresponding to detection box A is (100, 200), and according to the rectangular starting point coordinate of the detection box, the defect position of the high-density PCB image can be determined.

[0135] YOLOv7 is composed of Backbone, Neck and Head, Backbone, i.e. the backbone network, realizes efficient feature extraction by stacking ELAN modules, and ELAN modules enhance feature learning ability by controlling gradient path length. Neck, i.e. the neck network, combines top-down FPN and bottom-up PAN to form a multi-scale feature fusion network to enhance the detection ability of targets of different sizes. Head, i.e. the head network, is used to predict the position of the detection box, the confidence of the object in the detection box, and the class probability of the object in the detection box.

[0136] The fusion feature of the embodiment is an image, if the fusion feature contains multiple targets, the fusion feature is input into the YOLOv7 model, and multiple detection boxes are detected, each detection box corresponds to one target. Adjust the confidence threshold, IoU threshold and maximum number of boxes per image of the YOLOv7 model to reduce the probability of multiple overlapping detection boxes appearing at the same target. If there are many detection boxes with low confidence, increase the confidence threshold. If there are small targets that are missed, increase the resolution of the input fusion feature.

[0137] Embodiment 4

[0138] The embodiment provides a lightweight Transformer defect detection device for high-density PCBs, comprising:

[0139] An image input module is configured to acquire a high-density PCB image.

[0140] A data preprocessing module is configured to perform data preprocessing on the high-density PCB image to obtain a preprocessed image, wherein the data preprocessing includes normalization and size adjustment.

[0141] A depth separable convolution module is configured to perform depth separable convolution on the preprocessed image to obtain a plurality of local features.

[0142] A dynamic sparse attention calculation module is configured to perform dynamic sparse attention calculation on the plurality of local features to obtain features of key regions.

[0143] A feature fusion module is configured to fuse features of different key regions using MobileViT lightweight convolution blocks to obtain a fusion feature.

[0144] The PCB defect detection module is configured to perform defect classification and location prediction on the fusion features, and obtain a PCB defect detection result. The PCB defect detection result includes a defect type and a defect location of the high-density PCB image.

[0145] The light-weighted Transformer defect detection device for high-density PCBs is configured to implement a light-weighted Transformer defect detection method for high-density PCBs.

[0146] It should be noted that in this document, the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, device, article, or method that comprises a list of elements does not only include those elements, but also other elements not explicitly listed, or inherent to such a process, device, article, or method. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, device, article, or method that includes the element.

[0147] The above description is merely preferred embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation based on the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A lightweight Transformer defect detection method for high-density PCBs, characterized by, The method comprises the following steps: obtaining a high-density PCB image; performing data preprocessing on the high-density PCB image to obtain a preprocessed image, wherein the data preprocessing comprises normalization and size adjustment; performing deep separable convolution on the preprocessed image to obtain a plurality of local features; performing dynamic sparse attention calculation on the plurality of local features to obtain features of key regions; fusing features of different key regions by using a MobileViT lightweight convolution block to obtain fused features; performing defect classification and positioning prediction on the fused features to obtain a PCB defect detection result; wherein the PCB defect detection result comprises a defect type and a defect position of the high-density PCB image.

2. The high-density PCB-oriented lightweight Transformer defect detection method of claim 1, wherein, The deep separable convolution on the preprocessed image to obtain a plurality of local features comprises: performing channel-by-channel convolution on the preprocessed image to obtain k first feature maps; wherein k is the same as the number of channels of the preprocessed image, and k≥2; inputting the k first feature maps into an input feature map sequence, and sequentially inputting the input feature map sequence into q point-by-point convolution kernels to perform q times of point-by-point convolution to obtain q local features, q≥2. 3.The high-density PCB-oriented lightweight Transformer defect detection method of claim 1, wherein, The dynamic sparse attention calculation on the plurality of local features to obtain features of key regions comprises: extracting features of key regions by using the following formula: ； wherein SparseAttn is a dynamic sparse attention, Q is a query matrix, K is a key matrix, and V is a value matrix; Softmax is a normalized exponential function, K T is a transpose matrix of the key matrix, is an element-wise multiplication operation, M is a mask matrix, the mask matrix is generated in a dynamic screening manner based on feature responses or a multi-scale feature extraction manner, d length is a vector length of the key matrix. 4.The high-density PCB-oriented lightweight Transformer defect detection method of claim 1, wherein, The fusing features of different key regions by using a MobileViT lightweight convolution block comprises: performing convolution operation on different key regions to extract local features of different key regions; extracting global features of all key regions by using a Transformer model; performing feature splicing on the local features and the global features to obtain fused features. 5.The high-density PCB-oriented lightweight Transformer defect detection method of claim 2, wherein, The channel-by-channel convolution on the preprocessed image to obtain k first feature maps comprises: performing channel-by-channel convolution on the preprocessed image by using the following formula: ； Wherein, X is the pre-processed image, W std is a per-channel convolution kernel, Y i,j,c is the value of the i-th row and j-th column of the c-th first feature map, i is the row index of the first feature map, j is the column index of the first feature map, row is the total number of rows of the first feature map, col is the total number of columns of the first feature map; c is the channel index, k is the total number of channels, m is the row adjustment amount, n is the column adjustment amount, is a multiplication operation. 6.The high-density PCB-oriented lightweight Transformer defect detection method of claim 1, wherein, Before the fusing features of different key regions by using a MobileViT lightweight convolution block, the method further comprises: pruning the MobileViT lightweight convolution block to remove redundant network connections; converting the weights of the MobileViT lightweight convolution block from floating-point numbers to fixed-point numbers. 7.The high-density PCB-oriented lightweight Transformer defect detection method of claim 6, wherein, After the pruning the MobileViT lightweight convolution block, the method further comprises: calculating the model parameter quantity after pruning according to the following formula: ； where P pruned is the parameter quantity of the pruned model, P original is the original parameter quantity, and r is the pruning rate, is a multiplication operation. 8.The high-density PCB-oriented lightweight Transformer defect detection method of claim 4, wherein, The extracting global features of all key regions by using a Transformer model comprises: calculating the output features after nonlinear transformation by using the following formula: ； where LN is layer normalization, SparseAttn is dynamic sparse attention, MLP is multi-layer perceptron, F out is the output feature after nonlinear transformation, F in is the matrix of key regions; splicing all the output features after nonlinear transformation to obtain global features of all key regions. 9.The high-density PCB-oriented lightweight Transformer defect detection method of claim 1, wherein, The performing defect classification and positioning prediction on the fused features to obtain a PCB defect detection result comprises: inputting the fused features into a YOLOv7 model for defect detection to generate a plurality of detection boxes; each detection box corresponds to a defect type and a rectangular starting point coordinate of the high-density PCB image.

10. A lightweight Transformer defect detection device for high-density PCBs, characterized by, The method comprises the following steps: an image input module for obtaining a high-density PCB image; a data preprocessing module for performing data preprocessing on the high-density PCB image to obtain a preprocessed image, wherein the data preprocessing comprises normalization and size adjustment; a deep separable convolution module configured to perform deep separable convolution on the preprocessed image to obtain a plurality of local features; a dynamic sparse attention calculation module configured to perform dynamic sparse attention calculation on the plurality of local features to obtain features of key regions; a feature fusion module configured to fuse features of different key regions by using a MobileViT lightweight convolution block to obtain fused features; a PCB defect detection module configured to perform defect classification and positioning prediction on the fused features to obtain a PCB defect detection result, wherein the PCB defect detection result comprises a defect type and a defect position of the high-density PCB image.

Images