A PCB defect detection method based on LDLFModel

Abstract

The present application relates to a PCB defect detection method based on LDLFModel, which includes the following steps: enhancing a collected PCB dataset; constructing an LDLFModel network model, where the backbone network of the model is a MobileNet V3‑Small network; adding a position attention mechanism to the last layer and the Neck portion of the backbone network, and training the model using Eiou loss as the target box loss function of the model; inputting the PCB dataset to be detected into a cutting module for cutting, then inputting it into the trained model for prediction, and finally inputting the prediction results into a restoration module for restoration to a final visualization result. The present invention uses the LDLFModel network model as a detection model. On the basis of ensuring detection accuracy, it further compresses the number of model parameters and computational complexity, facilitates the deployment of lower-performance devices, and improves the efficiency and accuracy of defect detection.

Description

Technical Field

[0001] The present application relates to the field of PCB defect identification and detection, and specifically to a PCB defect detection method based on LDLFModel. Background Art

[0002] Printed circuit boards (PCBs), also known as printed circuit boards or PCBs, are essential components of various electronic products. Known as the "mother of electronic products," PCBs are essential components of any electronic device and product, and their performance significantly impacts the quality of these products. Virtually every electronic device relies on PCBs, from small electronic watches and calculators to large aerospace and military weapon systems. In recent years, with the continuous advancement of production technology, PCBs have rapidly evolved toward ultra-thin designs, featuring smaller components, higher density, and finer pitches. This trend inevitably presents numerous challenges and difficulties in quality inspection. Consequently, PCB defect detection has become a core issue in the PCB manufacturing process and a major concern for electronics manufacturers.

[0003] Currently, there are several main methods for PCB defect detection. The first is manual visual inspection, which relies primarily on the inspector's subjective judgment of the defect type and location. However, this method is prone to eye fatigue, which can result in PCB defects not being detected. Furthermore, manual visual inspection is inefficient. The second is automated optical inspection (AOI). This technology uses a computer vision system to input surface images of the finished PCB into a computer. Using image processing and pattern recognition techniques, the system processes and analyzes the surface images and compares them with standard images, thereby inspecting the surface quality of the PCB. Due to its multi-camera technology, this method can detect many defects that are normally difficult to detect, making it very powerful. However, it has a drawback: high cost. The high cost of imported equipment has placed significant financial pressure on domestic PCB manufacturers. Furthermore, these equipment are often not suitable for domestic operators, and their capabilities are not fully utilized during use, resulting in significant waste of resources. The third method is defect detection based on traditional digital image features, which has been widely used in nondestructive testing in recent years. Although this method has considerable detection accuracy and can significantly reduce the cost of detection equipment, it still has shortcomings. The disadvantage is that it requires manual setting of defect features, which is labor-intensive and inefficient. At the same time, the recognition accuracy of this method needs to be improved. The fourth type is a defect detection method based on deep learning. Because deep learning-based methods have greatly improved the effectiveness of defect detection, and because of their high detection accuracy and efficiency, deep learning-based defect detection methods have been widely used.

[0004] In recent years, with the development of deep learning, more and more scholars have applied deep learning to defect detection. The current PCB defect detection methods based on deep learning mainly use two-stage defect detection algorithms represented by Faster R-CNN and one-stage defect detection algorithms represented by YOLO. Among them, although Faster R-CNN has high detection accuracy and a certain detection speed, Faster R-CNN uses anchor boxes on the feature map to correspond to the original image, and the anchor boxes have undergone multiple downsampling operations and correspond to the original image. Figure 1Due to the large size of the block, Faster R-CNN is not very effective in detecting small PCB defects. To address this problem, the single-stage detection algorithm YOLO v5 can detect defects of different scales and has good results in detecting small defects. However, the large number of parameters and computational complexity of the YOLO v5 network makes it difficult to deploy on lightweight devices. The portability of the detection equipment needs to be improved. At the same time, detection accuracy still needs to be improved, and small defects may be missed during the detection process. Summary of the Invention

[0005] The purpose of the present invention is to provide a PCB defect detection method based on LDLFModel, using the LDLFModel network model as a first-stage detection model. On the basis of ensuring detection accuracy, the number of model parameters and the amount of calculation are further compressed to facilitate the deployment of lower-performance equipment, while improving the efficiency and accuracy of defect detection.

[0006] The technical solution adopted by the present invention is: a PCB defect detection method based on LDLFModel, comprising the following steps:

[0007] S1: Perform data enhancement on the collected PCB dataset to form training data;

[0008] S2: Construct a deep learning lightweight fusion network model, namely the LDLFModel network model. The LDLFModel network model consists of a backbone, a neck, and a prediction part. The backbone part uses MobileNet v3-Small as the feature extraction network, and adds a position attention mechanism to the last layer of MobileNet v3-Small. The neck part uses a combination of feature pyramid FPN and path fusion PAN for feature fusion, and adds a position attention mechanism to the neck part to obtain an enhanced feature layer. The prediction part is a YOLO head module for target prediction.

[0009] S3: Train the LDLFModel network model. The specific steps are as follows:

[0010] S301: Input the data-enhanced PCB dataset into the Backbone part of the LDLFModel network model, and sequentially downsample it through MobileNet v3-Small with an attention mechanism to obtain three effective feature layers with different resolutions;

[0011] S302: Input the three effective feature layers obtained in step S301 into the Neck part for feature fusion, perform upsampling and downsampling operations on the three effective feature layers in sequence, and use a fast normalization fusion method to perform feature fusion to obtain three enhanced effective feature layers;

[0012] S303: Input the three enhanced effective feature layers obtained in step S302 into the YOLO head module to predict the center point position of the PCB defect target, obtain the prediction box width and height, confidence and PCB defect target category information, and use the target box loss function Eiou loss as the loss function of the LDLFModel network model ζ EioU , improve the accuracy of prediction box positioning;

[0013] S4: The trained LDLFModel network model is used to identify PCB defects and obtain defect detection results. The specific method is: the PCB defect data set to be detected is input into the cutting module for cutting, and the PCB defect data set to be detected is cut into small images, and there is a 20% overlapping area between the small images adjacent to the left and right or the upper and lower adjacent small images. The cut small images are input into the trained LDLFModel network model for PCB defect detection, and the small images with detection results are output. The small images with detection results are input into the restoration module for splicing and restoration to obtain the final visual detection results.

[0014] Furthermore, the specific method of step S1 is: randomly flipping, randomly mirroring, randomly changing brightness, and randomly scaling the PCB dataset to expand the number of datasets, and then using Mosica data enhancement to obtain the final training data.

[0015] Furthermore, the position attention mechanism in step S2 is used to embed and generate coordinate information; the principle of coordinate information embedding is:

[0016] For a given input feature x, a pooling kernel of size (H, 1) or (1, W) is used to encode each channel along the horizontal and vertical coordinates respectively, and the output is The specific expression is:

[0017]

[0018] Among them, W is the width of the pooling kernel, m is the value of the pooling kernel in the W spatial range, and x c (h,m) is the feature map input tensor with height h and W taking m, is the output of the cth channel with height h;

[0019] The c-th pass output with width w The specific expression is:

[0020]

[0021] Among them, H is the height of the pooling kernel, n is the value of the pooling kernel in the H space range, x c (n,w) is the feature map input tensor with width w, H takes n, is the output of the cth channel with width w;

[0022] The generation principle of coordinate information is:

[0023] After the transformation in the coordinate information embedding, the transformed output result Perform Concat operation to get [z h ,z w ], and finally, the intermediate feature map f that encodes the spatial information in the horizontal and vertical directions is obtained through the nonlinear activation function δ. The specific expression is as follows:

[0024] f=δ(F1([z h ,z w ]))

[0025] Among them, F1 is a 1×1 convolution transformation function;

[0026] The intermediate feature map f is then decomposed into two separate tensors f along the spatial dimension h and f w , and then use two 1×1 convolution transformation functions F h and F w And the nonlinear activation function σ, that is, the sigmoid function, respectively converts f h and f w Transformed into a tensor g with the same number of channels h and g w , the specific expression is as follows:

[0027] g h =σ(F h (f h ))

[0028] g w =σ(F w (f w ))

[0029] Then the output y of the position attention mechanism is c The specific expression of (m,n) is:

[0030]

[0031] Among them, x c(m,n) is the number of channels, the width of the feature map is m, and the height of the feature map is n. A tensor with c as the number of channels, h as the height, and m as the width, A tensor with width w and height n, where c is the number of channels.

[0032] Furthermore, in step S302, the expression of fast normalization fusion is:

[0033]

[0034] Among them, O is the result of feature fusion, w i Represents the i-th input feature I i The corresponding learning weight, ∈, is a non-zero constant used to ensure the stability of the value; by i Perform a Relu operation to ensure w i The value of is greater than or equal to 0;

[0035] The fusion of level x and level x+1 feature layers at level x is specifically expressed as:

[0036]

[0037] Among them, Resize is the up and down sampling operation, Conv is the convolution operation of the feature, w i Represents the i-th input feature I i The corresponding learning weight, w i+1 Represents the i+1th input feature I i+1 The corresponding learning weights, represents the feature output after the k-th level fusion, represents the feature input of the kth level, Represents the feature input of the k+1th level.

[0038] Furthermore, in step S303, the loss function ζ EioU The specific expression is:

[0039]

[0040]

[0041] Among them, ρ(b p ,b gt ) is the Euclidean distance between the center point of the predicted box and the real box, ρ(w,w gt ) is the Euclidean distance between the predicted frame width and the true frame width, ρ(h,h gt ) is the Euclidean distance between the predicted box height and the real box height, t is the diagonal area of ​​the minimum circumscribed rectangle of the real box of the target box, T wis the width of the minimum bounding box, T h is the height of the minimum bounding box, IoU is the intersection-over-union ratio of the predicted box and the true box, A is the predicted box, and B is the true box.

[0042] Furthermore, in step S4, the cutting method of the cutting module is:

[0043] Set the cutting coefficient mul. When mul=1, the cutting layer is closed. When mul>1, the cutting layer is opened. The relationship between the number of small images N and mul is:

[0044] N=mul 2

[0045] First, get the length x of the PCB image to be tested size He Kuany size ; Then calculate the length x_smoc and width y_smoc of the cutting box respectively:

[0046]

[0047]

[0048] Then calculate the starting coordinates (x starpoint ,y starpoint ):

[0049]

[0050]

[0051] Where p and q are natural numbers;

[0052] Then calculate the relative coordinates of the last cutting position (x real ,y real )

[0053] x real =min(x starpoint +x_smoc,x size )

[0054] y real =min(y starpoint +y_smoc,y size )

[0055] Compare the length and width of the image to be detected with the relative coordinates of the last cutting position to determine whether the cutting is just completed. If it is not just completed, the last cutting frame position is taken as (x real ,y real ) to make the last cutting frame flush with the right border of the image to be detected;

[0056] The specific method of the restoration module to restore all the cut small pictures is as follows: starpoint ,y starpoint ) The cutout images with detection results are spliced ​​together, non-maximum suppression is performed, and redundant frames are removed to obtain the final visual detection results.

[0057] The beneficial effects of the present invention are:

[0058] (1) The present invention addresses the problem that the resolution of the PCB image to be inspected is too high and the target defect is small and difficult to detect. By adding a cutting module before the LDLFModel network model detection and a restoration module after the LDLFModel network model detection, the input data is Mosaic enhanced, and transfer learning is used to obtain better training effects, the problem of small defects being easily missed is effectively solved;

[0059] (2) The present invention introduces a position attention mechanism into the last layer and the Neck part of the backbone network of the LDLFModel network model, adopts the lightweight feature extraction network MobileNet V3-Small as the backbone network, adopts the Eiou_loss function with higher positioning accuracy as the loss function of the LDLFModel network model, and adopts a fast normalization fusion method for feature weighted fusion, thereby effectively solving the problems of traditional PCB defect detection methods, such as low detection efficiency, need to improve detection results, and high cost of detection equipment, thereby improving the precision and accuracy of defect detection positioning and facilitating the deployment of lightweight equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the network structure of MobileNet V3-Small in an embodiment of the present invention;

[0063] Figure 3 Schematic diagram of the structure of the position attention mechanism in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram of the structure of the LDLFModel network model in an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram of the cutting principle of a PCB image according to an embodiment of the present invention;

[0066] Figure 6 This is a schematic diagram of the PCB image defect detection results according to an embodiment of the present invention;

[0067] Figure 7 Schematic diagram of the confusion matrix of the detection results of an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0069] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning understood by persons of ordinary skill in the field described in this application. "First", "second" and similar words used in this patent application specification and claims do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, "one" or "an" and similar words do not indicate a quantity limitation, but rather indicate the existence of at least one. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0070] like Figures 1 to 4 As shown, a PCB defect detection method based on LDLFModel includes the following steps:

[0071] S1: Enhance the collected PCB data set to form training data. The specific method is: randomly flip, randomly mirror, randomly change the brightness, and randomly scale the PCB data set to expand the number of data sets, and then use Mosica data enhancement to obtain the final training data. The most original PCB data collected in the embodiment of the present invention contains six types of PCB defects, namely, missing solder holes (missing_hole), open circuits (open_circuit), short circuits (short), mouse bites (mouse_bite), burrs (spur), and spurious copper (spurious_copper). The most original PCB data are 65 pictures of missing solder holes, 81 pictures of open circuits, 82 pictures of short circuits, 72 pictures of mouse bites, 73 pictures of burrs, and 77 pictures of spurious copper, totaling 450 pictures. After data augmentation, each category was expanded 24-fold, resulting in a total dataset of 10,800 images. Mosica data augmentation was then used to randomly splice the PCB data images, followed by random scaling and rotation, to improve the model's detection performance for small objects. The image types and number of the augmented dataset are shown in Table 1.

[0072] Table 1 Types and numbers of defect datasets

[0073] Types of defects quantity Solder joint leakage 1560 Short Circuit 1968 open circuit 1944 Rat bite 1728 Remaining copper 1848 glitch 1752

[0074] S2: Construct a deep learning lightweight fusion network model, namely the LDLFModel network model; the structure of the LDLFModel network model is as follows Figure 4 The figure shows the Backbone, Neck, and Prediction parts. The Backbone uses MobileNet v3-Small as the feature extraction network and adds a position attention mechanism to the last layer of MobileNet v3-Small. The Backbone is used to extract features. The Neck uses a combination of Feature Pyramid Networks (FPN) and Path Aggregation Network (PAN) for feature fusion. A position attention mechanism is added to the Neck to obtain an enhanced feature layer. The Prediction part includes the YOLO head module for target prediction.

[0075] S3: Train the LDLFModel network model. The specific steps are as follows:

[0076] S301: Input the data-enhanced PCB dataset into the Backbone part of the LDLFModel network model. MobileNet v3-Small is mainly composed of an inverted residual structure with a linear bottleneck. The inverted residual structure of the linear bottleneck first passes the input features through a 1×1 Conv, then through a 3×3 Depthwise Separable Conv, and finally connects the input and output through a Short Cut. The specific structure diagram is as follows Figure 2 To further improve the detection effect, the last layer of MobileNetv3-Small is added as follows. Figure 3 The position attention mechanism shown in the figure. After sequential downsampling, MobileNet v3-Small obtains three effective feature layers with different resolutions: (120, 120, 24), (60, 60, 48), and (30, 30, 96).

[0077] The position attention mechanism includes the embedding and generation of coordinate information. During the coordinate information embedding process, in order to enable the position attention mechanism to capture long-range spatial interactions with precise position information, the input feature map is first decomposed through global pooling and converted into a one-to-one one-dimensional feature encoding operation. The principle of coordinate information embedding is:

[0078] For a given input feature x, a pooling kernel of size (H, 1) or (1, W) is used to encode each channel along the horizontal and vertical coordinates respectively, and the output z is c h The specific expression of (h) is:

[0079]

[0080] Among them, W is the width of the pooling kernel, m is the value of the pooling kernel in the W spatial range, and x c (h,m) is the feature map input tensor with height h and W taking m, is the output of the cth channel with height h;

[0081] The c-th pass output with width w The specific expression is:

[0082]

[0083] Among them, H is the height of the pooling kernel, n is the value of the pooling kernel in the H space range, x c (n,w) is the feature map input tensor with width w, H takes n, is the output of the cth channel with width w.

[0084] The above two transformations aggregate features along two spatial directions respectively to obtain a pair of direction-aware feature maps. These two transformations also enable the position attention mechanism to capture long-term dependencies along one spatial direction and preserve precise position information along another spatial direction, which helps to more accurately locate PCB defects.

[0085] The generation principle of coordinate information is:

[0086] After the transformation in the coordinate information embedding, the transformed output result Perform Concat operation to get [z h ,z w ], and finally, the intermediate feature map f that encodes the spatial information in the horizontal and vertical directions is obtained through the nonlinear activation function δ. The specific expression is as follows:

[0087] f=δ(F1([z h ,z w ]))

[0088] Among them, F1 is a 1×1 convolution transformation function;

[0089] The intermediate feature map f is then decomposed into two separate tensors f along the spatial dimension h and f w , and then use two 1×1 convolution transformation functions F h and F w And the nonlinear activation function σ, that is, the sigmoid function, respectively converts f h and f w Transformed into a tensor g with the same number of channels h and g w , the specific expression is as follows:

[0090] g h =σ(F h (f h ))

[0091] g w =σ(F w (f w ))

[0092] Then the output y of the position attention mechanism is c The specific expression of (m,n) is:

[0093]

[0094] Among them, x c (m,n) is the number of channels, the width of the feature map is m, and the height of the feature map is n. A tensor with c as the number of channels, h as the height, and m as the width, A tensor with width w and height n, where c is the number of channels.

[0095] like Figure 3 As shown in the figure, the algorithm idea of ​​the position attention mechanism is as follows: first, the features of the two dimensions are extracted from the input feature map respectively, and two sets of features in the height and width directions are extracted respectively. Then, the features of the two dimensions are concat-operated, and then a long-range dependency is constructed through a 1×1 convolution operation to obtain a mixed result of the height and width information. Then, after fast normalization and h-swish activation function for nonlinear activation, the output results are separated by the Split function and convolution operations are performed on the output results to obtain two sets of attention. Then, the sigmoid function is used to frame the values ​​of the two sets of attention between 0 and 1. Since the dimensions of the two sets of attention mechanisms are different, the two sets of attention mechanisms are multiplied by the broadcast mechanism to obtain the final attention with the same dimension as the feature map. Finally, the final attention is multiplied with the input feature map to complete the addition of the position attention mechanism. This can ensure the distinction of the importance of feature points on the feature map, which is conducive to the final location of the defect and improves the accuracy of the model.

[0096] In this way, the LDLFModel network model can not only capture long-range dependencies along one spatial direction, but also retain precise position information along another spatial direction. The resulting feature map is then separately encoded into a pair of direction-aware and position-sensitive attention maps, which can be complementary applied to the input feature map to enhance the representation of PCB defects.

[0097] S302: The three valid feature layers obtained in step S301 are input to the Neck part for feature fusion. The three valid feature layers are first upsampled and downsampled in sequence. Both upsampling and downsampling operations are the process of constructing a dual feature pyramid. During the construction of the dual feature pyramid, feature fusion is performed using a fast normalization fusion method. At the same time, to further improve the accuracy of the model, a positional attention mechanism is added after each 1×1 Conv in the Neck part. Finally, three enhanced valid feature layers are obtained: (120, 120, 33), (60, 60, 33), and (30, 30, 33).

[0098] The expression of fast normalized fusion is:

[0099]

[0100] Among them, O is the result of feature fusion, w i Represents the i-th input feature I iThe corresponding learning weight, ∈ is a non-zero constant used to ensure the stability of the value. In the embodiment of the present invention, ∈ is 0.0001 to ensure that the denominator is not 0. i Perform a Relu operation to ensure w i The value of is greater than or equal to 0; the value of each normalization weight is between 0 and 1, which makes the fast normalization fusion efficient and runs fast on the GPU.

[0101] The fusion of level x and level x+1 feature layers at level x is specifically expressed as:

[0102]

[0103] Among them, Resize is the up and down sampling operation, Conv is the convolution operation of the feature, w i Represents the i-th input feature I i The corresponding learning weight, w i+1 Represents the i+1th input feature I i+1 The corresponding learning weights, represents the feature output after the k-th level fusion, represents the feature input of the kth level, Represents the feature input of the k+1th level.

[0104] S303: The three enhanced effective feature layers obtained in step S302 are input into the YOLO head module to predict the center point position of the PCB defect target, the width and height of the prediction box, the confidence level, and the category information of the PCB defect target. In order to further improve the accuracy of the prediction box positioning, the LDLFModel network model adopts the target box loss function Eiou loss as the loss function of the LDLFModel network model. EioU , improve the accuracy of prediction frame positioning; loss function ζ EioU The specific expression is:

[0105]

[0106]

[0107] Among them, ρ(b p ,b gt ) is the Euclidean distance between the center point of the predicted box and the real box, ρ(w,w gt ) is the Euclidean distance between the predicted frame width and the true frame width, ρ(h,h gt ) is the Euclidean distance between the predicted box height and the real box height, t is the diagonal area of ​​the minimum circumscribed rectangle of the real box of the target box, T w is the width of the minimum bounding box, T his the height of the minimum bounding box, IoU is the intersection-over-union ratio of the predicted box and the true box, A is the predicted box, and B is the true box.

[0108] The prediction process of PCB defects is as follows:

[0109] The LDLFModel network model obtains 9 anchor frames through the Kmeans clustering algorithm. Every three anchor frames are responsible for large, medium and small feature maps respectively. The YOLO head module divides the input feature map into S×S grids according to the size of the three enhanced feature layers obtained, where S is the grid side length. If the center position of the detected defect falls within the grid, then this grid is responsible for detecting the PCB defect target. Each grid will obtain three initial prediction frames. The initial prediction frame is based on the anchor frame by calculating the offset of the initial prediction frame relative to the center coordinates of the anchor frame, and then calculating the scaling factor of the initial prediction frame relative to the width and height of the anchor frame according to the width and height of the anchor frame. The scale is scaled based on the anchor frame to obtain the initial, and finally the three initial prediction frames will calculate the intersection and union (IOU) of the real frame calibrated with the data set to obtain the best prediction frame. After obtaining the best prediction frame, the probability of the defect type in the prediction frame is calculated based on each dimensional vector of the enhanced feature layer. Finally, the confidence level is calculated. The confidence level indicates that the object to be detected exists in the prediction frame and the prediction frame accurately defines the location of the object to be detected. Through the above operations, the location of the final PCB defect type is predicted.

[0110] S4: The trained LDLFModel network model is used to identify PCB defects and obtain defect detection results. The specific method is: input the PCB defect data set to be detected into the cutting module for cutting, and cut the PCB defect data set to be detected into small images. Cutting the PCB defect data set to be detected into small images can improve the recognition ability of the LDLFModel network model for small defect targets, so as to reduce the missed detection rate of small PCB defect targets. In order to ensure that the small target defects of the PCB to be detected are not cut in half, there is a 20% overlapping area between the small images adjacent to the left and right or adjacent to the top and bottom in the embodiment of the present invention. The cut small images are input into the trained LDLFModel network model for PCB defect detection, and the small images with the detection results are output. The small images with the detection results are input into the restoration module for splicing and restoration to obtain the final visual detection results.

[0111] The cutting principle diagram of the cutting module is as follows: Figure 5 As shown in the figure, when cutting an image, the cutting coefficient mul is set. When mul=1, the cutting layer is closed, and when mul>1, the cutting layer is opened. The relationship between the number of small images N and mul is:

[0112] N=mul 2

[0113] First, get the length x of the PCB image to be tested size He Kuany size ; Then calculate the length x_smoc and width y_smoc of the cutting box respectively:

[0114]

[0115]

[0116] Then calculate the starting coordinates (x starpoint ,y starpoint ):

[0117]

[0118]

[0119] Where p and q are natural numbers;

[0120] Then calculate the relative coordinates of the last cutting position (x real ,y real )

[0121] x real =min(x starpoint +x_smoc,x size )

[0122] y real =min(y starpoint +y_smoc,y size )

[0123] Compare the length and width of the image to be detected with the relative coordinates of the last cutting position to determine whether the cutting is just completed. If it is not just completed, the last cutting frame position is taken as (x real ,y real ) to make the last cutting frame flush with the right border of the image to be detected;

[0124] The specific method of the restoration module to restore all the cut small pictures is as follows: starpoint ,y starpoint ) The cutout images with detection results are spliced ​​together, non-maximum suppression is performed, and redundant frames are removed to obtain the final visual detection results.

[0125] The test results obtained in the embodiment of the present invention are as follows Figure 6 As shown in the visualization results, the embodiment of the present invention has a high defect detection accuracy and defect location accuracy. The confusion matrix of the embodiment of the present invention is as follows Figure 7As shown in the figure, the confusion matrix shows that the overall detection effect of the embodiment of the present invention is good, and the accuracy of defect type distinction is high. Among them, mi_ho, mo_bi, op_ci, sh, sp and sp_co, ba_FP, ba_FN represent missing_hole, open_circuit, short, mouse_bite, spur, spurious_copper, background FP and background FN, respectively. BackgroundFP indicates that the background image is mistakenly identified as a defect, and background FN indicates that the defect image is mistakenly identified as the background.

[0126] The performance of the LDLFModel network model described in the embodiment of the present invention is compared with several commonly used detection models in the prior art, and the comparison results shown in Table 2 can be obtained.

[0127] Table 2 Comparison of experimental results

[0128] Model backbone network Map(%) Frame rate (frames / second) YOLO v3 Darknet 53 91.35 40 YOLO v5 CSPDarknet 53 92.21 79 Faster R-CNN Resnet 101 83.70 101 LDLFModel MobileNet V3-Small-CA 99.0 67

[0129] As can be seen from Table 2, the LDLFModel network model described in the embodiment of the present invention has a significant improvement in detection accuracy compared with YOLO v3, YOLO v5, and Faster R-CNN, and its detection speed is also much faster than YOLO v3. It has good comprehensive performance in detection accuracy and detection efficiency.

[0130] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A PCB defect detection method based on LDLFModel, characterized in that: The steps include: S1: Perform data enhancement on the collected PCB dataset to form training data; S2: Construct a deep learning lightweight fusion network model, namely the LDLFModel network model; the LDLFModel network model consists of a backbone part, a neck part, and a prediction part. The backbone part uses MobileNet v3-Small as the feature extraction network, and adds a position attention mechanism to the last layer of MobileNet v3-Small. The neck part uses a combination of feature pyramid FPN and path fusion PAN for feature fusion, and adds a position attention mechanism to the neck part to obtain an enhanced feature layer. The Prediction part is the YOLO head module, which is used for target prediction; S3: Train the LDLFModel network model. The specific steps are as follows: S301: Input the data-enhanced PCB dataset into the Backbone part of the LDLFModel network model, and sequentially downsample it through MobileNet v3-Small with an attention mechanism to obtain three effective feature layers with different resolutions; S302: The three effective feature layers obtained in step S301 are input to the Neck part for feature fusion, and the three effective feature layers are sequentially upsampled and downsampled. The feature fusion is performed using a fast normalization fusion method to obtain three enhanced effective feature layers; S303: Input the three enhanced effective feature layers obtained in step S302 into the YOLO head module to predict the center point position of the PCB defect target, obtain the prediction box width and height, confidence and PCB defect target category information, and use the target box loss function Eiou loss as the loss function of the LDLFModel network model ζ EioU , improve the accuracy of prediction box positioning; S4: The trained LDLFModel network model is used to identify PCB defects and obtain defect detection results. The specific method is: the PCB defect data set to be detected is input into the cutting module for cutting, and the PCB defect data set to be detected is cut into small images, and there is a 20% overlapping area between the small images adjacent to the left and right or the upper and lower adjacent small images. The cut small images are input into the trained LDLFModel network model for PCB defect detection, and the small images with detection results are output. The small images with detection results are input into the restoration module for splicing and restoration to obtain the final visual detection results.

2. A PCB defect detection method based on LDLFModel according to claim 1, characterized in that, The specific method of step S1 is: randomly flipping, randomly mirroring, randomly changing the brightness, and randomly scaling the PCB dataset to expand the number of datasets, and then using Mosica data enhancement to obtain the final training data.

3. A PCB defect detection method based on LDLFModel according to claim 1, characterized in that, The position attention mechanism in step S2 is used to embed and generate coordinate information; the principle of coordinate information embedding is: For a given input feature x, a pooling kernel of size (H, 1) or (1, W) is used to encode each channel along the horizontal and vertical coordinates respectively, and the output is The specific expression is: Among them, W is the width of the pooling kernel, m is the value of the pooling kernel in the W spatial range, and x c (h,m) is the feature map input tensor with height h and W taking m, is the output of the cth channel with height h; The c-th pass output with width w The specific expression is: Among them, H is the height of the pooling kernel, n is the value of the pooling kernel in the H space range, x c (n,w) is the feature map input tensor with width w, H takes n, is the output of the cth channel with width w; The generation principle of coordinate information is: After the transformation in the coordinate information embedding, the transformed output result Perform Concat operation to get [z h ,z w ], and finally, the intermediate feature map f that encodes the spatial information in the horizontal and vertical directions is obtained through the nonlinear activation function δ. The specific expression is as follows: f=δ(F1([z h ,z w ])) Among them, F1 is a 1×1 convolution transformation function; The intermediate feature map f is then decomposed into two separate tensors f along the spatial dimension h and f w , and then use two 1×1 convolution transformation functions F h and F w And the nonlinear activation function σ, that is, the sigmoid function, respectively converts f h and f w Transformed into a tensor g with the same number of channels h and g w , the specific expression is as follows: g h =σ(F h (f h )) g w =σ(F w (f w )) Then the output y of the position attention mechanism is c The specific expression of (m,n) is: Among them, x c (m,n) is the number of channels, the width of the feature map is m, and the height of the feature map is n. A tensor with c as the number of channels, h as the height, and m as the width, A tensor with width w and height n, where c is the number of channels.

4. A PCB defect detection method based on LDLFModel according to claim 1, characterized in that, In step S302, the expression of fast normalization fusion is: Among them, O is the result of feature fusion, w i Represents the i-th input feature I i The corresponding learning weight, ∈, is a non-zero constant used to ensure the stability of the value; by i Perform a Relu operation to ensure w i The value of is greater than or equal to 0; The fusion of level x and level x+1 feature layers at level x is specifically expressed as: Among them, Resize is the up and down sampling operation, Conv is the convolution operation of the feature, w i Represents the i-th input feature I i The corresponding learning weight, w i+1 Represents the i+1th input feature I i+1 The corresponding learning weights, represents the feature output after the k-th level fusion, represents the feature input of the kth level, Represents the feature input of the k+1th level.

5. A PCB defect detection method based on LDLFModel according to claim 1, characterized in that, In step S303, the loss function ζ EioU The specific expression is: Among them, ρ(b p ,b gt ) is the Euclidean distance between the center point of the predicted box and the real box, ρ(w,w gt ) is the Euclidean distance between the predicted frame width and the true frame width, ρ(h,h gt ) is the Euclidean distance between the predicted box height and the real box height, t is the diagonal area of ​​the minimum circumscribed rectangle of the real box of the target box, T w is the width of the minimum bounding box, T h is the height of the minimum bounding box, IoU is the intersection-over-union ratio of the predicted box and the true box, A is the predicted box, and B is the true box.

6. A PCB defect detection method based on LDLFModel according to claim 1, characterized in that, In step S4, the cutting method of the cutting module is: Set the cutting coefficient mul. When mul=1, the cutting layer is closed. When mul>1, the cutting layer is opened. The relationship between the number of small images N and mul is: N=many 2 First, get the length x of the PCB image to be tested size He Kuany size ; Then calculate the length x_smoc and width y_smoc of the cutting box respectively: Then calculate the starting coordinates (x starpoint ,y starpoint ): Where p and q are natural numbers; Then calculate the relative coordinates of the last cutting position (x real ,y real ) x real =min(x starpoint +x_smoke,x size ) and real =min(y starpoint +y_smoc,y size ) Compare the length and width of the image to be detected with the relative coordinates of the last cutting position to determine whether the cutting is just completed. If it is not just completed, the last cutting frame position is taken as (x real ,y real ) to make the last cutting frame flush with the right border of the image to be detected; The specific method of the restoration module to restore all the cut small pictures is as follows: starpoint ,y starpoint ) The cutout images with detection results are spliced ​​together, non-maximum suppression is performed, and redundant frames are removed to obtain the final visual detection results.

Images