Circuit board welding defect detection method based on edge detection and target recognition

Abstract

The invention discloses a circuit board welding defect detection method based on edge detection and target recognition, which belongs to the technical field of welding defect detection and solves the problems of missed detection, false detection and inaccurate positioning caused by defect superposition, occlusion and interference of complex production environment in the process of circuit board welding defect detection. The invention extracts the edge of the circuit board image by adaptive median filtering to enhance the recognition and retention effect of the edge information of welding defects. A multi-level cross-feature fusion network for anti-occlusion defect feature extraction is constructed, and the anti-interference analysis ability of the model for defect features is enhanced by the FFE module and the context information extraction based on RFB-Net and dilated convolution. The detection frame positioning optimization method based on weighted repulsive loss function improves the reliability and efficiency of circuit board welding detection while reducing the calculation burden. The invention is suitable for circuit board welding defect detection in multi-process continuous production and can realize high-precision defect positioning and recognition in complex environments.

Description

Technical Field

[0001] The invention belongs to the technical field of welding defect detection, and relates to a circuit board welding defect detection method based on edge detection and target recognition. Background Art

[0002] In the modern electronic manufacturing process, the size of electronic products is getting smaller and smaller, and the performance requirements are getting higher and higher. As the core component of electronic equipment, the welding quality of circuit boards directly affects the performance and reliability of the whole machine. Welding defects such as cold solder joints, open solder joints, short circuits, excessive or insufficient solder, etc., will cause the circuit board to fail to function. In severe cases, it may even cause the complete failure of the equipment, which will have an adverse impact on the market performance of the product, the company's reputation and consumer safety. The traditional circuit board welding defect detection method mainly relies on manual inspection, that is, the operator visually inspects the solder joints with the help of a magnifying glass or microscope. This method is not only time-consuming and labor-intensive, but also has large subjective errors, which is prone to missed detection or false detection. In addition, due to the increasing complexity and integration of electronic products, modern circuit boards often contain thousands of solder joints, and the efficiency of manual inspection is far from meeting the needs of large-scale production.

[0003] In recent years, with the rapid development of computer vision and artificial intelligence technology, especially the application of deep learning technology, automated detection methods based on image recognition have gradually become an efficient and accurate solution. However, despite the huge potential of image recognition technology, its application in circuit board welding defect detection is still in its infancy. On the one hand, due to the large morphological differences between different types of welding defects, the training of deep learning models requires a large amount of accurately labeled data. On the other hand, external factors such as noise interference and changes in lighting conditions during image processing will also affect the detection results. Therefore, how to further improve the accuracy and efficiency of detection by optimizing algorithms, improving model robustness, and improving image acquisition quality is an important research direction in the current technical field.

[0004] Specifically, there are three key issues in the detection of circuit board welding defects: 1) The detection problem of defect superposition and occlusion: In the continuous production process of circuit boards, the superposition or occlusion of defects caused by different processes can easily affect the accuracy of detection. In addition, factors such as ambient light reflection and surface residues further increase the complexity of the detection system, resulting in high missed detection and false detection rates. Therefore, it is key to improve the system's ability to identify defects in complex environments. 2) The problem of edge information retention and noise interference: The edge of welding defects contains rich identification information, but the existing denoising method easily misjudges the noise as an edge, resulting in blurring of the detection results and loss of details. How to effectively remove noise and retain more real edge information is an important direction for improving detection accuracy. 3) The problem of efficiency and accuracy of multi-scale feature extraction: The comprehensive identification of welding defects depends on the effective extraction of multi-scale features. However, the existing feature extraction methods are prone to problems such as large computational burden and overfitting, and it is difficult to balance the requirements of high accuracy and high efficiency. Therefore, it is crucial to develop solutions that can efficiently extract multi-scale features while reducing computational overhead to meet the actual needs of industrial applications. Summary of the invention

[0005] The technical solution of the present invention is used to solve the problems of missed detection, false detection and inaccurate positioning caused by defect superposition, occlusion and interference from complex production environment during the circuit board welding defect detection process.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] The present invention provides a circuit board welding defect detection method based on edge detection and target recognition, comprising the following steps:

[0008] S1, perform homomorphic filtering on the input image circuit board welding image to limit the dynamic range of the image grayscale; use adaptive median filtering to process the noise;

[0009] S2, calculate the image gradient value, perform non-maximum suppression on the edge information, set the high threshold and low threshold of the gradient to perform threshold classification on all non-zero points that are not suppressed, and connect the classified edge points to obtain the edge information of the entire circuit board welding image;

[0010] S3, classifying and marking all defect information in the circuit board welding image containing edge information;

[0011] S4. Design the FFE module based on the YOLOv8 backbone network for feature extraction;

[0012] S5. Combining the RFB-Net structure and the dilated convolution structure, a multi-scale context information extraction module is constructed to obtain the characteristics of welding defects;

[0013] S6. Design a weighted repulsion loss function to locate and classify circuit board welding defects.

[0014] Furthermore, the method of performing homomorphic filtering on the input image circuit board welding image to limit the dynamic range of the image grayscale described in step S1 is specifically as follows:

[0015] Using S-curve to construct high-pass filter transfer function , the formula of the high-pass filter transfer function is as follows:

[0016] (1)

[0017] in, Indicates frequency point To filter center The distance is calculated as: , is the filter weight;

[0018] Applying the high-pass filter transfer function to the frequency domain image, the weighted frequency components are as follows:

[0019] (2)

[0020] in, is the original frequency domain image.

[0021] Furthermore, the method of processing the noise points by using adaptive median filtering in step S1 is specifically as follows:

[0022] Expand the image to the maximum window size according to the outermost boundary pixel value, using The size of the filter window is the point in the image Conduct traversal;

[0023] Mark the maximum grayscale value in each image window and the minimum value And calculate the gray median If you do not meet , then expand the filter window size to ;

[0024] If satisfied , judgment point Pixel value Is it satisfied? If it is satisfied, it means there is no noise in the window, and the original grayscale value is output. If it is not satisfied, the window is enlarged to ; This is done every time the window is enlarged judgment;

[0025] when When , the window expansion behavior is terminated and the minimum grayscale difference in the current window is calculated , the calculation formula is as follows:

[0026] (3)

[0027] Calculate the denoising threshold at the same time , the calculation formula is as follows:

[0028] (4)

[0029] in, is a constant coefficient;

[0030] if , the image grayscale value is not modified; if it does not meet , it is considered that there are noise points in the window, and the weighted filtering values ​​of all pixels in the window are calculated. , the calculation formula is as follows:

[0031] (5)

[0032] in, and They are the grayscale mean and grayscale median calculated after removing the grayscale extreme values, and m is the weight.

[0033] Furthermore, the method for calculating the image gradient value described in step S2 is specifically as follows:

[0034] Pixels The gradient values ​​in the x and y directions are:

[0035] (6)

[0036] (7)

[0037] That is The gradient value at and the gradient direction They are:

[0038] (8)

[0039] (9)

[0040] in, express The gray value at express The gray value at express The gray value at express The gray value at .

[0041] Furthermore, the method for non-maximum suppression of edge information described in step S2 is specifically as follows: determining a non-zero point in the gradient direction, and then searching for two adjacent points along the directional derivative of the point; since the edge is defined as the place where the brightness changes most dramatically in the image, if the amplitude of the two adjacent points exceeds the center point, then the non-zero point does not belong to the edge of the image, and its edge strength is set to 0;

[0042] The method of setting the high threshold and low threshold of the gradient to perform threshold classification on all unsuppressed non-zero points is specifically as follows: setting the high threshold and low threshold of the gradient, performing threshold classification on all unsuppressed non-zero points, setting the points with a gradient greater than the high threshold as strong edge points, and setting their pixel values ​​to 255; setting the points with gradient values ​​between the high and low thresholds as weak edge points, and keeping their pixel values ​​unchanged; and suppressing the points with pixel values ​​less than the low threshold, and setting their pixel values ​​to 0.

[0043] Furthermore, the method for designing an FFE module for feature extraction based on the YOLOv8 backbone network described in step S4 is as follows:

[0044] In the Bottleneck structure of the convolutional network, the jump connection between 1x1 convolution kernels is represented as:

[0045] (10)

[0046] Among them, p and q are the input and output features of the convolution module respectively. represents the learning parameters of the module, Function is the residual mapping function;

[0047] The formula of the FFE module is:

[0048] (11)

[0049] in, Represents the learning parameters of different branches in the module;

[0050] For the input feature map channel And the output feature map channels , the FFE module at each different depth Different numbers of convolution kernels are embedded in the branches of ,in and , Indicates the number of branches and the maximum convolution depth of the module; as the network depth increases, the number of group convolutions in the nth branch is set to ;

[0051] The final FFE module parameter count It is expressed as:

[0052] (12)

[0053] Floating point operations It is expressed as:

[0054] (13)

[0055] The parameters and computational size of the FFE module and the standard residual module are compared as follows:

[0056] (14)

[0057] (15)

[0058] in, and Represents the width and height of the feature map, represents the convolution kernel size, Indicates the number of input feature map channels, Indicates the number of output feature map channels corresponding to different depths.

[0059] Furthermore, the method described in step S5 for combining the RFB-Net structure and the dilated convolution structure to construct a multi-scale context information extraction module to obtain the characteristics of welding defects is as follows:

[0060] By introducing the expansion rate and using different spacing zero-filling operations during the convolution operation, the receptive field is increased without changing the size of the output feature map;

[0061] By setting different dilation rates, different feature information is extracted and combined to increase the diversity of context information;

[0062] When the dilation rate is 1, the dilated convolution has the same effect as the normal convolution. When the dilation rate is greater than 1, the convolution operation samples the element intervals in the feature map according to the dilation rate.

[0063] A normal convolution calculation is defined as:

[0064] (16)

[0065] A dilated convolution calculation is defined as:

[0066] (17)

[0067] in, is the input feature map, are the coordinates of the elements in the feature map, for Convolution kernel, Represented as input feature map and convolution kernel After the convolution operation The value at the point, is the expansion rate, and is the coordinate of the convolution kernel, and the range is ,in is the radius of the convolution kernel, Refers to the convolution kernel at position elements.

[0068] Furthermore, the method for locating and classifying the soldering defects of the circuit board by designing the weighted repulsion loss function described in step S6 is specifically as follows:

[0069] The repulsive force function is composed of the attractive term and exclusions , It consists of three parts:

[0070] (18)

[0071] in, Play an attractive role; and As an exclusion item; and As a weight factor, used to balance auxiliary losses;

[0072] The loss function is expressed as:

[0073] (19)

[0074] in, and Control the positioning accuracy of the prediction box in the model, Used to represent confidence loss, is the classification loss;

[0075] The positioning loss consists of CIoU loss and RepBoxLoss loss. The CIoU gap between the predicted coordinate point position and length and width of the rectangular box and the corresponding Groundtruth is calculated to train the model for the positioning of the predicted box size and position.

[0076] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above-mentioned circuit board welding defect detection method based on edge detection and target recognition, and the processor is configured to execute the program stored in the memory.

[0077] The present invention also provides a storage medium on which a computer program is stored. When the computer program is run by a processor, the steps of the circuit board welding defect detection method based on edge detection and target recognition are executed.

[0078] The advantages of the present invention are:

[0079] The present invention designs a circuit board image edge extraction method based on adaptive median filtering, which greatly reduces the influence of complex electronic component structures on detection results while enhancing the recognition and retention of welding defect edge information; constructs a multi-level cross-feature fusion network for anti-occlusion defect feature extraction, and effectively enhances the model's anti-interference analysis capability for defect features by implementing multi-width and multi-depth fusion feature extraction modules (FFE modules) and context information extraction based on RFB-Net and dilated convolution, especially when the defects are occluded, it can still maintain a high detection accuracy and avoid the omission of minor defects, meeting the needs of large-scale detection in circuit board industrial production; designs a detection frame positioning optimization method based on weighted repulsive loss function, which reduces the computational burden and improves the reliability and efficiency of circuit board welding detection; the present invention is suitable for circuit board welding defect detection in multi-process continuous production, and can achieve high-precision defect positioning and identification in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is an overall flow chart of a circuit board welding defect detection method based on edge detection and target recognition according to Embodiment 1 of the present invention;

[0081] Figure 2 A diagram of a multi-scale feature extraction module (FFE module) of a circuit board welding defect detection method based on edge detection and target recognition according to Embodiment 1 of the present invention;

[0082] Figure 3 This is a multi-scale context information extraction module diagram of a circuit board welding defect detection method based on edge detection and target recognition according to the first embodiment of the present invention;

[0083] Figure 4 Schematic diagram of hole convolution expansion of the circuit board welding defect detection method based on edge detection and target recognition according to the first embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments:

[0086] Embodiment 1

[0087] like Figure 1 As shown, the circuit board welding defect detection method based on edge detection and target recognition according to an embodiment of the present invention comprises the following steps:

[0088] Step 1: PCB welding image preprocessing

[0089] Step 1.1: In order to eliminate the influence of noise in the original welding image and enhance the overall image quality, the input image needs to be grayscaled and filtered at the beginning, and the dynamic range of the image grayscale is limited by homomorphic filtering.

[0090] First, use the S curve to construct a high-pass filter transfer function , which represents the degree of processing of different frequencies in the frequency domain, suppresses low-frequency brightness information and enhances high-frequency detail information. The formula of the high-pass filter transfer function is as follows:

[0091] (1)

[0092] in, Indicates frequency point To filter center The distance is calculated as: The smaller the distance, the larger the value of the high-pass filter transfer function, and the stronger the suppression effect on the low-frequency part; the larger the distance, the smaller the value of the high-pass filter transfer function, and the more obvious the enhancement effect on the high-frequency part; is the filter weight, which affects the strength of the filter and is usually set to 3.

[0093] Applying the high-pass filter transfer function to the frequency domain image, the weighted frequency components are as follows:

[0094] (2)

[0095] in, is the original frequency domain image; the image after homomorphic filtering It can keep the brightness uniform and improve the contrast of the overall image.

[0096] Step 1.2: Use adaptive median filtering to process noise points

[0097] Expand the image to the maximum window size according to the outermost boundary pixel value, using The size of the filter window is the point in the image Traverse; (the initial value of n is usually 3). Mark the maximum value of grayscale in each image window and the minimum value And calculate the gray median If you do not meet , then expand the filter window size to Under the above premise, the judgment point Pixel value Is it satisfied? If it is satisfied, it means there is no noise in the window and the original grayscale value can be output. If it is not satisfied, the window will continue to be enlarged to . This is done every time the window is enlarged. This step is to limit the maximum window range of the filter.

[0098] when When , the window expansion behavior is terminated and the minimum grayscale difference in the current window is calculated , the calculation formula is as follows:

[0099] (3)

[0100] Calculate the denoising threshold at the same time , the calculation formula is as follows:

[0101] (4)

[0102] in, is a constant coefficient, usually taken as 0.5.

[0103] if , the image grayscale value is not modified; if it does not meet , it is considered that there are noise points in the window, and the weighted filtering values ​​of all pixels in the window are calculated. , the calculation formula is as follows:

[0104] (5)

[0105] in, and They are the grayscale mean and grayscale median calculated after removing the grayscale extreme values, and m is the weight, which is usually 0.2.

[0106] Repeat the above step 1.2 until all the pixels have been traversed, and then delete the peripheral pixel values ​​expanded in step 1.2. The above method can retain most of the non-noise extreme points, while eliminating the influence of noise and avoiding the smoothing of pixel information, thus retaining more image detail information.

[0107] Step 2: Extract edge information of circuit board welding image

[0108] Step 2.1: In order to realize edge extraction, it is necessary to calculate the image gradient value. The gradient direction represents the direction of change of image pixel intensity.

[0109] Pixels The gradient values ​​in the x and y directions are:

[0110] (6)

[0111] (7)

[0112] That is The gradient value at and the gradient direction They are:

[0113] (8)

[0114] (9)

[0115] in, express The gray value at express The gray value at express The gray value at express The gray value at .

[0116] Step 2.2: Since edge information is judged only by gradient information, it will be affected by the pixel value of noise in the image, resulting in erroneous edge information extraction. Therefore, after determining the gradient, it is necessary to perform non-maximum suppression on the edge information. The operation process of non-maximum suppression is to first determine the non-zero point in the gradient direction, and then find two adjacent points along the directional derivative of the point; since the edge is usually defined as the place where the brightness changes most dramatically in the image, if the amplitude of the two adjacent points exceeds the center point, then this non-zero point does not belong to the edge of the image, and may be a misjudgment under the influence of noise rather than the actual edge position, then its edge strength is set to 0; by traversing all non-zero points, only real and useful edge information is retained in the end, which can effectively suppress the existence of pseudo-edges.

[0117] Step 2.3: In order to determine the potential edge information, first set a high threshold and a low threshold for the gradient, and perform threshold classification on all the non-zero points that were not suppressed in step 2.2. For those with gradients greater than the high threshold, they are set as strong edge points, and their pixel values ​​are set to 255; for those with gradient values ​​between the high and low thresholds, they are set as weak edge points, and their pixel values ​​remain unchanged; and for points with pixel values ​​less than the low threshold, they are suppressed and their pixel values ​​are set to 0.

[0118] Step 2.4: Connect all weak edge points determined in step 2.3 with strong edge points. If there is a strong edge point within the 8-pixel area of ​​a weak edge point, set its pixel value to 255 and retain it. If there are no other edge points around the point, it is judged as an isolated point and its pixel value is set to 0 for suppression. Finally, the edge information of the entire circuit board welding image can be obtained.

[0119] Step 3: Identification of circuit board welding defects

[0120] Step 3.1: In order to obtain the identification information of all circuit board welding defects, it is necessary to classify and annotate all defect information in the circuit board welding image containing edge information finally obtained in step 2. These include defects such as cold welding, virtual welding, bridging, solder cracks, and solder bubbles. After the annotation is completed, it is divided into a training set, a test set, and a validation set. The test set is trained using the YOLOv8 target detection algorithm.

[0121] Step 3.2: Figure 2 As shown in the figure, a multi-width and multi-depth fusion feature extraction module (FFE module) is designed based on the YOLOv8 backbone network to optimize feature extraction. By embedding convolutions of different receptive fields and depths in the convolution layer, multi-scale and multi-level welding defect features are extracted, and the comprehensive recognition ability of the model is improved through channel splicing.

[0122] The specific design idea of ​​the FFE module is to express the jump connection between 1x1 convolution kernels in the Bottleneck structure of the convolutional network as follows:

[0123] (10)

[0124] Among them, p and q are the input and output features of the convolution module respectively. represents the learning parameters of the module, The function is a residual mapping function; it consists of three convolutions with kernel sizes of 1x1, 3x3 and 1x1. The first 1x1 convolution is used to reduce the channel dimension, the 3x3 convolution is mainly used to extract features, and the second 1x1 convolution is responsible for restoring the channel dimension to the initial state. The 3x3 convolution is improved using multiple learnable convolution branches of different depths. The entire FFE module can be formulated as:

[0125] (11)

[0126] in, Represents the learning parameters of different branches in the FFE module.

[0127] The FFE module is designed to extract features of different sizes and shapes with less computation and parameters. Specifically, the FFE module divides the convolution into n groups and then connects the feature maps generated by different branches in series for subsequent processing. The input feature map is cut into different branches to improve the model in the width direction; by embedding different numbers of convolutional layers in each branch, the model can be encouraged to learn texture features and semantic features of different depths, thereby enhancing the model's visual feature representation ability.

[0128] For the input feature map channel And the output feature map channels , the FFE module at each different depth Different numbers of convolution kernels are embedded in the branches of ,in and , Indicates the number of branches and the maximum convolution depth of the module; as the network depth increases, the number of group convolutions in the nth branch is set to .

[0129] The final FFE module parameter count It can be expressed as:

[0130] (12)

[0131] in, represents the convolution kernel size, Indicates the number of input feature map channels, Indicates the number of output feature map channels corresponding to different depths.

[0132] Floating point operations It can be expressed as:

[0133] (13)

[0134] in, and Represents the width and height of the feature map.

[0135] The parameters and computational size of the FFE module and the standard residual module are compared as follows:

[0136] (14)

[0137] (15)

[0138] The FFE module designed above can extract multi-scale and multi-level welding defect features while also reducing unnecessary parameters and computational complexity.

[0139] Step 3.3: In order to better obtain defect features and context information and balance the environmental understanding between foreground and background, the following is constructed by combining the RFB-Net structure and the dilated convolution structure: Figure 3 The multi-scale context information extraction module shown in the figure enhances the model's understanding and balance of global features and foreground information in complex production environments, and effectively obtains the characteristics of welding defects. By introducing the expansion rate, different spacing zero-filling operations are used during the convolution operation to increase the receptive field without changing the output feature map size, replacing the information loss caused by repeated pooling operations. By setting different expansion rates to extract different feature information, combining them increases the diversity of context information. When the expansion rate is 1, the atrous convolution has the same effect as the ordinary convolution. When the expansion rate is greater than 1, the convolution operation samples the element intervals in the feature map according to the expansion rate.

[0140] A normal convolution calculation is defined as:

[0141] (16)

[0142] A dilated convolution calculation is defined as:

[0143] (17)

[0144] in, is the input feature map, are the coordinates of the elements in the feature map, for Convolution kernel, Represented as input feature map and convolution kernel After the convolution operation The value at the point, is the expansion rate, and is the coordinate of the convolution kernel, and the range is ,in is the radius of the convolution kernel, Refers to the convolution kernel at position elements.

[0145] like Figure 4 As shown, it is a schematic diagram of the expansion of the dilated convolution used in the present invention. After the dilated convolution operation, the receptive field area increases, indicating that the dilated convolution operation can expand the receptive field without losing feature resolution and keep the relative spatial position of pixels unchanged. When multiple dilated convolutions with different expansion rates are superimposed, multi-scale contextual information can be obtained. Compared with the ordinary convolution operation of convolution followed by pooling, the dilated convolution can introduce no additional hyperparameters and reduce the amount of computation. Therefore, for the detection of occluded objects, it can help improve the global and local representation of features and improve detection performance.

[0146] Considering the effectiveness of dilated convolution in improving feature expression during occluded target detection, the present invention combines dilated convolution with the multi-branch convolution structure of RFB-Net to construct a multi-branch context information defect feature extraction unit in the backbone network, which can better capture the context information in the feature map and avoid the negative impact of redundant noise in the background on the detection performance. The context information feature extraction unit based on RFB-Net adopts a multi-branch convolution structure, which stacks convolution layers and dilated convolutions of different sizes and numbers on different branches, and combines feature maps of different resolutions and different expansion rates to compensate for the information loss between different scales. At the same time, different expansion degrees of different branches can obtain context information of different regions.

[0147] Step 3.4, the YOLOv8 method detects defects by regression, and uses bounding box regression to reduce the difference between the predicted box and the true box. To ensure that the anchor boxes of different scales can match the target more accurately, the YOLOv8 model measures the matching degree between the anchor boxes of each grid and the target through classification loss and positioning loss. However, the detection of soldering defects in circuit boards is different from conventional target detection. Conventional targets usually have complete features and sparse position distribution, while soldering defects often overlap and occlude each other, resulting in significant differences between the predicted feature vector and the actual target feature. In addition, the color features of defects on the surface of circuit boards are single and the gap between classes is small. Especially for regional defects that require global analysis, the YOLOv8 model tends to include overlapping or dense defects in the same prediction box during training. This situation often causes some features of the occluded defects to be confused with the occluders, making it impossible for the model to accurately distinguish the features in the overlapping area, which in turn causes the prediction box to shift. Simply relying on positioning loss and classification loss cannot fully solve this problem, and it is difficult for the model to accurately locate and classify these overlapping defects. In order to enhance the model's learning ability to distinguish similar objects when detecting occlusion problems and its ability to accurately locate objects under background and dense target interference, the attraction and repulsion idea of ​​repulsion loss is combined with the prediction box generation strategy of the YOLOv8 model to design a weighted repulsion loss function to help the model accurately locate and classify circuit board welding defects. The repulsion function consists of the attraction term and exclusions , It consists of three parts:

[0148] (18)

[0149] in, It plays an attractive role, making the prediction box close to its specified target. and As a repulsive term, the predicted box is kept away from the Groundtruth of other surrounding objects and other target prediction boxes with different categories; and As a weight factor, it is used to balance the auxiliary loss. The overall loss function of the model based on the idea of ​​the repulsion function can be expressed as:

[0150] (19)

[0151] in, and Control the positioning accuracy of the prediction box in the model, Used to represent confidence loss, is the classification loss.

[0152] The positioning loss is composed of CIoU loss and RepBoxLoss loss. The CIoU gap between the predicted coordinate point position and length and width of the rectangular box and the corresponding Groundtruth is calculated to train the model for the positioning of the predicted box size and position. The final detection box trained in this way can greatly avoid defect overlap or dense interference, and the final detection box is closer to the actual defect range.

[0153] In the multi-process continuous production of circuit boards, the superposition and occlusion of welding defects, as well as the interference of ambient light source reflection and board residues in complex production environments, easily lead to missed detection, false detection and inaccurate positioning in the defect detection process. These problems directly affect the accuracy and reliability of circuit board welding defect detection, and the missed detection rate requirement of industrial production for defect detection is usually higher than the accuracy of defect classification. The present invention provides an end-to-end YOLOv8 anti-occlusion welding defect detection method. By constructing a context information extraction structure based on RFB-Net and a multi-level cross feature fusion structure, the method of the present invention can effectively enhance the network's anti-interference analysis ability for welding defect features, especially when the defects are occluded or superimposed, it can still maintain a high detection accuracy. In addition, the present invention also proposes a multi-width and multi-depth fusion feature extraction module (FFE module) based on a convolutional neural network to enhance the ability to capture comprehensive features in the task of circuit board welding defect detection. The module can effectively extract multi-scale and multi-level features of welding defects by embedding convolution branches with different receptive fields and depths in a single convolution layer, avoiding missing potential minor defects. These features are spliced ​​in the channel dimension to improve the model's ability to identify different types of defects. In order to reduce the common overfitting problems and high computational overhead in industrial applications, the FFE module introduces efficient computing modules such as group convolution and hole convolution. This design can significantly improve the performance of the model in welding defect detection tasks while ensuring a low number of parameters and computational complexity. In order to optimize the accurate positioning capability of the prediction frame, the present invention designs a detection frame generation method based on a weighted repulsion loss function to ensure that the system improves the reliability and accuracy of detection while operating efficiently.

[0154] In the detection of welding defects in circuit boards, it is also crucial to obtain defect edge information. The edge of the welding defect is the transition area between the defect area and the normal surface, which contains rich feature information. If the edge information is lost or blurred during the image processing process, it will directly affect the accuracy and reliability of defect detection. Therefore, it is necessary to develop a denoising method that can retain more edge information while removing noise for subsequent defect edge detection and recognition. Traditional denoising methods usually rely on image gradients to detect edges, but because the gradient value is easily disturbed by noise, areas with large noise are often mistakenly identified as edges, resulting in blurred edges and loss of detail information. In response to this problem, the present invention proposes an edge recognition algorithm based on adaptive median filtering, which is specifically used to enhance the detection effect of the edges and texture details of welding defects in circuit boards. Through this method, more accurate defect edge information can be retained while removing image noise, making the detected defects clearer. This method can significantly improve the edge and texture details of welding defect images, improve the accuracy and reliability of overall defect detection, and provide a more effective technical means for circuit board detection in industrial production.

[0155] Test verification

[0156] In order to verify the effectiveness of the circuit board welding defect detection method based on edge detection and target recognition in the actual circuit board welding defect detection, a data set of circuit board welding defects was first constructed, which includes five types of welding defects: cold solder joints, cold solder joints, open solder joints, short circuits, and solder anomalies. As shown in Table 1, the performance of YOLOv7, SSD, YOLOv5m, Faster R-CNN, and YOLOv8 on the data set is compared. The results show that the method of the present invention is significantly better than other algorithms in terms of accuracy and recall, reaching 95.3% and 93.8% respectively. And the average processing time of the method of the present invention is 37 milliseconds per frame, which fully meets the needs of real-time fault detection.

[0157] Table 1 Comparison of accuracy, recall and processing time of different algorithms on the dataset

[0158]

[0159] Embodiment 2

[0160] An electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the circuit board welding defect detection method based on edge detection and target recognition in Example 1, and the processor is configured to execute the program stored in the memory.

[0161] Embodiment 3

[0162] A storage medium stores a computer program, and when the computer program is run by a processor, the steps of the circuit board welding defect detection method based on edge detection and target recognition in embodiment 1 are executed.

[0163] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A circuit board welding defect detection method based on edge detection and target recognition, characterized in that: The following steps are involved: S1, perform homomorphic filtering on the input image circuit board welding image to limit the dynamic range of the image grayscale; use adaptive median filtering to process the noise; S2, calculate the image gradient value, perform non-maximum suppression on the edge information, set the high threshold and low threshold of the gradient to perform threshold classification on all non-zero points that are not suppressed, and connect the classified edge points to obtain the edge information of the entire circuit board welding image; S3, classifying and marking all defect information in the circuit board welding image containing edge information; S4. Design the FFE module based on the YOLOv8 backbone network for feature extraction. The specific method is as follows: In the Bottleneck structure of the convolutional network, the jump connection between 1x1 convolution kernels is represented as: （10） Among them, p and q are the input and output features of the convolution module respectively. represents the learning parameters of the module, Function is the residual mapping function; The formula of the FFE module is: （11） in, Represents the learning parameters of different branches in the module; For the input feature map channel And the output feature map channels , the FFE module at each different depth Different numbers of convolution kernels are embedded in the branches of ,in and , Indicates the number of branches and the maximum convolution depth of the module; as the network depth increases, the number of group convolutions in the nth branch is set to ; The final FFE module parameter count It is expressed as: （12） Floating point operations It is expressed as: （13） The parameters and computational size of the FFE module and the standard residual module are compared as follows: （14） （15） in, and Represents the width and height of the feature map, represents the convolution kernel size, Indicates the number of input feature map channels, Indicates the number of output feature map channels corresponding to different depths; S5. Combining the RFB-Net structure and the dilated convolution structure, a multi-scale context information extraction module is constructed to obtain the characteristics of welding defects; S6. Design a weighted repulsion loss function to locate and classify circuit board welding defects.

2. The circuit board welding defect detection method based on edge detection and target recognition according to claim 1 is characterized in that: The method described in step S1 of performing homomorphic filtering on the input image circuit board welding image to limit the dynamic range of the image grayscale is specifically as follows: Using S-curve to construct high-pass filter transfer function , the formula of the high-pass filter transfer function is as follows: （1） in, Indicates frequency point To filter center The distance is calculated as: , is the filter weight; Applying the high-pass filter transfer function to the frequency domain image, the weighted frequency components are as follows: （2） in, is the original frequency domain image.

3. The circuit board welding defect detection method based on edge detection and target recognition according to claim 1 is characterized in that: The method for processing noise points by using adaptive median filtering described in step S1 is specifically as follows: Expand the image to the maximum window size according to the outermost boundary pixel value, using The size of the filter window is the point in the image Conduct traversal; Mark the maximum grayscale value in each image window and minimum value And calculate the gray median If you do not meet , then expand the filter window size to ; If satisfied , judgment point Pixel value Is it satisfied? If it is satisfied, it means there is no noise in the window, and the original grayscale value is output. If it is not satisfied, the window is enlarged to ; This is done every time the window is enlarged judgment; when When , the window expansion behavior is terminated and the minimum grayscale difference in the current window is calculated , the calculation formula is as follows: （3） Calculate the denoising threshold at the same time , the calculation formula is as follows: （4） in, is a constant coefficient; if , the image grayscale value is not modified; if it does not meet , it is considered that there are noise points in the window, and the weighted filtering values ​​of all pixels in the window are calculated. , the calculation formula is as follows: （5） in, and They are the grayscale mean and grayscale median calculated after removing the grayscale extreme values, and m is the weight.

4. The circuit board welding defect detection method based on edge detection and target recognition according to claim 1 is characterized in that: The method for calculating the image gradient value described in step S2 is specifically as follows: Pixels The gradient values ​​in the x and y directions are: （6） （7） That is The gradient value at and the gradient direction They are: （8） （9） in, express The gray value at express The gray value at express The gray value at express The gray value at .

5. The circuit board welding defect detection method based on edge detection and target recognition according to claim 1 is characterized in that: The method for non-maximum suppression of edge information described in step S2 is specifically as follows: determining a non-zero point in the gradient direction, and then searching for two adjacent points along the directional derivative of the point; since the edge is defined as the place where the brightness changes most dramatically in the image, if the amplitude of the two adjacent points exceeds the center point, then the non-zero point does not belong to the edge of the image, and its edge strength is set to 0; The method of setting the high threshold and low threshold of the gradient to perform threshold classification on all unsuppressed non-zero points is specifically as follows: setting the high threshold and low threshold of the gradient, performing threshold classification on all unsuppressed non-zero points, setting the points with a gradient greater than the high threshold as strong edge points, and setting their pixel values ​​to 255; setting the points with gradient values ​​between the high and low thresholds as weak edge points, and keeping their pixel values ​​unchanged; and suppressing the points with pixel values ​​less than the low threshold, and setting their pixel values ​​to 0.

6. The circuit board welding defect detection method based on edge detection and target recognition according to claim 1 is characterized in that: The method described in step S5 for combining the RFB-Net structure and the dilated convolution structure to construct a multi-scale context information extraction module to obtain the characteristics of welding defects is as follows: By introducing the expansion rate and using different spacing zero-filling operations during the convolution operation, the receptive field is increased without changing the size of the output feature map; By setting different dilation rates, different feature information is extracted and combined to increase the diversity of context information; When the dilation rate is 1, the dilated convolution has the same effect as the normal convolution. When the dilation rate is greater than 1, the convolution operation samples the element intervals in the feature map according to the dilation rate. A normal convolution calculation is defined as: （16） A dilated convolution calculation is defined as: （17） in, is the input feature map, are the coordinates of the elements in the feature map, for Convolution kernel, Represented as input feature map and convolution kernel After the convolution operation The value at the point, is the expansion rate, and is the coordinate of the convolution kernel, and the range is ,in is the radius of the convolution kernel, Refers to the convolution kernel at position elements.

7. The circuit board welding defect detection method based on edge detection and target recognition according to claim 1 is characterized in that: The method for locating and classifying circuit board welding defects by designing a weighted repulsion loss function described in step S6 is specifically as follows: The repulsive force function is composed of the attractive term and exclusions , It consists of three parts: （18） in, Play an attractive role; and As an exclusion item; and As a weight factor, used to balance auxiliary losses; The loss function is expressed as: （19） in, and Control the positioning accuracy of the prediction box in the model, Used to represent confidence loss, is the classification loss; The positioning loss consists of CIoU loss and RepBoxLoss loss. The CIoU gap between the predicted coordinate point position and length and width of the rectangular box and the corresponding Groundtruth is calculated to train the model for the positioning of the predicted box size and position.

8. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the circuit board welding defect detection method based on edge detection and target recognition as described in any one of claims 1 to 7, and the processor is configured to execute the program stored in the memory.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the circuit board welding defect detection method based on edge detection and target recognition according to any one of claims 1 to 7 are executed.

Images