Welding quality detection method and device based on machine vision

Abstract

The invention provides a welding quality detection method and device based on machine vision, and belongs to the technical field of machine vision and intelligent detection.The method comprises the steps that a to-be-detected target image is obtained, and the to-be-detected target image is an image containing a printed circuit board; a target image to be detected is input into the defect detection model for defect detection, a defect detection result is obtained, and the defect detection result is used for representing the defect type of the printed circuit board. According to the welding quality detection method and device based on machine vision, the accuracy of welding defect detection can be improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of machine vision and intelligent detection, and more particularly to a welding quality detection method and device based on machine vision. BACKGROUND

[0002] As a core component of electronic devices, the welding quality of a printed circuit board (PCB) is directly related to the performance and reliability of the product. Welding defects such as virtual welding, abnormal soldering, and bridging may cause circuit failure or even safety accidents. The existing PCB welding quality detection has a high false detection rate and is difficult to meet the needs of large-scale production. Machine vision and traditional machine learning methods based on rules rely on manual feature setting and threshold setting, and have poor adaptability to complex scenes. Therefore, there is an urgent need for a high-precision and strong generalization capability PCB welding quality detection technology to meet the automation needs of modern electronic manufacturing. SUMMARY

[0003] The purpose of the present application is to provide a welding quality detection method and device based on machine vision to improve the accuracy and reliability of machine vision in PCB welding quality detection.

[0004] The first aspect of the embodiment of the present application provides a welding quality detection method based on machine vision, comprising: acquiring a target image to be detected, the target image to be detected being an image containing a printed circuit board; inputting the target image to be detected into a defect detection model for defect detection to obtain a defect detection result, the defect detection result being used to represent the defect type of the printed circuit board; The method of defect detection comprises: performing initial feature extraction on the target image to be detected to obtain initial image features; performing multi-scale feature extraction on the initial image features to obtain a plurality of high-level semantic features of different scales; performing feature fusion on the plurality of high-level semantic features of different scales to obtain fused features; performing defect detection based on the fused features to obtain the defect detection result.

[0005] The second aspect of the embodiment of the present application provides a welding quality detection device based on machine vision, comprising: an image acquisition module configured to acquire a target image to be detected, the target image to be detected being a printed circuit board; a quality detection module configured to input the target image to be detected into a defect detection model to obtain a defect detection result, the defect detection result being used to represent the defect type of the printed circuit board; The method of defect detection comprises: perform initial feature extraction on the target image to be detected to obtain initial image features; perform multi-scale feature extraction on the initial image features to obtain a plurality of high-level semantic features of different scales; perform feature fusion on the plurality of high-level semantic features of different scales to obtain fused features; perform defect detection based on the fused features to obtain a defect detection result.

[0006] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the welding quality detection method based on machine vision when executing the computer program.

[0007] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the welding quality detection method based on machine vision when executed by a processor.

[0008] The welding quality detection method and device based on machine vision provided in the embodiments of the present application have the following advantages: the embodiments of the present application significantly improve the accuracy and efficiency of welding quality detection by performing a systematic defect detection process on a target image containing a printed circuit board. Specifically, the embodiments of the present application capture basic image information through initial feature extraction, and then obtain high-level semantic features of different scales through multi-scale feature extraction, which not only retains the detailed information of small defects but also covers the semantic information of the overall morphology of defects. Through the fusion of multi-scale features, the complementary and enhancement of different levels of features are realized, so that the fused features can more comprehensively represent various welding defects. Finally, defect detection is performed based on the fused features, which effectively improves the recognition accuracy of various welding defects of the printed circuit board, such as pinhole, pad loss, and pin skew.

[0009] Meanwhile, the embodiments of the present application enhance the adaptability to defects of different sizes and types through multi-scale feature processing and fusion, significantly improve the robustness of detection, and can meet the high-quality detection needs in large-scale production of printed circuit boards. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0011] Figure 1A flowchart of a welding quality detection method based on machine vision provided by an embodiment of the present application is shown in FIG. 1. Figure 2 A structural block diagram of a welding quality detection device based on machine vision provided by an embodiment of the present application is shown in FIG. 2. Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 3. DETAILED DESCRIPTION

[0012] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, persons having ordinary skill in the art will readily recognize that embodiments of the application can be practiced without these specific details. In other instances, well-known structures, devices, circuits, and processes have not been described in detail so as not to unnecessarily obscure aspects of the application.

[0013] In order to make the objects, technical solutions and advantages of the present application clearer, the following will be described in detail with reference to the accompanying drawings.

[0014] Reference will be made to Figure 1 , Figure 1 A flowchart of a welding quality detection method based on machine vision provided by an embodiment of the present application is shown in FIG. 1. The method can include: S101: obtaining a target image to be detected, the target image to be detected being an image containing a printed circuit board.

[0015] In the present embodiment, in order to overcome the limitations of a single light source in PCB welding detection, such as reflection, shadow, low contrast, etc., a multi-spectral, multi-angle composite light source illumination system can be adopted, combined with intelligent light control strategies, to ensure high-quality imaging of solder joints, pads and traces.

[0016] Specifically, a ring-shaped LED array can be used as a main light source to provide uniform basic illumination for the vision system and reduce shadow interference. Coaxial light or low-angle ring light can be used in the layout to highlight the three-dimensional topography of the solder joints. Secondly, a linear polarizing plate is installed in front of the light source, and a cross-polarized filter is configured at the camera end to eliminate the specular reflection of the PCB surface metal and enhance the visibility of the solder wetting edge.

[0017] In the present embodiment, a high-precision area array camera (resolution ≥ 1392x1040) can be used, cooperating with an 8mm low-distortion industrial lens, to realize a continuous shooting capability of 30 frames per second, and a shooting pipeline obtains the PCB image to be detected. At least three groups of visual imaging structures can be arranged on both sides of the pipeline, uniformly distributed around the PCB at 180°, to ensure full coverage imaging of the PCB. For example, the camera module can be equipped with an adjustable angle holder, supporting ±15° pitch angle adjustment, suitable for PCBs of different areas.

[0018] S102: inputting the target image to be detected into a defect detection model for defect detection, to obtain a defect detection result, which represents the defect type of the printed circuit board.

[0019] The defect detection method comprises: performing initial feature extraction on the target image to be detected to obtain initial image features; performing multi-scale feature extraction on the initial image features to obtain a plurality of high-level semantic features of different scales; performing feature fusion on the plurality of high-level semantic features of different scales to obtain fused features; performing defect detection based on the fused features to obtain a defect detection result.

[0020] In the present embodiment, the defect detection result represents the defect detection type. The defect types of the printed circuit board can be divided into four categories, including: small size defects, color difference defects, large size defects, round hole defects, and other types of defects.

[0021] Specifically, the small size defects can include: missing holes, hair, and narrow wire spacing, etc. The color difference defects can include: character defects and copper plate color difference, etc. The large size defects can include: large size mis-etching, multiple holes, missing lines, and large size shielding, etc. The round hole defects can include: offset holes, pinholes, and hole impurities covering, etc. The other types of defects can include: open circuits, short circuits, burrs, and mouse bites, etc. For example, the large size mis-etching is caused by soldering bubbles or improper curing of solder. The open circuit can be caused by hard objects on the cutting machine or uneven chemical concentration of the hole forming agent, resulting in incomplete connection of the circuit. The mouse bite can be caused by uneven soldering or sharp objects on the machine, which can cause the circuit board to be open or other faults. In addition, operation errors or equipment failures are also common causes of short circuits, which can cause product scrap. Hair is mainly caused by not using a hole screen or excessive ink residue during the solder mask process, which affects the appearance and function of the circuit board. Burrs are usually caused by using blunt scissors or improper installation of scissors when cutting the PCB, which affects signal transmission. Short circuits are usually caused by running out of solder or incomplete etching, which is often caused by using improper stripping chemicals or improper etching chemical ratio, which can damage the circuit board.

[0022] In the embodiment, the target image to be detected can be a PCB image, and the defect detection model can be trained based on a yolov8 network.

[0023] For example, the embodiment can use a DWR module and an EMA attention mechanism to perform initial feature extraction on the target image to be detected, thereby improving the extraction capability of multi-scale features of PCB defects. The DWR module includes an RR module, which performs initial image feature extraction on the target image to be detected to obtain initial image features.

[0024] In the embodiment, the RS module can be selected to perform high-level semantic feature extraction on the initial image features. Specifically, the RS module includes three different convolution kernels, which can group and convolve the input initial image features using different expansion rates to finally obtain a plurality of high-level semantic features of different scales.

[0025] In the embodiment, the improved BiFPN can be used to fuse the plurality of high-level semantic features of different scales. Specifically, based on the original top-down path, a transverse cross-layer connection is added to enable direct interaction between shallow high-resolution features and deep semantic features, thereby improving the detail retention capability of the micro solder joints.

[0026] As can be seen from the above, the embodiment significantly improves the accuracy and efficiency of the solder quality detection by performing a systematic defect detection process on the target image containing the printed circuit board. Specifically, the embodiment captures the basic information of the image through initial feature extraction, and then obtains high-level semantic features of different scales through multi-scale feature extraction, which not only retains the detailed information of the micro defects but also covers the semantic information of the overall shape of the defects. Through the fusion of multi-scale features, the complementary and enhancement of different levels of features are realized, so that the fused features can more comprehensively represent various solder defects. Finally, based on the fused features, the recognition accuracy of various solder defects (such as solder joint pinholes, pad missing, and pin skewing) of the printed circuit board is effectively improved.

[0027] Meanwhile, the embodiment enhances the adaptability to defects of different sizes and types through multi-scale feature processing and fusion, significantly improves the robustness of the detection, and can meet the high-quality detection needs in large-scale production of printed circuit boards.

[0028] In an embodiment of the present application, the defect detection model includes a convolution layer; initial feature extraction is performed on the target image to be detected to obtain initial image features, including: performing initial feature extraction on the target image to be detected through the convolution layer to obtain initial image features.

[0029] In this embodiment, first, the PCB image to be detected is input into a 3x3 convolution layer, and a multi-channel feature map is generated through convolution operation. Then, a normalization processing mechanism is introduced to effectively solve the internal covariate shift problem commonly existing in deep neural networks, and to create conditions for stable training of subsequent network layers. Then, a ReLU activation function is used to perform nonlinear transformation on the features, which has the dual advantages of improving feature sparsity and alleviating gradient vanishing problem. The formula can be expressed as:

[0030] wherein, is the initial image feature, is the input data, and are the activation function and the convolution operation, respectively, is the normalization operation.

[0031] Specifically, from the perspective of feature sparsity optimization, the sparse neuron mechanism constructed by the ReLU activation function is essentially an efficient feature selection strategy. In actual operation, neurons with zero activation output can accurately filter out irrelevant background information such as PCB substrate and silk-screen text, while neurons that maintain a non-zero activation state focus on key detection areas such as solder joints and pins, significantly improving the robustness of the model by suppressing irrelevant responses. For the gradient vanishing problem, its root cause lies in the exponential decay of gradient in each layer during backpropagation, which makes it difficult to effectively update the parameters of deep network. The ReLU activation function introduces a one-sided suppression linear activation characteristic, keeping the gradient constant at 1 in the positive interval, effectively alleviating the gradient decay phenomenon caused by traditional activation functions such as Sigmoid and Tanh in deep networks, and providing feasibility support for building deeper neural network structures.

[0032] As can be seen from the above, the convolution kernel size of the embodiment is optimized and designed, which can effectively suppress noise interference while preserving the geometric details of the solder joints, significantly improving the model training convergence speed and laying a foundation for stable training of the subsequent multi-scale feature extraction module.

[0033] In an embodiment of the present application, multi-scale feature extraction is performed on the initial image feature to obtain a plurality of high-level semantic features of different scales, including: Based on a plurality of inflation rates, the initial image feature is subjected to dilated depth separable convolution processing in sequence to obtain a plurality of high-level semantic features of different scales.

[0034] In the embodiment, multiple different expansion rates can be set, and the initial image features are processed using dilated depth separable convolution with different expansion rates. Since the dilated convolution (i.e., dilated convolution) introduces a fixed interval called expansion rate between the weights of the convolution kernel, when the expansion rate is equal to 1, the dilated convolution is equivalent to the regular convolution, and when the expansion rate is greater than 1, the convolution kernel will skip some pixel points, thereby covering a wider input region and being able to fully utilize the feature maps of all regions. The formula for obtaining multiple high-level semantic features of different scales based on multiple expansion rates for dilated depth separable convolution processing of the initial image features in sequence can be expressed as:

[0035] wherein, is the multiple high-level semantic features of different scales, is the deep convolution, and are a 3*3 convolution kernel and a 5*5 convolution kernel, respectively, is a batch normalization operation, is a morphological filtering operation.

[0036] As can be seen from the above, the embodiment realizes multi-scale high-level semantic feature extraction through an innovative grouped dilated convolution architecture, and parallel processing is performed using deep separable convolution with different expansion rates, which can accurately capture microscopic details of the welding spot and identify macro defects, so that the model has stronger feature expression capability, effectively enhances the robustness of the model to different types of defects, and significantly improves the detection rate of small solder defects, thereby providing a more reliable solution for welding quality detection.

[0037] In an embodiment of the present application, feature fusion is performed on the multiple high-level semantic features of different scales to obtain fused features, including: connecting the multiple high-level semantic features of different scales based on up-sampling operations and down-sampling operations to obtain the fused features.

[0038] In the embodiment, the multiple high-level semantic features of different scales are connected based on up-sampling operations and down-sampling operations, specifically including: performing up-sampling operations on the multiple high-level semantic features lower than the target scale to obtain multiple first high-level semantic features after sampling; performing down-sampling operations on the multiple high-level semantic features higher than the target scale to obtain multiple second high-level semantic features after sampling; the resolutions of the multiple first high-level semantic features and the multiple second high-level semantic features are the same; for each high-level semantic feature in the multiple first high-level semantic features and the multiple second high-level semantic features, performing channel information fusion on the high-level semantic feature through a convolution layer to obtain a third high-level semantic feature; The number of channels of all third high-level semantic features is the same; all third high-level semantic features are spliced in the channel dimension.

[0039] In the embodiment, the target scale refers to a reference scale for unifying the resolution of each high-level semantic feature space, which is usually determined according to the typical size of the defect to be detected, for example, for a printed circuit board solder joint defect, 32x32 pixels or 64x64 pixels can be set to balance detail capture and computational efficiency.

[0040] The upsampling operation is used to enlarge the high-level semantic feature with a resolution lower than the target scale to the target scale, and the technical means adopted include bilinear interpolation or transposed convolution. The bilinear interpolation realizes parameter-free enlargement by weighted calculation of neighborhood pixel values, and is suitable for scenarios with higher real-time requirements; the transposed convolution realizes parameterized enlargement by learning convolution kernel parameters, and can better preserve edge details, and the parameters include the convolution kernel size (such as 4x4), the step (such as 2) and the padding value (such as 1), and the specific values are set according to the magnification ratio required, for example, when the feature of 16x16 pixels is enlarged to 32x32 pixels, the step is set to 2.

[0041] The downsampling operation is used to reduce the high-level semantic feature with a resolution higher than the target scale to the target scale, and the maximum pooling or convolution with a step of 2 is adopted. The maximum pooling realizes dimension reduction by selecting the maximum value in a 2x2 window, which can highlight the local strong response area; the convolution with a step of 2 realizes dimension reduction while extracting features, and the convolution kernel size is usually 3x3 and the padding value is 1 to avoid loss of edge information, for example, when the feature of 64x64 pixels is reduced to 32x32 pixels, the operation can be completed at one time.

[0042] The first high-level semantic feature refers to the feature with a resolution consistent with the target scale after upsampling processing, and the second high-level semantic feature refers to the feature with a resolution consistent with the target scale after downsampling processing, both of which retain small-scale detail information (such as a small pinhole) and large-scale semantic information (such as the overall morphology of a pad) due to different sources.

[0043] The convolution layer here is a 1x1 convolution layer, which is used for channel information fusion of the first and second high-level semantic features, and the parameters include the output channel number (such as 256), the step (set to 1) and the padding value (set to 0). The 1x1 convolution can adjust the channel number without changing the spatial resolution, compress redundant features through cross-channel information interaction, make the channel number of each feature uniform to a preset value, i.e., the channel number of the third high-level semantic feature, and ensure that the subsequent splicing operation is feasible.

[0044] The third high-level semantic feature is a feature processed by a 1x1 convolution, which has a uniform channel number and retains the core information of each scale feature. Channel dimension splicing refers to combining all third high-level semantic features in the channel dimension to form a fusion feature with a dimension of target scale height x target scale width x (channel number x feature quantity), for example, 3 32x32 pixel features with a channel number of 256 are spliced to obtain a fusion feature of 32x32x768.

[0045] In the present embodiment, the present embodiment aligns the spatial resolutions of the high-level semantic features of multiple different scales by upsampling and downsampling, solving the problem that different scale features cannot be directly fused; the 1x1 convolution unifies the channel number and enhances the channel correlation; and the channel splicing aggregates multi-scale information, so that the fusion feature contains both detail and semantic information. The underlying logic is that printed circuit board welding defects exist in both small sizes (such as 0.1 mm pinholes) and require overall pad morphology judgment (such as missing pads). Multi-scale feature fusion can meet the detection needs of different size defects and improve the recognition ability of the model for various defects.

[0046] For example, the present embodiment can determine a target scale, set a suitable resolution based on the defect size statistical result, perform upsampling on features below the target scale, perform downsampling on features above the target scale, ensure that the resolutions of all features are consistent, then unify the channel numbers of the features to a preset value through a 1x1 convolution layer, and finally splice all processed features in the channel dimension to output a fusion feature for subsequent defect detection.

[0047] In the present embodiment, before the feature fusion of the high-level semantic features of multiple different scales to obtain a fusion feature, the method further includes: performing feature enhancement processing on the high-level semantic features of multiple different scales respectively to obtain the high-level semantic features of multiple different scales after feature enhancement.

[0048] In the present embodiment, an EMA module is introduced to perform feature enhancement processing on the high-level semantic features of multiple different scales. The first branch respectively performs average pooling in the x direction and the y direction. A 3*3 convolution is used to extract the attention weight of the high-level semantic features of multiple different scales.

[0049] Specifically, in the first branch, the x and y direction information is encoded by adaptive global average pooling to realize effective cross-channel information interaction, and the two encoded features are spliced and share a 1x1 convolution. To further improve the global spatial information, the output of the first branch is encoded by global average pooling and converted to a corresponding shape to realize effective aggregation of cross-spatial information.

[0050] In the 3*3 convolution, a 3*3 size convolution kernel is used to process the input feature map to obtain feature information at different scales, and then a global average pooling operation is performed on the feature map obtained by convolution to extract global context information. The spatial attention value of the output feature map obtained by calculating the two generated weights is obtained, and the final feature map can retain complete spatial position information. Then, a Sigmoid activation function is used for processing to highlight the regions more important to the global context at the pixel level. Finally, the feature map emphasized by the attention mechanism is subjected to batch normalization processing to improve the stability and convergence effect of model training.

[0051] As can be seen from the above, the embodiment can effectively improve the representation ability of high-level semantic features by introducing the EMA module for feature enhancement. The first branch realizes cross-channel information depth interaction through adaptive global average pooling in the x and y directions and feature splicing, and combines global average pooling to strengthen global spatial information aggregation, so that the features more accurately reflect the spatial distribution characteristics of defects. The attention weight extracted by the 3*3 convolution is integrated with the global context information and Sigmoid activation, which can highlight the pixel features of key regions such as solder defects, and enhance the recognition of different scale defects such as micro pinholes and missing pads. Batch normalization processing improves the stability of model training and speeds up the convergence speed. In summary, the feature enhancement process can significantly improve the quality of the subsequent fused features, thereby improving the accuracy and robustness of printed circuit board welding defect detection.

[0052] In an embodiment of the present application, the welding quality detection method based on machine vision further comprises: obtaining a plurality of sample image sets containing printed circuit boards, and taking the plurality of sample image sets containing printed circuit boards as a training data set; wherein each sample image set containing a printed circuit board corresponds to a welding method of a printed circuit board; A multi-task decoupling loss function is constructed based on a classification loss function, a regression loss function and a direction perception loss function; The initial model is trained based on the training data set and the multi-task decoupling loss function to obtain a defect detection model.

[0053] In the embodiment, before constructing the multi-task decoupling loss function based on the classification loss function, the regression loss function and the direction perception loss function, it further comprises: A classification loss function is constructed based on a standard class loss function and a sample perception weight; A regression loss function is constructed based on a normalized Gaussian distance; A direction perception loss function is constructed based on a sinusoidal function to smooth the angle difference.

[0054] In the embodiment, the standard category loss function refers to a basic function for measuring the difference between the defect type classification result and the true label, commonly such as cross-entropy loss, and the parameters include the number of defect categories and the labels of each category. The sample perception weight is a weight value dynamically adjusted according to the sample characteristics, which is related to the sample quantity, category balance and sample difficulty, for example, giving a higher weight to rare defect samples to improve the model's attention to them. The classification loss function is constructed by multiplying the standard category loss function and the sample perception weight, realizing the differential optimization of different samples and enhancing the classification ability of the model for each defect.

[0055] The normalized Gaussian distance refers to the value of the distance between the regression target such as defect position and size and the predicted value after Gaussian distribution normalization processing, and the parameters include the standard deviation of Gaussian distribution and the dimension of the regression target. The regression loss function is constructed based on this distance, which measures the deviation between the predicted value and the true value, guides the model to accurately predict the spatial position and geometric size of the defect, and is suitable for quantitative detection of defects such as weld point deviation and size anomaly.

[0056] In the direction perception loss function, the angle difference refers to the deviation between the actual direction of the defect and the predicted direction, and the sine function is used to smooth the difference, and the parameters include the angle value range and the smoothing coefficient. By converting the angle difference into a continuous loss value, the optimization problem caused by the periodicity of the angle is solved, which is suitable for the detection of defects with direction attributes such as pin skew. The multi-task decoupling loss function is constructed by summing the weighted coefficients of the classification loss function, the regression loss function and the direction perception loss function, and the weighted coefficients are set according to the importance and convergence speed of each task.

[0057] In the embodiment, the sample image set covers multiple welding methods to ensure that the model adapts to different scenarios; the classification loss function optimizes defect type judgment, the regression loss function improves spatial positioning accuracy, and the direction perception loss function enhances direction recognition ability; the multi-task decoupling loss function realizes independent optimization and collaborative convergence of each task, avoiding interference between tasks.

[0058] For example, the embodiment can collect printed circuit board sample images of multiple welding methods, label defect categories, positions, sizes and directions, and construct a training data set; generate a classification loss function based on cross-entropy loss combined with sample perception weight; design a regression loss function using normalized Gaussian distance; construct a direction perception loss function by smoothing the angle difference with a sine function; assign weights to the three loss functions and sum them to get a multi-task decoupling loss function; input the training data set into the initial model, and iteratively train based on the loss function until the model converges to get a defect detection model.

[0059] Exemplarily, there are many PCB inspection standards, among which the most recognized by the industry is the IPC standard, which is formulated by the IPC (Institute for Printed Circuits) and covers the whole process of PCB manufacturing and sets detailed acceptance criteria. In particular, the IPC-A-600 standard focuses on the appearance quality inspection of PCBs, including the evaluation of visible defects on the surface. In this embodiment, the PCB dataset is collected by a linear scanning CCD camera with a resolution of about 48 pixels per millimeter. After manually screening and removing template images without defects from the sampled images, the initial size of the template and test images is usually 16000x16000 pixels, which are then cropped into multiple 640x640 pixel sub-images and calibrated using template matching technology. In addition, a threshold is set for image binarization to reduce the interference caused by illumination, and finally a PCB dataset containing 1500 images is generated.

[0060] Exemplarily, for the defect classification task, this embodiment uses a classification task loss function to solve the class imbalance and difficult sample imbalance problems; for the regression task of defect position and size, a loss function based on the regression task is designed; for the common defects with direction characteristics on the PCB, such as bridging and sharpness, a direction perception loss is designed. Based on the training dataset and the multi-task decoupling loss function, the initial model is trained to obtain a defect detection model.

[0061] Exemplarily, in the PCB defect detection scene, the sample imbalance problem is particularly prominent. The sample perception weight mechanism used in this scheme contains three key design dimensions. Specifically, the class balance factor uses a modified inverse frequency weighting method, which introduces a smoothing coefficient to avoid assigning too much weight to extremely rare classes. This nonlinear adjustment method can not only alleviate class imbalance, but also prevent the model from focusing too much on rare classes and sacrificing overall performance, while maintaining the detection accuracy of common defects. The dynamic difficult sample focusing strategy allows the model to adaptively adjust the attention to difficult samples at different training stages, avoiding the problem of excessive focus in the later training caused by traditional loss function values. The spatial importance weight is generated by a Gaussian kernel function to generate a weight distribution map centered on the defect, emphasizing the importance of the defect area at the pixel level. It can cover the surrounding area of typical defects without introducing too much background noise.

[0062] In this embodiment, the traditional regression loss function faces two main challenges in dealing with PCB defect detection: insufficient sensitivity to small defects and direction insensitivity. This embodiment adopts a normalized Gaussian distance loss to solve these problems. This method is more sensitive to small shifts in the bounding box, encodes direction information automatically through the covariance matrix, provides a continuous metric space, and facilitates gradient propagation. In this embodiment, the detection of direction-sensitive defects such as bridging and sharp pulling requires a special angle representation method. This embodiment uses a periodic loss function based on the sine function to solve the two inherent problems of traditional angle regression.

[0063] As can be seen from the above, by constructing a multi-task decoupling loss function, the performance of the defect detection model is significantly improved. The classification loss function combines with the sample-aware weight, effectively solves the class imbalance and difficult sample imbalance problem, and enhances the recognition ability of rare and difficult defects. The regression loss function is based on normalized Gaussian distance, which improves the sensitivity to small defect shifts, and encodes direction information through the covariance matrix to optimize the position and size prediction accuracy. The direction-aware loss uses a sine function to handle the angle periodicity problem, accurately detecting direction-sensitive defects such as bridging and sharp pulling. The multi-task decoupling design realizes the cooperative optimization of each task, combined with diversified sample training, so that the model adapts to multiple welding methods, greatly improving the accuracy and robustness of printed circuit board welding defect detection.

[0064] The machine vision-based welding quality detection method of the above embodiment, Figure 2 The structure block diagram of the machine vision-based welding quality detection device provided by an embodiment of the present application is shown. For ease of illustration, only the parts related to the embodiments of the present application are shown. For reference Figure 2 The machine vision-based welding quality detection device 20 includes an image acquisition module 21 and a quality detection module 22. The image acquisition module is used to acquire the target image to be detected, and the target image to be detected is a printed circuit board; The quality detection module is used to input the target image to be detected into a defect detection model to obtain a defect detection result; the defect detection result is used to represent the defect type of the printed circuit board; The defect detection method includes: initial feature extraction is performed on the target image to be detected to obtain initial image features; multi-scale feature extraction is performed on the initial image features to obtain a plurality of high-level semantic features of different scales; feature fusion is performed on the plurality of high-level semantic features of different scales to obtain fused features; defect detection is performed based on the fused features to obtain a defect detection result.

[0065] In an embodiment of the present application, the defect detection model comprises a convolutional layer; and the quality detection module 22 is specifically configured to: perform initial feature extraction on the target image to be detected through the convolutional layer to obtain initial image features.

[0066] In an embodiment of the present application, the quality detection module 22 is specifically further configured to: perform dilated depth separable convolution processing on the initial image features in sequence based on multiple dilation rates to obtain multiple high-level semantic features of different scales.

[0067] In an embodiment of the present application, the quality detection module 22 is specifically further configured to: connect the multiple high-level semantic features of different scales based on an up-sampling operation and a down-sampling operation to obtain fused features.

[0068] In an embodiment of the present application, the quality detection module 22 is specifically further configured to: perform the up-sampling operation on the multiple high-level semantic features lower than the target scale to obtain multiple first high-level semantic features after sampling; perform the down-sampling operation on the multiple high-level semantic features higher than the target scale to obtain multiple second high-level semantic features after sampling; the resolutions of the multiple first high-level semantic features and the multiple second high-level semantic features are the same; for each high-level semantic feature in the multiple first high-level semantic features and the multiple second high-level semantic features, perform channel information fusion on the high-level semantic feature through a convolutional layer to obtain a third high-level semantic feature; the channel numbers of all the third high-level semantic features are the same; all the third high-level semantic features are spliced in the channel dimension.

[0069] In an embodiment of the present application, the welding quality detection device based on machine vision 20 further comprises a training module configured to: obtain multiple sample image sets containing printed circuit boards, and take the multiple sample image sets containing printed circuit boards as a training data set; wherein each sample image set containing a printed circuit board corresponds to a welding mode of the printed circuit board; construct a multi-task decoupling loss function based on a classification loss function, a regression loss function and a direction perception loss function; train the initial model based on the training data set and the multi-task decoupling loss function to obtain a defect detection model.

[0070] In an embodiment of the present application, the training module is specifically configured to: construct the classification loss function based on a standard category loss function and a sample perception weight; construct the regression loss function based on a normalized Gaussian distance; construct the direction perception loss function based on a sinusoidal function to smooth the angle difference.

[0071] Referring to Figure 3 , Figure 3 A schematic block diagram of an electronic device according to an embodiment of the present application is provided. As shown in the figure, Figure 3 The electronic device 300 in the embodiment can include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The above-mentioned processor 301, input device 302, output device 303, and memory 304 complete communication with each other through a communication bus 305. The memory 304 is used to store a computer program, and the computer program includes program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. Among them, the processor 301 is configured to call the program instructions to execute the functions of each module in the above-mentioned device embodiments, for example Figure 2 The functions of the image acquisition module 21 and the quality detection module 22 shown in the figure.

[0072] It should be understood that in the embodiments of the present application, the processor 301 can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0073] The input device 302 can include a touchpad, a fingerprint acquisition sensor (used to acquire fingerprint information and direction information of the fingerprint of a user), a microphone, etc., and the output device 303 can include a display (LCD, etc.), a speaker, etc.

[0074] The memory 304 can include read-only memory and random access memory, and provide instructions and data to the processor 301. A part of the memory 304 can also include non-volatile random access memory.

[0075] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present application can execute the implementation manner described in the welding quality detection method based on machine vision provided by the embodiments of the present application, and can also execute the implementation manner of the electronic device described in the embodiments of the present application, which will not be described here.

[0076] In another embodiment of the present application, a computer readable storage medium is provided, which stores a computer program. The computer program includes program instructions, which, when executed by a processor, implement all or part of the processes of the above-mentioned embodiment methods. The computer program can also instruct related hardware to complete the implementation. The computer program can be stored in a computer readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0077] The computer readable storage medium can be an internal storage unit of the electronic device of any of the preceding embodiments, such as a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the electronic device. The computer readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0078] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0079] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic device and the units described above can refer to the corresponding processes in the above-mentioned method embodiments, which will not be described here.

[0080] In several embodiments provided in the present application, it should be understood that the disclosed electronic device and method can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic; the division of the units is merely logical function division; an actual implementation can be divided into different units depending on actual conditions; or a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, or can be in electrical, mechanical or other forms.

[0081] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0082] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0083] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto; any skilled person in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for detecting welding quality based on machine vision, characterized in that, The method comprises the following steps: obtaining a target image to be detected, the target image to be detected being an image containing a printed circuit board; inputting the target image to be detected into a defect detection model for defect detection to obtain a defect detection result, the defect detection result being used to represent a defect type of the printed circuit board; wherein the defect detection method comprises the following steps: performing initial feature extraction on the target image to be detected to obtain initial image features; performing multi-scale feature extraction on the initial image features to obtain a plurality of high-level semantic features of different scales; performing feature fusion on the plurality of high-level semantic features of different scales to obtain fused features; performing defect detection based on the fused features to obtain the defect detection result.

2. The machine vision-based welding quality inspection method of claim 1, wherein, The defect detection model comprises a convolution layer; the initial feature extraction on the target image to be detected to obtain initial image features comprises the following steps: performing initial feature extraction on the target image to be detected through the convolution layer to obtain initial image features.

3. The machine vision-based welding quality inspection method of claim 1, wherein, the multi-scale feature extraction on the initial image features to obtain a plurality of high-level semantic features of different scales comprises the following steps: based on a plurality of inflation rates, performing dilated depth separable convolution processing on the initial image features in sequence to obtain a plurality of high-level semantic features of different scales.

4. The machine vision-based welding quality inspection method of claim 1, wherein, the feature fusion on the plurality of high-level semantic features of different scales to obtain fused features comprises the following steps: connecting the plurality of high-level semantic features of different scales based on up-sampling operations and down-sampling operations to obtain fused features.

5. The machine vision-based welding quality inspection method of claim 4, wherein, the connection of the plurality of high-level semantic features of different scales based on up-sampling operations and down-sampling operations comprises the following steps: performing up-sampling operations on a plurality of high-level semantic features lower than a target scale to obtain a plurality of first high-level semantic features after sampling; performing down-sampling operations on a plurality of high-level semantic features higher than the target scale to obtain a plurality of second high-level semantic features after sampling; the resolutions of the plurality of first high-level semantic features and the plurality of second high-level semantic features are the same; for each high-level semantic feature in the plurality of first high-level semantic features and the plurality of second high-level semantic features, performing channel information fusion on the high-level semantic feature through a convolution layer to obtain a third high-level semantic feature; the channel numbers of all third high-level semantic features are the same; splicing all third high-level semantic features in the channel dimension.

6. The machine vision-based welding quality inspection method of claim 1, wherein, The method further comprises the following steps: obtaining a plurality of sample image sets containing printed circuit boards, and taking the plurality of sample image sets containing printed circuit boards as a training data set; wherein each sample image set containing a printed circuit board corresponds to a soldering mode of a printed circuit board; constructing a multi-task decoupling loss function based on a classification loss function, a regression loss function and a direction perception loss function; training an initial model based on the training data set and the multi-task decoupling loss function to obtain the defect detection model.

7. The machine vision-based welding quality inspection method of claim 6, wherein, Before the multi-task decoupling loss function is constructed based on the classification loss function, the regression loss function and the direction perception loss function, the method further comprises the following steps: constructing a classification loss function based on a standard category loss function and a sample perception weight; constructing a regression loss function based on a normalized Gaussian distance; The direction perception loss function is constructed based on a sine function to smooth angle differences.

8. A method and apparatus for detecting welding quality based on machine vision, characterized in that, The method comprises the steps of: An image acquisition module is configured to acquire a target image to be detected, wherein the target image to be detected is a printed circuit board. A quality detection module is configured to input the target image to be detected into a defect detection model to obtain a defect detection result. The defect detection result is used to represent a defect type of the printed circuit board. The defect detection method comprises the steps of: initial feature extraction is performed on the target image to be detected to obtain initial image features; multi-scale feature extraction is performed on the initial image features to obtain a plurality of high-level semantic features of different scales; feature fusion is performed on the plurality of high-level semantic features of different scales to obtain fused features; defect detection is performed based on the fused features to obtain the defect detection result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 7.

Images