Welding quality inspection method and device based on machine vision

Abstract

This application provides a machine vision-based welding quality inspection method and apparatus, belonging to the field of machine vision and intelligent inspection technology. The method includes: acquiring a target image to be inspected, wherein the target image is an image containing a printed circuit board; inputting the target image to be inspected into a defect detection model for defect detection, and obtaining a defect detection result, which is used to characterize the defect type of the printed circuit board. The machine vision-based welding quality inspection method and apparatus provided in this application can improve the accuracy of welding defect detection.

Description

Technical Field

[0001] This application belongs to the field of machine vision and intelligent inspection technology, and more specifically, it relates to a welding quality inspection method and device based on machine vision. Background Technology

[0002] Printed Circuit Boards (PCBs), as core components of electronic devices, are directly affected by their soldering quality, which impacts product performance and reliability. Soldering defects such as cold solder joints, abnormal soldering, and bridging can lead to circuit failures or even safety accidents. Existing PCB soldering quality inspection methods suffer from high false positive rates, making them unsuitable for large-scale production. Rule-based machine vision and traditional machine learning methods rely on manual feature and threshold settings, resulting in poor adaptability to complex scenarios. Therefore, there is an urgent need for a high-precision, highly generalizable PCB soldering quality inspection technology to meet the automation requirements of modern electronic manufacturing. Summary of the Invention

[0003] The purpose of this application is to provide a welding quality inspection method and device based on machine vision, so as to improve the accuracy and reliability of machine vision in PCB welding quality inspection.

[0004] A first aspect of this application provides a welding quality inspection method based on machine vision, comprising: acquiring a target image to be inspected, wherein the target image to be inspected is an image containing a printed circuit board;

[0005] The target image to be detected is input into the defect detection model for defect detection, and the defect detection results are obtained. The defect detection results are used to characterize the defect type of the printed circuit board.

[0006] The defect detection methods include:

[0007] Initial feature extraction is performed on the target image to be detected to obtain initial image features;

[0008] Multi-scale feature extraction is performed on the initial image features to obtain multiple high-level semantic features at different scales;

[0009] Feature fusion is performed on multiple high-level semantic features at different scales to obtain fused features;

[0010] Defect detection is performed based on fusion features to obtain defect detection results.

[0011] A second aspect of this application provides a welding quality inspection device based on machine vision, comprising:

[0012] The image acquisition module is used to acquire the target image to be detected, which is a printed circuit board.

[0013] The quality inspection module is used to input the target image to be inspected into the defect detection model to obtain the defect detection results; the defect detection results are used to characterize the defect type of the printed circuit board.

[0014] The defect detection methods include:

[0015] Initial feature extraction is performed on the target image to be detected to obtain initial image features;

[0016] Multi-scale feature extraction is performed on the initial image features to obtain multiple high-level semantic features at different scales;

[0017] Feature fusion is performed on multiple high-level semantic features at different scales to obtain fused features;

[0018] Defect detection is performed based on fusion features to obtain defect detection results.

[0019] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described machine vision-based welding quality inspection method.

[0020] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the machine vision-based welding quality inspection method described above.

[0021] The beneficial effects of the machine vision-based welding quality inspection method and apparatus provided in this application are as follows: This application significantly improves the accuracy and efficiency of welding quality inspection by systematically performing a defect detection process on target images containing printed circuit boards. Specifically, this application captures basic image information through initial feature extraction, and then obtains high-level semantic features at different scales through multi-scale feature extraction. This retains the detailed information of minute defects and covers the semantic information of the overall shape of the defects. By fusing multi-scale features, the complementarity and enhancement of features at different levels are achieved, enabling the fused features to more comprehensively represent various welding defects. Finally, defect detection is performed based on the fused features, effectively improving the recognition accuracy of various welding defects on printed circuit boards (such as solder pinholes, missing pads, and misaligned pins).

[0022] Meanwhile, the embodiments of this application enhance the adaptability to defects of different sizes and types through multi-scale feature processing and fusion, significantly improving the robustness of detection and meeting the high-quality inspection requirements in the large-scale production of printed circuit boards. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A schematic flowchart of a machine vision-based welding quality inspection method provided in an embodiment of this application;

[0025] Figure 2 A structural block diagram of a machine vision-based welding quality inspection device provided in an embodiment of this application;

[0026] Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0029] Please refer to Figure 1 , Figure 1 A flowchart illustrating a machine vision-based welding quality inspection method according to an embodiment of this application is provided. The method may include:

[0030] S101: Acquire the target image to be detected. The target image to be detected is an image containing a printed circuit board.

[0031] In this embodiment, to overcome the limitations of a single light source in PCB soldering inspection, such as reflection, shadow, and low contrast, a multi-spectral, multi-angle composite light source illumination system can be adopted, combined with an intelligent light control strategy, to ensure high-quality imaging of solder joints, pads, and traces.

[0032] Specifically, a ring-shaped LED array can be used as the main light source to provide uniform base illumination for the vision system, reducing shadow interference. In terms of layout, coaxial lighting or low-angle ring lighting can be used to highlight the three-dimensional morphology of the solder joints. Secondly, a linear polarizer is installed in front of the light source, and a cross-polarizing filter is configured at the camera end to eliminate specular reflections from the PCB surface metal, enhancing the visibility of the solder wetting edges.

[0033] In this embodiment, a high-precision area array camera (resolution ≥ 1392×1040) paired with an 8mm low-distortion industrial lens can be used to achieve continuous shooting capability of 30 frames per second, capturing images of the PCB to be inspected on the production line. At least three sets of visual imaging structures can be set on both sides of the production line, evenly distributed around the PCB at 180° to ensure full coverage imaging of the PCB. For example, the camera module can be equipped with an adjustable angle gimbal, supporting ±15° pitch angle adjustment to adapt to PCBs of different sizes.

[0034] S102: Input the target image to be detected into the defect detection model to perform defect detection and obtain the defect detection result. The defect detection result is used to characterize the defect type of the printed circuit board.

[0035] The defect detection method includes: extracting initial features from the target image to be detected to obtain initial image features;

[0036] Multi-scale feature extraction is performed on the initial image features to obtain multiple high-level semantic features at different scales;

[0037] Feature fusion is performed on multiple high-level semantic features at different scales to obtain fused features;

[0038] Defect detection is performed based on fusion features to obtain defect detection results.

[0039] In this embodiment, the defect detection results are used to characterize the defect type. Printed circuit board defects can be categorized into four main types: small-size defects, color difference defects, large-size defects, round hole defects, and other types of defects.

[0040] Specifically, small-size defects can include: missing holes, hair-like inclusions, and excessively narrow conductor spacing. Color difference defects can include: character defects and copperplate color differences. Large-size defects can include: large-size mis-corrosion, multiple holes, missing lines, and large-size obstructions. Round hole defects can include: off-center holes, pinholes, and holes covered with debris. Other types of defects can include: open circuits, short circuits, burrs, and rodent bites. For example, large-size mis-corrosion is caused by solder bubbles or inadequate solder curing. Open circuits may be caused by hard objects on the cutting machine or uneven chemical concentration of the hole-fixing agent, resulting in incomplete circuit connection. Rodent bites may be caused by uneven soldering or sharp objects on the machine; these defects may lead to open circuits or other malfunctions on the circuit board. In addition, operational errors or equipment failures are also common causes of short circuits, which may lead to product scrap. Hair-like inclusions are mainly caused by the failure to use through-hole mesh during the solder mask process or excessive ink residue, affecting the appearance and function of the circuit board. Burrs are usually caused by using dull scissors or improper installation of scissors when cutting PCB boards, which can affect signal transmission. Short circuits are usually caused by solder run-out or incomplete etching, often due to the use of improper stripping solutions or incorrect etching solution ratios, which can damage the circuit board.

[0041] In this embodiment, the target image to be detected can be a PCB image, and the defect detection model can be trained based on the YOLOv8 network.

[0042] For example, this embodiment can use the DWR module and the EMA attention mechanism to perform initial feature extraction on the target image to be detected, thereby improving the ability to extract multi-scale features of PCB defects. The DWR module includes an RR module to perform initial image feature extraction on the target image to be detected, thereby obtaining initial image features.

[0043] In this embodiment, the RS module can be selected to extract advanced semantic features from the initial image features. Specifically, the RS module contains three different convolutional kernels, which can group the input initial image features and convolve them using different dilation rates to ultimately obtain multiple advanced semantic features at different scales.

[0044] In this embodiment, an improved BiFPN can be used to fuse multiple high-level semantic features at different scales. Specifically, based on the original top-down path, lateral cross-layer connections are added, enabling direct interaction between shallow high-resolution features and deep semantic features, thereby improving the ability to preserve details of tiny solder joints.

[0045] As can be seen from the above, this embodiment significantly improves the accuracy and efficiency of welding quality inspection by implementing a systematic defect detection process on target images containing printed circuit boards. Specifically, this embodiment captures basic image information through initial feature extraction, and then obtains high-level semantic features at different scales through multi-scale feature extraction. This retains the detailed information of minor defects while also covering the semantic information of the overall defect morphology. By fusing multi-scale features, the complementarity and enhancement of features at different levels are achieved, enabling the fused features to more comprehensively characterize various welding defects. Finally, defect detection is performed based on the fused features, effectively improving the recognition accuracy of various welding defects on printed circuit boards (such as solder pinholes, missing pads, and misaligned pins).

[0046] Meanwhile, this embodiment enhances the adaptability to defects of different sizes and types through multi-scale feature processing and fusion, significantly improving the robustness of detection and meeting the high-quality inspection requirements in the large-scale production of printed circuit boards.

[0047] In one embodiment of this application, the defect detection model includes a convolutional layer; initial feature extraction of the target image to be detected to obtain initial image features includes: initial feature extraction of the target image to be detected through the convolutional layer to obtain initial image features.

[0048] In this embodiment, the PCB image to be detected is first input into a 3×3 convolutional layer to generate a multi-channel feature map through convolution operations. Then, a normalization mechanism is introduced to effectively solve the internal covariate shift problem commonly found in deep neural networks, creating conditions for stable training of subsequent network layers. Next, the ReLU activation function is used to perform a non-linear transformation on the features. This operation has the dual advantages of improving feature sparsity and alleviating the gradient vanishing problem. Its formula can be expressed as:

[0049]

[0050] in, As initial image features, For input data, and These are the activation function and the convolution operation, respectively. This is a normalization operation.

[0051] Specifically, from the perspective of feature sparsity optimization, the sparse neuron mechanism constructed by the ReLU activation function is essentially an efficient feature selection strategy. In actual computation, neurons with zero activation output can accurately filter irrelevant background information such as PCB substrate and silkscreen text, while neurons that maintain a non-zero activation state focus on key detection areas such as solder joints and pins, significantly improving the robustness of the model by suppressing irrelevant responses.

[0052] The vanishing gradient problem stems from the exponential decay of gradients layer by layer during backpropagation, making it difficult to effectively update parameters in deep networks. The ReLU activation function, by introducing linear activation properties with one-sided inhibition, maintains an ideal propagation state where the gradient is constant at 1 in the positive interval. This effectively alleviates the gradient decay phenomenon caused by traditional activation functions such as Sigmoid and Tanh in deep networks, providing feasible support for building deeper neural network structures.

[0053] As can be seen from the above, the convolution kernel size in this embodiment has been optimized, which can effectively suppress noise interference while preserving the geometric details of the solder joints, and significantly improve the model training convergence speed, laying the foundation for the stable training of the subsequent multi-scale feature extraction module.

[0054] In one embodiment of this application, multi-scale feature extraction is performed on the initial image features to obtain multiple high-level semantic features at different scales, including:

[0055] By sequentially performing depthwise separable convolution on the initial image features using multiple dilation rates, high-level semantic features at multiple different scales are obtained.

[0056] In this embodiment, multiple different dilation rates can be set, and depthwise separable convolutions with different dilation rates can be used to process the initial image features. Since dilated convolution (i.e., dilated convolution) introduces a fixed interval called the dilation rate between the weights of the convolution kernel, when the dilation rate is equal to 1, dilated convolution is equivalent to regular convolution; when the dilation rate is greater than 1, the convolution kernel will skip some pixels, thereby covering a wider input region and fully utilizing the feature maps of all regions. The formula for obtaining multiple high-level semantic features at different scales by sequentially performing depthwise separable convolution processing on the initial image features based on multiple dilation rates can be expressed as:

[0057]

[0058] in, These are high-level semantic features at multiple different scales. For depthwise convolution, and They are 3x3 convolution kernels and 5x5 convolution kernels, respectively. For batch normalization operations, This is a morphological filtering operation.

[0059] As can be seen from the above, this embodiment achieves multi-scale high-level semantic feature extraction through an innovative grouped dilated convolutional architecture. It uses depthwise separable convolutions with different dilation rates for parallel processing, which can accurately capture the microscopic details of solder joints and identify macroscopic defects, giving the model a stronger feature expression capability and effectively enhancing the model's robustness to different types of defects. In particular, it significantly improves the detection rate of micro solder defects, providing a more reliable solution for soldering quality inspection.

[0060] In one embodiment of this application, feature fusion is performed on multiple high-level semantic features of different scales to obtain fused features, including: connecting multiple high-level semantic features of different scales based on upsampling and downsampling operations to obtain fused features.

[0061] In this embodiment, multiple high-level semantic features at different scales are connected based on upsampling and downsampling operations. Specifically, this includes performing upsampling operations on multiple high-level semantic features below the target scale to obtain multiple sampled first high-level semantic features.

[0062] For multiple high-level semantic features that are higher than the target scale, downsampling operations are performed to obtain multiple sampled second high-level semantic features; the multiple first high-level semantic features and the multiple second high-level semantic features have the same resolution;

[0063] For each of the multiple first-level high-level semantic features and multiple second-level high-level semantic features, a third-level high-level semantic feature is obtained by fusing the channel information of the high-level semantic feature through a convolutional layer.

[0064] All third-level semantic features have the same number of channels; all third-level semantic features are concatenated by channel dimension.

[0065] In this embodiment, the target scale refers to a pre-set benchmark scale used to unify the spatial resolution of various high-level semantic features. It is usually determined based on the typical size of the defect to be detected. For example, for solder joint defects on printed circuit boards, it can be set to 32×32 pixels or 64×64 pixels to balance detail capture and computational efficiency.

[0066] Upsampling operations are used to upscale high-level semantic features with resolutions lower than the target scale to the target scale. Techniques employed include bilinear interpolation or transposed convolution. Bilinear interpolation achieves parameter-free upsampling by weighting neighboring pixel values, making it suitable for scenarios with high real-time requirements. Transposed convolution, on the other hand, performs parameterized upsampling by learning kernel parameters, which better preserves edge details. Its parameters include kernel size (e.g., 4×4), stride (e.g., 2), and padding (e.g., 1). Specific values ​​are set according to the desired upsampling factor; for example, when upsampling a 16×16 pixel feature to 32×32 pixels, a stride of 2 is used.

[0067] Downsampling operations are used to reduce high-level semantic features with resolutions higher than the target scale to the target scale, employing either max pooling or convolution with a stride of 2. Max pooling achieves dimensionality reduction by selecting the maximum value within a 2×2 window, highlighting strong local response regions. Convolution with a stride of 2 extracts features while reducing dimensionality; its kernel size is typically 3×3 with padding of 1 to avoid losing edge information. For example, reducing a 64×64 pixel feature to 32×32 pixels can be accomplished with a single downsampling operation.

[0068] The first high-level semantic feature refers to the feature whose resolution is consistent with the target scale after upsampling. The second high-level semantic feature refers to the feature whose resolution is consistent with the target scale after downsampling. Due to their different sources, the two retain small-scale detail information (such as tiny pinholes) and large-scale semantic information (such as the overall shape of the pads), respectively.

[0069] The convolutional layer here is a 1×1 convolutional layer, used to fuse channel information of the first and second high-level semantic features. Its parameters include the number of output channels (e.g., 256), stride (set to 1), and padding value (set to 0). The 1×1 convolution can adjust the number of channels without changing the spatial resolution, compressing redundant features through cross-channel information interaction, so that the number of channels of each feature is unified to the preset value, that is, the number of channels of the third high-level semantic feature, ensuring that subsequent concatenation operations are feasible.

[0070] The third-level semantic features are features processed by 1×1 convolution, which have a uniform number of channels and retain the core information of features at each scale. Channel dimension concatenation refers to merging all third-level semantic features along the channel dimension to form a fused feature with the dimension of target scale height × target scale width × (number of channels × number of features). For example, concatenating three 32×32 pixel features with 256 channels results in a fused feature of 32×32×768.

[0071] In this embodiment, upsampling and downsampling are used to align the spatial resolution of multiple high-level semantic features at different scales, solving the problem of direct fusion of features at different scales. 1×1 convolution unifies the number of channels and enhances channel correlation. Channel splicing aggregates multi-scale information, enabling the fused features to contain both detailed and semantic information. The underlying logic is that printed circuit board soldering defects can be both minute in size (e.g., pinholes of 0.1mm) and require consideration of the overall pad shape (e.g., missing pads). Multi-scale feature fusion can address the detection needs of defects of different sizes, improving the model's ability to identify various defects.

[0072] For example, this embodiment can determine the target scale and set an appropriate resolution based on the statistical results of defect size; then, upsampling is performed on features below the target scale and downsampling is performed on features above the target scale to ensure that all feature resolutions are consistent; then, the number of channels of each feature is unified to a preset value through a 1×1 convolutional layer; finally, all processed features are concatenated in the channel dimension to output fused features for subsequent defect detection.

[0073] In this embodiment, before fusing multiple high-level semantic features at different scales to obtain fused features, the method further includes: performing feature enhancement processing on the multiple high-level semantic features at different scales respectively to obtain multiple high-level semantic features at different scales after feature enhancement.

[0074] In this embodiment, an EMA module is introduced to perform feature enhancement processing on high-level semantic features at multiple different scales. The first branch performs average pooling in the x and y directions, respectively. A 3*3 convolution is used to extract attention weights for high-level semantic features at multiple different scales.

[0075] Specifically, in the first branch, information in the x and y directions is encoded using adaptive global average pooling to achieve effective cross-channel information interaction, and the two encoded features are concatenated and share a 1×1 convolution. To further enhance global spatial information, the output of the first branch is encoded using global average pooling and converted into a corresponding shape to achieve effective aggregation of cross-spatial information.

[0076] In the 3x3 convolution, a 3x3 kernel is used to process the input feature map, obtaining feature information at different scales. Then, global average pooling is performed on the resulting feature map to extract global contextual information. The spatial attention value of the output feature map is calculated using the two generated weights, resulting in a feature map that retains complete spatial location information. A sigmoid activation function is then applied to highlight regions that are more important to the global context at the pixel level. Finally, the feature maps emphasized by the attention mechanism are batch normalized to improve the stability and convergence of the model training.

[0077] As shown above, this embodiment effectively enhances the representational ability of high-level semantic features by introducing the EMA module for feature enhancement. The first branch achieves deep interaction of cross-channel information through adaptive global average pooling and feature concatenation in the x and y directions. Combined with global average pooling, it strengthens the aggregation of global spatial information, making the features more accurately reflect the spatial distribution characteristics of defects. The attention weights extracted by 3×3 convolution, after integration of global context information and sigmoid activation, can highlight the pixel features of key areas such as solder joint defects, and enhance the recognition of defects of different scales such as micro-pinholes and missing pads. Batch normalization processing improves the stability of model training and accelerates the convergence speed. In summary, this feature enhancement process can significantly improve the quality of subsequent fused features, thereby improving the accuracy and robustness of printed circuit board soldering defect detection.

[0078] In one embodiment of this application, the welding quality inspection method based on machine vision further includes: acquiring multiple sample image sets containing printed circuit boards, and using the multiple sample image sets containing printed circuit boards as training datasets; wherein each sample image set containing printed circuit boards corresponds to a welding method of printed circuit boards.

[0079] A multi-task decoupling loss function is constructed based on classification loss function, regression loss function and orientation-aware loss function;

[0080] The initial model is trained based on the training dataset and the multi-task decoupling loss function to obtain the defect detection model.

[0081] In this embodiment, before constructing the multi-task decoupling loss function based on the classification loss function, regression loss function, and orientation-aware loss function, the following is also included:

[0082] A classification loss function is constructed based on the standard category loss function and sample perception weights;

[0083] A regression loss function is constructed based on normalized Gaussian distance;

[0084] A direction-aware loss function is constructed based on the sine function to smooth out angle differences.

[0085] In this embodiment, the standard category loss function refers to the basic function used to measure the difference between the defect type classification result and the true label, such as cross-entropy loss, whose parameters include the number of defect categories and the label of each category. The sample-aware weights are weight values ​​that are dynamically adjusted according to sample characteristics, and are related to the number of samples, class balance, and sample difficulty. For example, rare defect samples are given higher weights to increase the model's attention to them. The classification loss function is constructed by multiplying the standard category loss function by the sample-aware weights, achieving differentiated optimization for different samples and enhancing the model's ability to classify various defects.

[0086] Normalized Gaussian distance refers to the distance between the regression target (defect location, size, etc.) and the predicted value, normalized by the Gaussian distribution characteristics. Parameters include the standard deviation of the Gaussian distribution and the dimension of the regression target. The regression loss function is built upon this distance, measuring the deviation between the predicted and true values ​​to guide the model in accurately predicting the spatial location and geometric dimensions of defects. It is suitable for the quantitative detection of defects such as weld point misalignment and dimensional anomalies.

[0087] In the orientation-aware loss function, the angle difference refers to the deviation between the actual direction of the defect and the predicted direction. A sine function is used to smooth this difference, and the parameters include the angle range and the smoothing coefficient. By converting the angle difference into a continuous loss value, the optimization problem caused by the periodicity of the angle is solved, making it suitable for detecting defects with directional attributes, such as pin misalignment. The multi-task decoupling loss function is constructed by summing the classification loss function, regression loss function, and orientation-aware loss function after assigning weight coefficients. The weight coefficients are set according to the importance of each task and the convergence speed.

[0088] In this embodiment, the sample image set covers a variety of welding methods to ensure that the model can adapt to different scenarios; the classification loss function optimizes the defect type judgment, the regression loss function improves the spatial positioning accuracy, and the direction perception loss function enhances the direction recognition capability; the multi-task decoupling loss function enables each task to be optimized independently and converge collaboratively, avoiding interference between tasks.

[0089] For example, this embodiment can collect printed circuit board sample images of various soldering methods, label the defect type, location, size and orientation, and construct a training dataset; generate a classification loss function based on cross-entropy loss combined with sample perception weights; design a regression loss function using normalized Gaussian distance; construct an orientation perception loss function by smoothing angle differences using a sine function; assign weights to the three loss functions and sum them to obtain a multi-task decoupling loss function; input the training dataset into the initial model, iteratively train based on this loss function until the model converges, and obtain a defect detection model.

[0090] For example, there are numerous PCB inspection standards, among which the most widely recognized in the industry is the IPC standard, developed by the IPC (Institute for Printed Circuits). It covers the entire PCB manufacturing process and sets detailed acceptance criteria. In particular, the IPC-A-600 standard focuses on the visual quality inspection of PCBs, including the assessment of visible surface defects. In this embodiment, the PCB dataset is acquired using a linear scanning CCD camera with a resolution of approximately 48 pixels per millimeter. After manually filtering and removing defect-free template images from the sampled images, the initial size of the templates and test images is typically 16000×16000 pixels. These images are then cropped into multiple 640×640 pixel sub-images and calibrated using template matching technology. Furthermore, a threshold is set for image binarization to reduce interference caused by lighting, ultimately generating a PCB dataset containing 1500 images.

[0091] For example, for the defect classification task, this embodiment adopts a classification task loss function to solve the problems of class imbalance and imbalance between easy and difficult samples; for the regression task of defect location and size, a regression task-based loss function is designed; for common directional defects on PCBs, such as bridging and spikes, an orientation-aware loss function is designed. The initial model is trained based on the training dataset and the multi-task decoupling loss function to obtain the defect detection model.

[0092] For example, sample imbalance is particularly prominent in PCB defect detection scenarios. The sample-aware weighting mechanism employed in this solution incorporates three key design dimensions. Specifically, the class balance factor uses a modified inverse frequency weighting method, introducing a smoothing coefficient to avoid assigning excessive weight to extremely rare classes. This non-linear adjustment method alleviates class imbalance, prevents the model from over-focusing on rare classes at the expense of overall performance, and maintains consistent detection accuracy for common defects. The dynamic hard sample focusing strategy allows the model to adaptively adjust its focus on hard samples at different training stages, avoiding the over-focusing problem in the later stages of training caused by traditional loss function values. The spatial importance weight generates a weight distribution map centered on the defect using a Gaussian kernel function, emphasizing the importance of the defect region at the pixel level. This covers the surrounding areas of typical defects without introducing excessive background noise.

[0093] In this embodiment, traditional regression loss functions face two main challenges in PCB defect detection: insufficient sensitivity to small defects and lack of orientation awareness. This embodiment employs normalized Gaussian distance loss to address these issues. This method is more sensitive to small offsets in the bounding box, automatically encodes orientation information through the covariance matrix, and provides a continuous metric space, which is beneficial for gradient propagation. In this embodiment, the detection of orientation-sensitive defects such as bridging and spikes requires a special angle representation method. This embodiment uses a periodic loss function based on a sine function to solve the two inherent problems of traditional angle regression.

[0094] As shown above, this embodiment significantly improves the performance of the defect detection model by constructing a multi-task decoupling loss function. The classification loss function, combined with sample-aware weights, effectively solves the problems of class imbalance and imbalance between easy and difficult samples, enhancing the ability to identify rare and difficult defects. The regression loss function, based on normalized Gaussian distance, improves the sensitivity to small defect offsets and optimizes the accuracy of position and size prediction by encoding directional information through the covariance matrix. The direction-aware loss uses a sine function to handle the periodicity of angles, accurately detecting direction-sensitive defects such as bridging and pointed defects. The multi-task decoupling design enables collaborative optimization of each task, and combined with diverse sample training, it allows the model to adapt to various soldering methods, significantly improving the accuracy and robustness of printed circuit board soldering defect detection.

[0095] Corresponding to the machine vision-based welding quality inspection method in the above embodiments, Figure 2 This is a structural block diagram of a machine vision-based welding quality inspection device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The machine vision-based welding quality inspection device 20 includes an image acquisition module 21 and a quality inspection module 22.

[0096] The image acquisition module is used to acquire the target image to be detected, which is a printed circuit board.

[0097] The quality inspection module is used to input the target image to be inspected into the defect detection model to obtain the defect detection results; the defect detection results are used to characterize the defect type of the printed circuit board.

[0098] The defect detection methods include:

[0099] Initial feature extraction is performed on the target image to be detected to obtain initial image features;

[0100] Multi-scale feature extraction is performed on the initial image features to obtain multiple high-level semantic features at different scales;

[0101] Feature fusion is performed on multiple high-level semantic features at different scales to obtain fused features;

[0102] Defect detection is performed based on fusion features to obtain defect detection results.

[0103] In one embodiment of this application, the defect detection model includes a convolutional layer; the quality detection module 22 is specifically used to: extract initial features from the target image to be detected through the convolutional layer to obtain initial image features.

[0104] In one embodiment of this application, the quality inspection module 22 is further configured to:

[0105] By sequentially performing depthwise separable convolution on the initial image features using multiple dilation rates, high-level semantic features at multiple different scales are obtained.

[0106] In one embodiment of this application, the quality detection module 22 is further configured to: connect multiple high-level semantic features of different scales based on upsampling and downsampling operations to obtain fused features.

[0107] In one embodiment of this application, the quality inspection module 22 is further configured to:

[0108] Upsampling is performed on multiple high-level semantic features below the target scale to obtain multiple sampled first high-level semantic features;

[0109] For multiple high-level semantic features that are higher than the target scale, downsampling operations are performed to obtain multiple sampled second high-level semantic features; the multiple first high-level semantic features and the multiple second high-level semantic features have the same resolution;

[0110] For each of the multiple first-level high-level semantic features and multiple second-level high-level semantic features, a third-level high-level semantic feature is obtained by fusing the channel information of the high-level semantic feature through a convolutional layer.

[0111] All third-level semantic features have the same number of channels;

[0112] All third-level semantic features are concatenated along the channel dimension.

[0113] In one embodiment of this application, the machine vision-based welding quality inspection device 20 further includes: a training module, used to acquire multiple sample image sets containing printed circuit boards, and use the multiple sample image sets containing printed circuit boards as training datasets; wherein, each sample image set containing printed circuit boards corresponds to a welding method of printed circuit boards.

[0114] A multi-task decoupling loss function is constructed based on classification loss function, regression loss function and orientation-aware loss function;

[0115] The initial model is trained based on the training dataset and the multi-task decoupling loss function to obtain the defect detection model.

[0116] In one embodiment of this application, the training module is specifically used to construct a classification loss function based on a standard category loss function and sample-aware weights;

[0117] A regression loss function is constructed based on normalized Gaussian distance;

[0118] A direction-aware loss function is constructed based on the sine function to smooth out angle differences.

[0119] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 2 The functions of the image acquisition module 21 and the quality detection module 22 shown are illustrated.

[0120] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0121] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0122] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory.

[0123] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in the machine vision-based welding quality inspection method provided in the embodiments of this application, or they can execute the implementation methods of the electronic devices described in the embodiments of this application, which will not be repeated here.

[0124] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0125] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0126] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0127] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0130] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0131] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A welding quality inspection method based on machine vision, characterized in that, include: Acquire a target image to be detected, wherein the target image to be detected is an image containing a printed circuit board; The target image to be detected is input into the defect detection model for defect detection to obtain the defect detection result, which is used to characterize the defect type of the printed circuit board. The defect detection methods include: Initial feature extraction is performed on the target image to be detected to obtain initial image features; The initial image features are sequentially dilated using depthwise separable convolutions at multiple dilation rates to obtain multiple high-level semantic features at different scales. The formula for obtaining these high-level semantic features at different scales by sequentially dilating the initial image features using depthwise separable convolutions at multiple dilation rates is expressed as follows: in, These are high-level semantic features at multiple different scales. For depthwise convolution, and They are 3x3 convolution kernels and 5x5 convolution kernels, respectively. For batch normalization operations, This is a morphological filtering operation. Initial image features; For multiple high-level semantic features below the target scale, upsampling operations are performed to obtain multiple sampled first high-level semantic features; for multiple high-level semantic features above the target scale, downsampling operations are performed to obtain multiple sampled second high-level semantic features; the multiple first high-level semantic features and the multiple second high-level semantic features have the same resolution; for each of the multiple first high-level semantic features and the multiple second high-level semantic features, channel information is fused through a convolutional layer to obtain a third high-level semantic feature; all third high-level semantic features have the same number of channels; all third high-level semantic features are concatenated by channel dimension to obtain a fused feature; Defect detection is performed based on the fusion features to obtain the defect detection results; The method further includes: acquiring multiple sample image sets containing printed circuit boards, and using the multiple sample image sets containing printed circuit boards as training datasets; wherein each sample image set containing printed circuit boards corresponds to a soldering method of printed circuit boards; constructing a classification loss function based on a standard class loss function and sample perception weights; constructing a regression loss function based on normalized Gaussian distance; constructing a direction perception loss function based on sine function smoothing angle differences; constructing a multi-task decoupling loss function based on the classification loss function, regression loss function, and direction perception loss function; and training an initial model based on the training dataset and the multi-task decoupling loss function to obtain the defect detection model.

2. The welding quality inspection method based on machine vision as described in claim 1, characterized in that, The defect detection model includes convolutional layers; The initial feature extraction of the target image to be detected, to obtain initial image features, includes: The initial image features are obtained by extracting initial features from the target image to be detected through the convolutional layer.

3. A welding quality inspection device based on machine vision, characterized in that, include: An image acquisition module is used to acquire an image of a target to be detected, wherein the target image is a printed circuit board; The quality inspection module is used to input the target image to be inspected into the defect detection model to obtain the defect detection result; The defect detection results are used to characterize the defect types of the printed circuit board; The defect detection methods include: Initial feature extraction is performed on the target image to be detected to obtain initial image features; The initial image features are sequentially dilated using depthwise separable convolutions at multiple dilation rates to obtain multiple high-level semantic features at different scales. The formula for obtaining these high-level semantic features at different scales by sequentially dilating the initial image features using depthwise separable convolutions at multiple dilation rates is expressed as follows: in, These are high-level semantic features at multiple different scales. For depthwise convolution, and They are 3x3 convolution kernels and 5x5 convolution kernels, respectively. For batch normalization operations, This is a morphological filtering operation. Initial image features; For multiple high-level semantic features below the target scale, upsampling operations are performed to obtain multiple sampled first high-level semantic features; for multiple high-level semantic features above the target scale, downsampling operations are performed to obtain multiple sampled second high-level semantic features; the multiple first high-level semantic features and the multiple second high-level semantic features have the same resolution; for each of the multiple first high-level semantic features and the multiple second high-level semantic features, channel information is fused through a convolutional layer to obtain a third high-level semantic feature; all third high-level semantic features have the same number of channels; all third high-level semantic features are concatenated by channel dimension to obtain a fused feature; Defect detection is performed based on the fusion features to obtain the defect detection results; The training module is used to acquire multiple sample image sets containing printed circuit boards (PCBs), and use these multiple sample image sets as training datasets. Each sample image set containing a PCB corresponds to a soldering method of the PCB. A classification loss function is constructed based on a standard class loss function and sample perception weights. A regression loss function is constructed based on normalized Gaussian distance. A direction perception loss function is constructed based on sine function smoothing of angle differences. A multi-task decoupling loss function is constructed based on the classification loss function, regression loss function, and direction perception loss function. The initial model is trained based on the training dataset and the multi-task decoupling loss function to obtain the defect detection model.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 2.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 2.

Images