Real-time PCB appearance defect detection algorithm based on edge computing

Abstract

The application discloses a PCB appearance defect real-time detection algorithm based on edge computing and belongs to the technical field of computer vision and intelligent manufacturing. The algorithm extracts multi-scale features of a PCB surface image through a multi-modal feature extraction network, realizes feature dimension compression under edge computing resource constraints in combination with a lightweight convolutional neural network, and balances the calculation efficiency and the detection accuracy. An adaptive attention algorithm is used to dynamically adjust the weight distribution of local defect features and global structure features, and the saliency expression of micro defects is enhanced. Through a process feature mapping mechanism of a PCB process parameter database and in combination with a temperature sensitivity coefficient of real-time environmental temperature sensing, the adaptability of detection results to production conditions is improved. A lightweight classifier is used to realize low-delay defect classification decision.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent manufacturing technology, and in particular relates to a real-time detection algorithm for PCB appearance defects based on edge computing. Background Technology

[0002] Printed Circuit Boards (PCBs) are a key component of modern electronic devices, and their manufacturing quality directly affects the performance and reliability of electronic products. In the PCB production process, appearance defect detection is a crucial step in ensuring product quality. Traditional inspection methods have several shortcomings: manual visual inspection is inefficient, subjective, and prone to fatigue, making it difficult to keep pace with automated production lines; automated inspection systems based on traditional image processing algorithms have limited accuracy when faced with complex and diverse PCB defect features, lacking the ability to identify small and hidden defects, and are prone to misjudgments and missed detections. With the continuous evolution of electronic devices, the integration and complexity of PCBs are increasing, and the types and forms of their appearance defects are becoming increasingly diverse. Against this backdrop, edge computing technology has emerged, pushing computing, data storage, and processing capabilities to the network edge, closer to data sources and users, providing new ideas and solutions for real-time PCB appearance defect detection. Edge computing offers advantages such as low latency, high bandwidth, and localized data processing, perfectly meeting the needs of real-time PCB inspection for rapid response, efficient processing, and data privacy protection. Deep learning technology, especially convolutional neural networks, has achieved great success in the field of image recognition. However, conventional deep learning models often have a large number of parameters and consume a lot of computing resources, making them difficult to deploy directly in resource-constrained edge computing environments. Summary of the Invention

[0003] Therefore, it is necessary to provide a real-time PCB appearance defect detection algorithm based on edge computing that can solve the above-mentioned technical problems.

[0004] Firstly, this application provides a real-time PCB appearance defect detection algorithm based on edge computing, including:

[0005] A multimodal feature extraction network is used to process PCB surface image data to obtain a first feature set containing multi-scale features;

[0006] Based on the constraints of edge computing resources, a lightweight convolutional neural network is used to compress the feature dimension of the first feature set to obtain the second feature set;

[0007] The local defect features and global structural features of the second feature set are extracted, and the feature weights are dynamically adjusted according to the defect scale and type using an adaptive attention algorithm to obtain the third feature set;

[0008] The process features in the third feature set are nonlinearly mapped and associated with the preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information.

[0009] A lightweight classifier is used to perform real-time detection on the fourth feature set to generate defect detection results.

[0010] In one embodiment, the lightweight convolutional neural network includes the following lightweight feature compression operator:

[0011]

[0012]

[0013] in, , representing the input feature tensor , indicating a depthwise separable convolution kernel, This represents the depthwise separable convolution operator. This represents the batch normalization function. Represents a non-linear activation function. This indicates the channel rearrangement operator. , which represents the upper bound of the Lipschitz constant.

[0014] In one embodiment, the adaptive attention algorithm includes the following formula:

[0015]

[0016]

[0017] in, Indicates position Feature weights, This represents a measure of the salience of local defect features. Indicates feature map in gradient vector, Represents the numerical stability constant. This represents a fusion of global structural features and technological priors. This represents a trainable weight matrix. This indicates the results of a process database query. , representing a nonlinear enhancement function for local defect characteristics, , which represents the normalization function of global structural features.

[0018] In one embodiment, the process features of the third feature set are non-linearly mapped and associated with the process features contained in a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information, including:

[0019] The following process parameter correlation function is used to perform process feature mapping and correlation to generate a fourth feature set containing process adaptation information:

[0020]

[0021]

[0022] in, Representing the third feature set With the i-th process parameter The degree of correlation, This represents the feature vector after adaptive attention. This represents the i-th parameter vector in the process parameter database. Represents the inner product of the elasticity norm. This indicates that process parameters are mapped to a high-dimensional semantic space. Represents the elasticity norm mixing coefficient. Describing the L1 norm, Describing the L2 norm, This represents the temperature sensitivity coefficient, where t represents the real-time ambient temperature of the PCB. This indicates the standard test temperature corresponding to the process parameter pi. The normalized decay function represents the temperature difference.

[0023] In one embodiment, a lightweight classifier is used to perform real-time detection on the fourth feature set to generate defect detection results, including:

[0024] The fourth feature set is used for real-time detection using the following formula to generate defect detection results:

[0025]

[0026]

[0027] in, This indicates the final defect detection result. This indicates that all possible categories have been traversed. This represents the feature vector after process mapping association. This represents the weight coefficient of the m-th feature corresponding to the c-th type of defect. This represents a dynamic feature selection operator. This indicates the number of features selected based on real-time available resources. This represents the characteristic attenuation coefficient of temperature sensing on edge devices. Indicates the temperature sensitivity coefficient. Indicates the rated operating temperature threshold of the equipment. Indicates the real-time available memory of the edge device. This represents the Hadamard product with temperature compensation for eigenvalues.

[0028] In one embodiment, the algorithm further includes:

[0029] By integrating the fourth feature set with multimodal sensing features, and using residual connection networks to jointly represent minute defects and large-scale anomalies in the features, a fifth feature set containing defect classification information is generated.

[0030] Multi-class defect probability prediction is performed on the fifth feature set, the confidence score of each defect category is calculated, and compared with the preset dynamic confidence threshold:

[0031] If the confidence score of any defect category is lower than the corresponding threshold, the attention weight of the fifth feature set is optimized using the range adjustment algorithm to generate the sixth feature set and input it into the lightweight classifier.

[0032] Otherwise, the fifth feature set is directly used as the sixth feature set and input into the lightweight classifier;

[0033] A lightweight classifier is used to detect defects in the sixth feature set in real time, and the defect detection results are output.

[0034] In one embodiment, the algorithm further includes:

[0035] False positives and false negatives are acquired in real time and stored in a circular buffer.

[0036] When the number of samples in the cache reaches a preset threshold:

[0037] The stochastic gradient descent algorithm with momentum is used to adjust the parameters of the cross-modal fusion layer in the multimodal feature extraction network, and the FocalLoss function is used to adjust the parameters of the classifier's output layer.

[0038] Secondly, this application also provides a real-time PCB appearance defect detection device based on edge computing, comprising:

[0039] The multimodal feature extraction module is used to process PCB surface image data using a multimodal feature extraction network to obtain a first feature set containing multi-scale features;

[0040] The feature dimension compression module is used to compress the feature dimensions of the first feature set based on edge computing resource constraints, using a lightweight convolutional neural network to obtain the second feature set;

[0041] The adaptive attention calculation module is used to extract local defect features and global structural features of the second feature set, and dynamically adjusts the feature weights according to the defect scale and type through the adaptive attention algorithm to obtain the third feature set;

[0042] The process feature mapping module is used to perform non-linear mapping association between the process features of the third feature set and the process features contained in the preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information.

[0043] The real-time classification decision module is used to perform real-time detection on the fourth feature set using a lightweight classifier and generate defect detection results.

[0044] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned steps of the real-time detection algorithm for PCB appearance defects based on edge computing.

[0045] Fourthly, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned real-time PCB appearance defect detection algorithm based on edge computing.

[0046] The aforementioned edge computing-based real-time PCB appearance defect detection algorithm, device, computer equipment, and storage medium achieve multi-scale feature fusion of PCB surface images through a multimodal feature extraction network. Combined with feature dimension compression technology of a lightweight convolutional neural network, it reduces model parameter quantity and computational complexity while maintaining detection accuracy, effectively solving the resource bottleneck problem of traditional deep learning models when deployed on edge devices. An adaptive attention algorithm dynamically balances the weight allocation of local defect features and global structural features, and a nonlinear enhancement function strengthens the gradient feature expression of minute defects, overcoming the deficiency of conventional algorithms in sensitivity to minute defects. Through the elastic norm correlation mechanism of the process parameter database and temperature sensitivity coefficient compensation, production environment parameters are dynamically integrated into the feature space, improving the robustness of detection results to complex industrial scenarios. A lightweight classifier combined with a dynamic feature selection operator adaptively adjusts the computation path based on real-time resource status and edge device temperature, achieving an optimal balance between detection accuracy and computational efficiency while maintaining low-latency decision-making, comprehensively addressing the technical shortcomings of traditional detection methods in terms of real-time performance, accuracy, and environmental adaptability. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a real-time PCB appearance defect detection algorithm based on edge computing according to the present invention.

[0049] Figure 2 This is a structural diagram of a real-time PCB appearance defect detection device based on edge computing according to the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] This application discloses a hardware architecture for a real-time PCB appearance defect detection algorithm based on edge computing, comprising an industrial camera and sensor, a computing terminal, and a process database. The industrial camera and sensor provide multi-source data input, the computing terminal performs efficient feature compression and dynamic decision-making, and the process database is continuously optimized. Inter-hardware communication protocols enable closed-loop control.

[0052] In one embodiment, such as Figure 1 As shown, a real-time PCB appearance defect detection algorithm based on edge computing is provided. This embodiment illustrates the algorithm's application to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the algorithm includes the following steps:

[0053] S101 uses a multimodal feature extraction network to process PCB surface image data and obtain a first feature set containing multi-scale features.

[0054] Among them, the multimodal feature extraction network is a neural network architecture capable of fusing and extracting features from multiple different types of data, i.e., multimodal data features. It can simultaneously process and analyze different aspects of data from PCB (printed circuit board) surface images, such as image features at different resolutions and spectral features at different wavelengths. PCB surface image data refers to images of the PCB surface acquired through industrial cameras or other imaging equipment, containing various information about the PCB surface, such as the shape of the lines, the quality of the solder joints, and the location of the holes. After data processing, a first feature set containing multi-scale PCB surface features is obtained.

[0055] S102, based on the constraints of edge computing resources, a lightweight convolutional neural network is used to compress the feature dimension of the first feature set to obtain the second feature set.

[0056] The resource constraints of edge computing refer to the fact that edge devices, such as embedded systems and industrial controllers, typically have limited computing power, memory, and energy supply. Algorithms running on these devices need to meet these resource constraints to minimize computational load and memory consumption. By employing an optimized convolutional neural network designed to operate efficiently in resource-constrained environments, the first feature set is compressed. This reduces the dimensionality of the features while retaining important feature information. While maintaining feature expressive power, it reduces the computational burden of subsequent processing, resulting in a lower-dimensional second feature set that is more suitable for further processing and analysis in edge computing environments.

[0057] S103, extract the local defect features and global structural features of the second feature set, and dynamically adjust the feature weights according to the defect scale and type using an adaptive attention algorithm to obtain the third feature set.

[0058] Local defect features are defined as follows: local defect features (small areas in the PCB surface image) are used to capture features of local defects such as scratches, holes, and uneven solder joints, for identifying defects in specific locations; global structural features are defined as the large-scale structure and layout of the PCB surface image, such as circuit routing and component distribution, which help in understanding the overall structure and layout of the PCB. An adaptive attention algorithm is used to automatically assign different weights to local defect features and global structural features based on the scale and type of the defect. For defect scale: small local defects are given higher weights, while larger defects or structural problems are given higher weights to global structural features. For defect type: different types of defects may be more pronounced in local or global features. For example, uneven solder joints are mainly reflected in local features, while circuit layout errors are more reflected in global features. The feature weights are automatically adjusted according to the defect type to highlight the features that best represent the essence of the defect. The resulting third feature set allows the feature representation to better meet the needs of actual defect detection.

[0059] S104, perform nonlinear mapping association between the process features of the third feature set and the process features contained in the preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information.

[0060] The pre-set PCB manufacturing process feature library stores a large amount of data on known PCB manufacturing process parameters and their corresponding appearance features. This data is representative and instructive, serving as a standard reference to help determine whether the current PCB's process meets requirements. Since the relationship between PCB process features and appearance features is often not a simple linear one but rather a complex nonlinear correlation, a nonlinear mapping method can more accurately describe and associate this complex relationship. Nonlinear patterns in the data can be automatically captured using learning algorithms such as neural networks and support vector machines. The process feature vectors from the third feature set are input into the nonlinear mapping model. Based on the standard process parameters and their corresponding appearance features in the pre-set PCB manufacturing process feature library, the model finds the combination of process parameters most similar or most correlated with the current process feature, determining the position and similarity of the current PCB's process feature in the database. A fourth feature set containing process adaptation information is generated, reflecting the degree of correlation and differences with the standard process features.

[0061] S105 uses a lightweight classifier to perform real-time detection on the fourth feature set and generates defect detection results.

[0062] The lightweight classifier is a classification algorithm designed to run efficiently in resource-constrained environments. When performing real-time detection on the fourth feature set, it quickly generates defect detection results based on pre-trained model parameters and classification rules. These results include the determination of whether a defect exists, the type of defect such as scratches, holes, uneven solder joints, the location information of the defect such as its coordinates or region in the PCB image, and the severity of the defect such as minor, moderate, or severe.

[0063] This invention provides a real-time PCB appearance defect detection algorithm based on edge computing. It processes PCB surface image data through a multimodal feature extraction network, fusing multiple data features. Under edge computing resource constraints, a lightweight convolutional neural network is used to compress the dimension of a first feature set to obtain a second feature set, adapting to the limited resources of edge devices. Local defect features and global structural features are extracted from the second feature set, and an adaptive attention algorithm dynamically adjusts feature weights based on defect scale and type to obtain a third feature set, enhancing the specificity of feature expression. The process features in the third feature set are nonlinearly mapped and associated with a pre-defined PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information, achieving dynamic integration of process parameters. A lightweight classifier is used to perform real-time detection on the fourth feature set, generating detection results containing information such as defect judgment, type, location, and severity, meeting the real-time requirements of the production line. By integrating technologies such as multimodal data fusion, lightweight model design, adaptive feature adjustment, and process mapping correlation, this method effectively solves the problems of low efficiency, strong subjectivity, insufficient ability to identify small and hidden defects, difficulty in adapting to complex and diverse defect features, and difficulty in deploying deep learning models in edge computing environments in traditional PCB appearance defect detection methods. It improves the accuracy, efficiency, and adaptability of PCB appearance defect detection, reduces deployment costs, realizes real-time monitoring and optimization of the production process, and promotes the intelligent development of the PCB manufacturing industry.

[0064] In one embodiment, the lightweight convolutional neural network includes the following lightweight feature compression operator:

[0065]

[0066]

[0067] in, , representing the input feature tensor , indicating a depthwise separable convolution kernel, This represents the depthwise separable convolution operator. This represents the batch normalization function. Represents a non-linear activation function. This indicates the channel rearrangement operator. , which represents the upper bound of the Lipschitz constant.

[0068] For example, the input feature tensor Typically, it's a multi-dimensional array containing feature information extracted from PCB surface images. This information can come from a multimodal feature extraction network and exhibits multi-scale and multi-channel characteristics. Batch normalization function. Used to normalize feature maps, by standardizing each feature value to zero mean and unit variance, accelerating the network training process and improving the model's stability and generalization ability. Non-linear activation function. Used to introduce non-linear characteristics, enabling the network to learn complex feature representations, setting negative features to zero, and retaining positive features, thereby enhancing the model's expressive power and robustness. Channel rearrangement operator. Used to rearrange feature channels and optimize feature representation. By rearranging channels, the correlation between features can be better utilized, improving model efficiency and performance. Upper bound of the Lipschitz constant. The Lipschitz continuity is used to control the network. Lipschitz continuity ensures that changes in the network output do not exceed a certain multiple of changes in the input, thus improving the model's stability and robustness. By using lightweight feature compression operators such as depthwise separable convolutional kernels, batch normalization functions, nonlinear activation functions, and channel rearrangement operators, computational cost and memory usage can be significantly reduced while maintaining feature expressiveness. The combination of these operators not only improves the model's efficiency but also ensures its real-time performance and stability on resource-constrained edge devices. By controlling the upper bound of the Lipschitz constant, the model's robustness and noise resistance are further enhanced, enabling lightweight convolutional neural networks to operate efficiently and accurately in PCB surface defect detection.

[0069] In one embodiment, the adaptive attention algorithm includes the following formula:

[0070]

[0071]

[0072] in, Indicates position Feature weights, This represents a measure of the salience of local defect features. Indicates feature map in gradient vector, Represents the numerical stability constant. This represents a fusion of global structural features and technological priors. This represents a trainable weight matrix. This indicates the results of a process database query. , representing a nonlinear enhancement function for local defect characteristics, , which represents the normalization function of global structural features.

[0073] Specifically, the feature weights at each position (i,j) are dynamically calculated using an adaptive attention mechanism. This is used for subsequent feature fusion, and the weight value reflects the importance of this location in defect detection. Local defect feature saliency measurement. Through gradient vector It captures edge or texture variations in localized areas, and for large-scale defects such as large-area stains, the gradient distribution range is wide. The numerator is generally higher, but the denominator will also increase, ultimately... It may tend to level off, at which point the model dependency... Global process knowledge suppresses misjudgments. For small-scale defects such as fine scratches: the gradient is concentrated locally. The numerator is significantly higher than that of the surrounding region, and the denominator is normalized due to the global expectation. It remains at a high value at this time. Weights are increased through nonlinear enhancements such as beta amplification. A fusion representation of global structural features and process priors is employed. gradient vector Results of querying the process database Fusion is achieved by suppressing outliers using the tanh activation function. For process-related defects, such as standard solder joint deviations, Prior features containing similar defects, after fusion The value is relatively high. The output approaches 1, thus increasing the weight of this type of defect. For non-process defects such as random stains... , Unrelated features The value is low. The output approaches 0, suppressing the weight of this type of defect. Nonlinear enhancement function. The enhancement intensity of local features is controlled by β. The normalization function representing global structural features can be a fusion feature... Compressing to the (−1,1) interval balances the contributions of local and global features. Defect scales are distinguished based on gradient strength. This formula utilizes... Defect scales are distinguished based on gradient intensity; utilizing and Adjusting the sensitivity of local features. Utilizing and By combining process knowledge to distinguish defect types, the model can efficiently process real-time data in edge computing scenarios while also taking into account the complex characteristics of different defects.

[0074] In one embodiment, the process features of the third feature set are non-linearly mapped and associated with the process features contained in a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information, including:

[0075] The following process parameter correlation function is used to perform process feature correlation, generating a fourth feature set containing process adaptation information:

[0076]

[0077]

[0078] in, Representing the third feature set With the i-th process parameter The degree of correlation, This represents the feature vector after adaptive attention. This represents the i-th parameter vector in the process parameter database. Represents the inner product of the elasticity norm. This indicates that process parameters are mapped to a high-dimensional semantic space. Represents the elasticity norm mixing coefficient. Describing the L1 norm, Describing the L2 norm, This represents the temperature sensitivity coefficient, where t represents the real-time ambient temperature of the PCB. This indicates the standard test temperature corresponding to the process parameter pi. The normalized decay function represents the temperature difference.

[0079] For example, the elasticity norm inner product Feature vectors X are mapped to a high-dimensional semantic space M. Similarity is calculated by mixing L1 and L2 norms to balance the sparsity and smoothness of features, transforming the original features into more discriminative representations. For example, features such as PCB surface texture and color are mapped to a defect-sensitive semantic space. A normalized decay function for temperature differences is also used. Based on real-time temperature t and standard test temperature The correlation is dynamically adjusted based on temperature differences; the greater the temperature difference, the more significant the correlation decay. During PCB manufacturing, temperature variations affect material properties such as solder flow and coating adhesion. By using a normalized decay function for temperature differences, the model can automatically reduce the correlation confidence at non-standard temperatures, thus reducing false positives. Normalize the inner product of the elasticity norm to ensure that the correlation degree ranges between (-1, 1). Combine feature similarity and temperature factors to calculate the final correlation degree. .

[0080] In one embodiment, a lightweight classifier is used to perform real-time detection on the fourth feature set to generate defect detection results, including:

[0081] The fourth feature set is used for real-time detection using the following formula to generate defect detection results:

[0082]

[0083]

[0084] in, This indicates the final defect detection result. This indicates that all possible categories have been traversed. This represents the feature vector after process correlation. This represents the weight coefficient of the m-th feature corresponding to the c-th type of defect. This represents a dynamic feature selection operator. This indicates the number of features selected based on real-time available resources. This represents the characteristic attenuation coefficient of temperature sensing on edge devices. Indicates the temperature sensitivity coefficient. Indicates the rated operating temperature threshold of the equipment. Indicates the real-time available memory of the edge device. This represents the Hadamard product with temperature compensation for eigenvalues.

[0085] For example, dynamic feature selection operator Based on real-time available memory Dynamically select the most important This feature reduces computational complexity and improves real-time performance. Temperature-sensing feature attenuation coefficient. Based on the importance of real-time temperature decay characteristics, this formula suppresses noise or false detections caused by temperature changes, such as temperature fluctuations during PCB manufacturing leading to changes in sensor sensitivity or shifts in material properties. Using this formula reduces redundant calculations, adapts to memory fluctuations, suppresses temperature interference, improves defect classification robustness, and leverages weights. Tracing the contribution of key features to classification.

[0086] In one embodiment, the algorithm further includes:

[0087] By integrating the fourth feature set with multimodal sensing features, and using residual connection networks to jointly represent minute defects and large-scale anomalies in the features, a fifth feature set containing defect classification information is generated.

[0088] Multi-class defect probability prediction is performed on the fifth feature set, the confidence score of each defect category is calculated, and compared with the preset dynamic confidence threshold:

[0089] If the confidence score of any defect category is lower than the corresponding threshold, the attention weight of the fifth feature set is optimized using the range adjustment algorithm to generate the sixth feature set and input it into the lightweight classifier.

[0090] Otherwise, the fifth feature set is directly used as the sixth feature set and input into the lightweight classifier;

[0091] A lightweight classifier is used to detect defects in the sixth feature set in real time, and the defect detection results are output.

[0092] Specifically, the fourth feature set is combined with multimodal sensing features such as temperature and vibration (non-visual data), and a residual connection network is used for joint representation. By preserving the original features and introducing auxiliary information, the model's ability to collaboratively capture minute defects and large-scale anomalies is enhanced, avoiding information loss in deep networks. The fused fifth feature set is then classified, and probability scores for each defect category, such as scratches and holes, are output.

[0093] The threshold is adjusted based on real-time production conditions such as temperature fluctuations and equipment status. For example, in high-temperature environments, the threshold can be relaxed to reduce the possibility of missed detections and ensure the reliability of classification results. If the confidence score of any category falls below the threshold, a range adjustment algorithm is triggered to dynamically optimize the attention weights of the fifth feature set, such as increasing the weights of high-gradient regions and suppressing the weights of background noise or irrelevant features. The optimized features are then re-input into the lightweight classifier to improve classification accuracy. If the confidence scores of all categories meet the threshold, the output of the fifth feature set is directly used to avoid redundant calculations and ensure real-time performance.

[0094] In one embodiment, the algorithm further includes:

[0095] False positives and false negatives are acquired in real time and stored in a circular buffer.

[0096] When the number of samples in the cache reaches a preset threshold:

[0097] The stochastic gradient descent algorithm with momentum is used to adjust the parameters of the cross-modal fusion layer in the multimodal feature extraction network, and the FocalLoss function is used to adjust the parameters of the classifier's output layer.

[0098] For example, classification results can be monitored in real time, and samples that are falsely detected (normal samples being classified as defective) and falsely detected (defective samples being classified as normal), along with their true labels, are stored in a circular buffer. A FIFO (First In First Out) queue can be used to ensure controllable memory usage while retaining recently occurring typical error samples. When the number of samples in the buffer reaches a preset threshold, the model parameter update process is initiated, which can avoid interference from single sample noise and improve the stability and convergence speed of parameter updates. A stochastic gradient descent algorithm with momentum is used to introduce historical gradient information, accelerate parameter convergence, suppress oscillations, and efficiently handle non-convex optimization problems. Based on the gradient of the loss function of the true labels of falsely detected / falsely detected samples in the buffer and the model prediction results, backpropagation is used to adjust the parameters of the cross-modal fusion layer in the multimodal feature extraction network. The FocalLoss function is used to reduce the loss contribution of samples with high probability of prediction, such as samples that are clearly without defects, to avoid the model overfitting simple samples; for samples that are difficult to distinguish, their loss weight is amplified, forcing the model to focus on error-prone samples. By dynamically capturing and batch optimizing false / missing samples, combined with multimodal feature fusion and robust loss function design, the accuracy and stability of PCB defect detection are significantly improved.

[0099] This application presents a real-time PCB appearance defect detection algorithm based on edge computing. It achieves PCB appearance defect detection in an edge computing environment by fusing the multi-scale analysis capabilities of a multimodal feature extraction network with the resource-constrained adaptability of a lightweight convolutional neural network. An adaptive attention mechanism dynamically optimizes the weight allocation between local defect features and global structural features, combined with nonlinear mapping and correlation techniques using a process parameter database, to achieve significant enhancement of the detection of minute defects and robust characterization of complex industrial scenarios. Furthermore, by fusing multimodal sensing features through a residual connection network, combined with dynamic confidence threshold determination and a closed-loop feedback optimization mechanism, the algorithm improves adaptability under complex operating conditions such as temperature fluctuations and equipment status changes. Under the premise of limited computing resources and ensuring real-time performance, it significantly improves the accuracy and generalization ability of defect classification, providing a low-latency, high-reliability visual inspection solution for high-density PCB intelligent manufacturing.

[0100] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0101] Based on the same inventive concept, this application also provides a computing device for implementing the aforementioned edge computing-based real-time PCB appearance defect detection algorithm. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more edge computing-based real-time PCB appearance defect detection algorithm embodiments provided below can be found in the above-described limitations of the edge computing-based real-time PCB appearance defect detection algorithm, and will not be repeated here.

[0102] In one exemplary embodiment, such as Figure 2 As shown, a computing device for real-time detection of PCB appearance defects based on edge computing is provided, comprising:

[0103] The multimodal feature extraction module 11 is used to process PCB surface image data using a multimodal feature extraction network to obtain a first feature set containing multi-scale features;

[0104] The feature dimension compression module 12 is used to compress the feature dimension of the first feature set based on edge computing resource constraints and using a lightweight convolutional neural network to obtain the second feature set.

[0105] The adaptive attention calculation module 13 is used to extract local defect features and global structural features of the second feature set, and dynamically adjust the feature weights according to the defect scale and type through the adaptive attention algorithm to obtain the third feature set;

[0106] The process feature association module 14 is used to perform nonlinear mapping association between the process features in the third feature set and the preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information.

[0107] The real-time classification decision module 15 is used to perform real-time detection on the fourth feature set using a lightweight classifier and generate defect detection results.

[0108] In one embodiment, the lightweight convolutional neural network in the feature dimension compression module 12 further includes the following lightweight feature compression operators:

[0109]

[0110]

[0111] in, , representing the input feature tensor , indicating a depthwise separable convolution kernel, This represents the depthwise separable convolution operator. This represents the batch normalization function. Represents a non-linear activation function. This indicates the channel rearrangement operator. , which represents the upper bound of the Lipschitz constant.

[0112] In one embodiment, the adaptive attention calculation module 13 further includes the following formula:

[0113]

[0114]

[0115] in, Indicates position Feature weights, This represents a measure of the salience of local defect features. Indicates feature map in gradient vector, Represents the numerical stability constant. This represents a fusion of global structural features and technological priors. This represents a trainable weight matrix. This indicates the results of a process database query. , representing a nonlinear enhancement function for local defect characteristics, , which represents the normalization function of global structural features.

[0116] In one embodiment, the process feature association module is further used for:

[0117] The following process parameter correlation function is used to perform process feature correlation, generating a fourth feature set containing process adaptation information:

[0118]

[0119]

[0120] in, Representing the third feature set With the i-th process parameter The degree of correlation, This represents the feature vector after adaptive attention. This represents the i-th parameter vector in the process parameter database. Represents the inner product of the elasticity norm. This indicates that process parameters are mapped to a high-dimensional semantic space. Represents the elasticity norm mixing coefficient. Describing the L1 norm, Describing the L2 norm, This represents the temperature sensitivity coefficient, where t represents the real-time ambient temperature of the PCB. This indicates the standard test temperature corresponding to the process parameter pi. The normalized decay function represents the temperature difference.

[0121] In one embodiment, the real-time classification decision module 15 is further configured to:

[0122] The fourth feature set is used for real-time detection using the following formula to generate defect detection results:

[0123]

[0124]

[0125] in, This indicates the final defect detection result. This indicates that all possible categories have been traversed. This represents the feature vector after process correlation. This represents the weight coefficient of the m-th feature corresponding to the c-th type of defect. This represents a dynamic feature selection operator. This indicates the number of features selected based on real-time available resources. This represents the characteristic attenuation coefficient of temperature sensing on edge devices. Indicates the temperature sensitivity coefficient. Indicates the rated operating temperature threshold of the equipment. Indicates the real-time available memory of the edge device. This represents the Hadamard product with temperature compensation for eigenvalues.

[0126] In one embodiment, the device is also used for:

[0127] By integrating the fourth feature set with multimodal sensing features, and using residual connection networks to jointly represent minute defects and large-scale anomalies in the features, a fifth feature set containing defect classification information is generated.

[0128] Multi-class defect probability prediction is performed on the fifth feature set, the confidence score of each defect category is calculated, and compared with the preset dynamic confidence threshold:

[0129] If the confidence score of any defect category is lower than the corresponding threshold, the attention weight of the fifth feature set is optimized using the range adjustment algorithm to generate the sixth feature set and input it into the lightweight classifier.

[0130] Otherwise, the fifth feature set is directly used as the sixth feature set and input into the lightweight classifier;

[0131] A lightweight classifier is used to detect defects in the sixth feature set in real time, and the defect detection results are output.

[0132] In one embodiment, the device is also used for:

[0133] False positives and false negatives are acquired in real time and stored in a circular buffer.

[0134] When the number of samples in the cache reaches a preset threshold:

[0135] The stochastic gradient descent algorithm with momentum is used to adjust the parameters of the cross-modal fusion layer in the multimodal feature extraction network, and the FocalLoss function is used to adjust the parameters of the classifier's output layer.

[0136] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a real-time PCB appearance defect detection algorithm based on edge computing as described above.

[0137] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described algorithm embodiments.

[0138] For the device embodiments, since they basically correspond to the algorithm embodiments, the relevant parts can be referred to in the description of the algorithm embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0139] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A real-time detection method for PCB appearance defects based on edge computing, characterized in that, The method includes: A multimodal feature extraction network is used to process PCB surface image data to obtain a first feature set containing multi-scale features; Based on the constraints of edge computing resources, a lightweight convolutional neural network is used to compress the feature dimension of the first feature set to obtain the second feature set; The local defect features and global structural features of the second feature set are extracted, and the feature weights are dynamically adjusted according to the defect scale and type using an adaptive attention algorithm to obtain the third feature set; The process features of the third feature set are nonlinearly mapped and associated with the process features contained in the preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information. A lightweight classifier is used to perform real-time detection on the fourth feature set to generate defect detection results; The adaptive attention algorithm includes the following formula: ； ； in, Indicates position Feature weights, This represents a measure of the salience of local defect features. Indicates feature map in gradient vector, Represents the numerical stability constant. This represents a fusion of global structural features and technological priors. This represents a trainable weight matrix. This indicates the results of a process database query. , representing a nonlinear enhancement function for local defect characteristics, , representing the normalization function of global structural features; The step of using a lightweight classifier to perform real-time detection on the fourth feature set and generate defect detection results includes: The fourth feature set is used for real-time detection using the following formula to generate defect detection results: ； ； in, This indicates the final defect detection result. This indicates that all possible categories have been traversed. This represents the feature vector after process correlation. This represents the weight coefficient of the m-th feature corresponding to the c-th type of defect. This represents a dynamic feature selection operator. This indicates the number of features selected based on real-time available resources. This represents the characteristic attenuation coefficient of temperature sensing on edge devices. Indicates the temperature sensitivity coefficient. Indicates the rated operating temperature threshold of the equipment. Indicates the real-time available memory of the edge device. This represents the Hadamard product with temperature compensation for eigenvalues.

2. The method according to claim 1, characterized in that, The lightweight convolutional neural network includes the following lightweight feature compression operators: ； ； in, , representing the input feature tensor , indicating a depthwise separable convolution kernel, This represents the depthwise separable convolution operator. This represents the batch normalization function. Represents a non-linear activation function. This indicates the channel rearrangement operator. , which represents the upper bound of the Lipschitz constant.

3. The method according to claim 1, characterized in that, The process features of the third feature set are nonlinearly mapped and associated with the process features contained in a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information, including: The following process parameter correlation function is used to perform nonlinear mapping and correlation of process features to generate a fourth feature set containing process adaptation information: ； ； in, Representing the third feature set With the i-th process parameter The degree of correlation, This represents the feature vector after adaptive attention. This represents the i-th parameter vector in the process parameter database. Represents the inner product of the elasticity norm. This indicates that process parameters are mapped to a high-dimensional semantic space. Represents the elasticity norm mixing coefficient. Describing the L1 norm, Describing the L2 norm, This represents the temperature sensitivity coefficient, where t represents the real-time ambient temperature of the PCB. This indicates the standard test temperature corresponding to the process parameter pi. The normalized decay function represents the temperature difference.

4. The method according to claim 1, characterized in that, The method further includes: By fusing the fourth feature set with multimodal sensing features, and using a residual connection network to jointly characterize the minute defects and large-scale anomalies in the features, a fifth feature set containing defect classification information is generated. Perform multi-category defect probability prediction on the fifth feature set, calculate the confidence score for each defect category, and compare it with a preset dynamic confidence threshold: If the confidence score of any defect category is lower than the corresponding threshold, the attention weight of the fifth feature set is optimized using the range adjustment algorithm to generate the sixth feature set and input into the lightweight classifier. Otherwise, the fifth feature set is directly input into the lightweight classifier as the sixth feature set; The lightweight classifier is used to detect the sixth feature set in real time, and the defect detection results are output.

5. The method according to claim 4, characterized in that, The method further includes: False positives and false negatives are acquired in real time and stored in a circular buffer. When the number of samples in the cache reaches a preset threshold: The parameters of the cross-modal fusion layer in the multimodal feature extraction network are adjusted using the stochastic gradient descent algorithm with momentum, and the parameters of the output layer of the classifier are adjusted using the FocalLoss function.

6. A real-time PCB appearance defect detection computing device based on edge computing, characterized in that, The device includes: The multimodal feature extraction module is used to process PCB surface image data using a multimodal feature extraction network to obtain a first feature set containing multi-scale features; The feature dimension compression module is used to compress the feature dimensions of the first feature set based on edge computing resource constraints, using a lightweight convolutional neural network to obtain the second feature set; An adaptive attention calculation module is used to extract local defect features and global structural features of the second feature set, and dynamically adjust the feature weights according to the defect scale and type through an adaptive attention algorithm to obtain a third feature set; The process feature association module is used to perform nonlinear mapping association between the process features of the third feature set and the process features contained in the preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information. The real-time classification decision module is used to perform real-time detection on the fourth feature set using a lightweight classifier and generate defect detection results. The adaptive attention algorithm includes the following formula: ； ； in, Indicates position Feature weights, This represents a measure of the salience of local defect features. Indicates feature map in gradient vector, Represents the numerical stability constant. This represents a fusion of global structural features and technological priors. This represents a trainable weight matrix. This indicates the results of a process database query. , representing a nonlinear enhancement function for local defect characteristics, , representing the normalization function of global structural features; The real-time classification decision module is further used to perform real-time detection on the fourth feature set using the following formula to generate defect detection results: The fourth feature set is used for real-time detection using the following formula to generate defect detection results: ； ； in, This indicates the final defect detection result. This indicates that all possible categories have been traversed. This represents the feature vector after process correlation. This represents the weight coefficient of the m-th feature corresponding to the c-th type of defect. This represents a dynamic feature selection operator. This indicates the number of features selected based on real-time available resources. This represents the characteristic attenuation coefficient of temperature sensing on edge devices. Indicates the temperature sensitivity coefficient. Indicates the rated operating temperature threshold of the equipment. Indicates the real-time available memory of the edge device. This represents the Hadamard product with temperature compensation for eigenvalues.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images