PCB appearance defect real-time detection algorithm based on edge calculation

Abstract

The invention discloses a PCB appearance defect real-time detection algorithm based on edge calculation, and belongs to the technical field of computer vision and intelligent manufacturing. According to the algorithm, multi-scale feature extraction is performed on a PCB surface image through a multi-modal feature extraction network, feature dimension compression is realized under the constraint of edge computing resources in combination with a lightweight convolutional neural network, and the computing efficiency and the detection precision are balanced. And dynamically adjusting the weight distribution of the local defect features and the global structure features by adopting an adaptive attention algorithm, and enhancing the significance expression of the tiny defects. By fusing a process feature mapping mechanism of a PCB process parameter database and combining a temperature sensitivity coefficient sensed by real-time environment temperature, the suitability of a detection result and a production condition is improved. And a low-delay defect classification decision is realized by using a lightweight classifier.

Description

Technical Field

[0001] The present invention belongs to the technical fields of computer vision and intelligent manufacturing, and particularly relates to a real-time detection algorithm for PCB appearance defects based on edge computing. Background Art

[0002] As a key basic component of modern electronic devices, the manufacturing quality of a PCB (Printed Circuit Board) directly affects the performance and reliability of electronic products. In the production process of a PCB, appearance defect detection is an important link to ensure product quality. Traditional detection means have some deficiencies: manual visual inspection is inefficient, subjective and prone to fatigue, making it difficult to meet the rhythm of an automated production line; automatic detection systems based on traditional image processing algorithms have limited detection accuracy when faced with complex and diverse PCB defect features, insufficient recognition ability for small and hidden defects, and are prone to false positives and false negatives. With the continuous evolution of electronic devices, the integration and complexity of PCBs are increasing day by day, and the types and forms of their appearance defects are becoming more and more complex. Against this background, edge computing technology has emerged. It pushes computing, data storage, and processing capabilities to the network edge, close to the data source and users, providing new ideas and solutions for real-time detection of PCB appearance defects. Edge computing has advantages such as low latency, high bandwidth, and local data processing, meeting the requirements of PCB real-time detection for fast response, efficient processing, and data privacy protection. Deep learning technology, especially convolutional neural networks, has achieved great success in the field of image recognition. However, conventional deep learning models often have a large number of parameters and high computational resource consumption, making it difficult to be directly deployed in resource-constrained edge computing environments. Summary of the Invention

[0003] Based on this, it is necessary to provide a real-time detection algorithm for PCB appearance defects based on edge computing that can solve the above technical problems.

[0004] In a first aspect, the present application provides a real-time detection algorithm for PCB appearance defects based on edge computing, including:

[0005] Processing PCB surface image data using a multi-modal feature extraction network to obtain a first feature set containing multi-scale features;

[0006] Based on the edge computing resource constraint conditions, using a lightweight convolutional neural network to compress the feature dimensions of the first feature set to obtain a second feature set;

[0007] Extracting local defect features and global structure features of the second feature set, and dynamically adjusting feature weights according to defect scale and type through an adaptive attention algorithm to obtain a third feature set;

[0008] Non-linearly map and associate the process features in the third feature set with a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information;

[0009] Use a lightweight classifier to perform real-time detection on the fourth feature set to generate a defect detection result.

[0010] In one embodiment, the lightweight convolutional neural network includes the following lightweight feature compression operators:

[0011]

[0012] Among them, represents the input feature tensor, represents the depthwise separable convolution kernel, represents the depthwise separable convolution operator, represents the batch normalization function, ξ represents the non-linear activation function, P represents the channel rearrangement operator, η max =R + , represents the upper bound of the Lipschitz constant.

[0013] In one embodiment, the adaptive attention algorithm includes the following formula:

[0014]

[0015] Among them, a ij represents the feature weight at position (i, j), Δs ij represents the local defect feature saliency measure, represents the gradient vector of the feature map at (i,j), ∈ = R + represents the numerical stability constant, g ij represents the fusion representation of the global structural feature and the process prior, W g represents the trainable weight matrix, represents the process database query result, φ(x)=log(1+e βx ), represents the non-linear enhancement function of the local defect feature, represents the normalization function of the global structural feature.

[0016] In one embodiment, non-linearly map and associate the process features in the third feature set with the process features included in a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information, including:

[0017] Use the following process parameter correlation function to perform process feature mapping and association to generate a fourth feature set containing process adaptation information:

[0018]

[0019] <x, y> M = (1 - γ)||X||1 + γ||X||2

[0020] where ρ(F (3) , p i ) represents the degree of association between the third feature set F (3) and the i-th process parameter p i , F (3) = R d represents the feature vector after adaptive attention, p i = R d represents the i-th parameter vector in the process parameter database, <x, y> M represents the elastic norm inner product, M = R d×d , which represents mapping the process parameters to a high-dimensional semantic space, γ represents the elastic norm mixing coefficient, ||X||1 represents the L1 norm, ||X||2 represents the L2 norm, β represents the temperature sensitivity coefficient, t represents the real-time collected PCB environmental temperature, t i represents the standard test temperature corresponding to the process parameter pi, and tanh(·) represents the normalized attenuation function of the temperature difference.

[0021] In one embodiment, a lightweight classifier is used to perform real-time detection on the fourth feature set to generate a defect detection result, including:

[0022] The following formula is used to perform real-time detection on the fourth feature set to generate a defect detection result:

[0023]

[0024] where, represents the final defect detection result, c ∈ C represents traversing all possible categories, F (4) represents the feature vector after process mapping association, represents the weight coefficient of the m-th feature corresponding to the c-th type of defect, TopK γ = R d → R γ represents the dynamic feature selection operator, γ represents the number of features selected determined by the real-time available resources, κ(T edge ) represents the feature attenuation coefficient of edge device temperature perception, k represents the temperature sensitivity coefficient, T0 represents the device rated operating temperature threshold, M free represents the real-time available memory of the edge device, and ⊙ represents the Hadamard product of feature value temperature compensation.

[0025] In one embodiment, the algorithm further includes:

[0026] Fuse the fourth feature set with the multi-modal sensing features, and use the residual connection network to jointly represent the minor defects and large-scale anomalies in the features, generating a fifth feature set containing defect classification information;

[0027] Perform multi-class defect probability prediction on the fifth feature set, calculate the confidence scores of each defect category, and compare them with the preset dynamic confidence threshold:

[0028] If the confidence score of any defect category is lower than the corresponding threshold, use the range adjustment algorithm to optimize the attention weights of the fifth feature set, generate a sixth feature set and input it into the lightweight classifier;

[0029] Otherwise, directly use the fifth feature set as the sixth feature set and input it into the lightweight classifier;

[0030] Use the lightweight classifier to perform real-time detection on the sixth feature set and output the defect detection result.

[0031] In one of the embodiments, the algorithm further includes:

[0032] Obtain misdetection and missed detection samples in real time and store them in the circular buffer;

[0033] When the number of samples in the buffer reaches the preset threshold:

[0034] Adopt the stochastic gradient descent algorithm with momentum to adjust the parameters of the cross-modal fusion layer in the multi-modal feature extraction network, and use the FocalLoss function to adjust the parameters of the output layer of the classifier.

[0035] In a second aspect, the present application also provides a real-time PCB appearance defect detection device based on edge computing, including:

[0036] A multi-modal feature extraction module, which is used to process the PCB surface image data by using a multi-modal feature extraction network to obtain a first feature set containing multi-scale features;

[0037] A feature dimension compression module, which is used to compress the feature dimension of the first feature set by using a lightweight convolutional neural network based on the edge computing resource constraint conditions to obtain a second feature set;

[0038] An adaptive attention calculation module, which is used to extract the local defect features and global structure features of the second feature set, and dynamically adjust the feature weights according to the defect scale and type through the adaptive attention algorithm to obtain a third feature set;

[0039] A process feature mapping module, which is used to perform non-linear mapping association between the process features of the third feature set and the process features included in the preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information;

[0040] A real-time classification decision module for using a lightweight classifier to perform real-time detection on the fourth feature set and generate a defect detection result.

[0041] In a third aspect, the present application also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned real-time PCB appearance defect detection algorithm based on edge computing are implemented.

[0042] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned real-time PCB appearance defect detection algorithm based on edge computing are implemented.

[0043] The above-mentioned real-time PCB appearance defect detection algorithm, device, computer device and storage medium based on edge computing realize multi-scale feature fusion of PCB surface images through a multi-modal feature extraction network, and combine the feature dimension compression technology of a lightweight convolutional neural network to reduce the model parameter quantity and computational complexity while ensuring the detection accuracy, effectively solving the resource bottleneck problem of traditional deep learning models when deployed on edge devices. The adaptive attention algorithm is used to dynamically balance the weight distribution between local defect features and global structure features, and the gradient feature expression of tiny defects is strengthened through a non-linear enhancement function, overcoming the defect of insufficient sensitivity of conventional algorithms to subtle defects. Through the elastic norm correlation mechanism of the process parameter database and temperature sensitivity coefficient compensation, the production environment parameters are dynamically incorporated into the feature space, improving the robustness of the detection result to complex industrial scenarios. Using a lightweight classifier combined with a dynamic feature selection operator, the calculation path is adaptively adjusted according to the real-time resource status and the temperature of the edge device, achieving the optimal balance between detection accuracy and computational efficiency while maintaining low-latency decision-making, and comprehensively solving the technical defects of traditional detection methods in terms of real-time performance, accuracy and environmental adaptability. Description of the Drawings

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0045] Figure 1 It is a flowchart of a real-time PCB appearance defect detection algorithm based on edge computing of the present invention;

[0046] Figure 2 It is a structural diagram of a real-time PCB appearance defect detection device based on edge computing of the present invention. Detailed Embodiments

[0047] In order to make the objectives, technical solutions and advantages of this application clearer and more understandable, the following further details this application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0048] The hardware architecture of a real-time PCB appearance defect detection algorithm based on edge computing in this application includes an industrial camera and a sensor, a computing terminal, and a process database. The industrial camera and the sensor provide multi-source data input, the computing terminal performs efficient feature compression and dynamic decision-making, and is optimized in the long term through the process database. The hardware realizes closed-loop control at the level through a low-latency communication protocol.

[0049] In one embodiment, as Figure 1 shown, a real-time PCB appearance defect detection algorithm based on edge computing is provided. In this embodiment, the application of this algorithm to a terminal is taken as an example for illustration. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is realized through the interaction between the terminal and the server. In this embodiment, the algorithm includes the following steps:

[0050] S101, Process the PCB surface image data by using a multi-modal feature extraction network to obtain a first feature set containing multi-scale features.

[0051] Among them, the multi-modal feature extraction network is a neural network architecture that can fuse and extract features from multiple different types of data, that is, multi-modal data features, and can simultaneously process and analyze data from different aspects of the PCB (printed circuit board) surface image, such as image features at different resolutions, spectral features in different bands, etc. The PCB surface image data refers to the image of the PBC surface obtained through an industrial camera or other imaging devices, and contains various information on the PCB surface, such as the shape of the circuit, the quality of the solder joints, the position of the holes, etc. After data processing, a first feature set containing multi-scale PCB surface features is obtained.

[0052] S102, Based on the edge computing resource constraint conditions, use a lightweight convolutional neural network to compress the feature dimension of the first feature set to obtain a second feature set.

[0053] Among them, the edge computing resource constraint condition means that edge devices such as embedded systems and industrial controllers usually have limited computing power, memory, and energy supply. The algorithms running on these devices need to meet the resource constraint conditions and minimize the computational load and memory occupancy. By using a convolutional neural network that has been optimized for efficient operation in resource-constrained environments, the first feature set is compressed. This can reduce the dimension of the features while retaining important feature information, reducing the computational burden of subsequent processing while ensuring the feature expression ability, obtaining a second feature set with a lower dimension, which is more suitable for further processing and analysis in the edge computing environment.

[0054] S103, extract the local defect features and global structure features of the second feature set, and dynamically adjust the feature weights according to the defect scale and type through an adaptive attention algorithm to obtain a third feature set.

[0055] Among them, local defect features: small-range areas in the PCB surface image, aiming to capture features of local defects such as scratches, holes, uneven solder joints, etc., for identifying defects at specific positions; global structure features: large-range structures and layouts of the PCB surface image, such as circuit traces, component distributions, etc. global features, which can help understand the overall structure and layout of the PCB. Through the adaptive attention algorithm, different weights can be automatically assigned to local defect features and global structure features according to the defect scale and type. For the defect scale: for tiny local defects, the algorithm gives higher weights to local defect features, and for larger defects or structural problems, higher weights are given to global structure features; for the defect type: different types of defects may be more obvious in local or global features. For example, uneven solder joints are mainly reflected in local features, and circuit layout errors are more reflected in global features. The feature weights are automatically adjusted according to the defect type to highlight the features that best represent the essence of the defects. The third feature set obtained after adjustment can make the feature expression more in line with the requirements of actual defect detection.

[0056] S104, perform a non-linear mapping association between the process features of the third feature set and the process features included in the preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information.

[0057] Among them, the preset PCB manufacturing process feature library stores a large amount of data on known PCB manufacturing process parameters and their corresponding appearance features, which is representative and guiding, and can be used as a standard reference to help detect whether the current PCB process meets the requirements. Since the relationship between the process features and appearance features of a PCB is often not a simple linear relationship, but a complex non-linear association, the use of non-linear mapping methods can more accurately describe and associate this complex relationship. Non-linear patterns in the data can be automatically captured using learning algorithms through neural networks, support vector machines, etc. The process feature vectors in the third feature set are input into the non-linear mapping model, and the model finds the process parameter combination that is most similar or most associated with the current process feature according to the standard process parameters and their corresponding appearance features in the preset PCB manufacturing process feature library, and determines the position and similarity of the process features of the current PCB in the database. A fourth feature set containing process adaptation information is generated, which can reflect the degree of association and differences with the standard process features.

[0058] S105, use a lightweight classifier to perform real-time detection on the fourth feature set to generate a defect detection result.

[0059] Among them, the lightweight classifier is a classification algorithm designed to operate efficiently in resource-constrained environments. When performing real-time detection on the fourth feature set, it quickly generates a defect detection result according to the pre-trained model parameters and classification rules, including judgments on whether there are defects, types of defects such as scratches, holes, uneven solder joints, etc., location information of the defects such as coordinates or regions in the PCB image, and severity levels of the defects such as minor, medium, severe, etc.

[0060] A real-time detection algorithm for PCB appearance defects based on edge computing provided by the present invention processes PCB surface image data through a multi-modal feature extraction network, fusing various data features; under the constraints of edge computing resources, a lightweight convolutional neural network is used to compress the dimension of the first feature set to obtain a second feature set, adapting to the limited resources of edge devices; local defect features and global structure features in the second feature set are extracted, and an adaptive attention algorithm is used to dynamically adjust the feature weights according to the defect scale and type to obtain a third feature set, enhancing the pertinence of feature expression; the process features in the third feature set are non-linearly mapped and associated with a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information, realizing the dynamic integration of process parameters; a lightweight classifier is used to perform real-time detection on the fourth feature set to generate a detection result containing information such as defect judgment, type, location, and severity, meeting the real-time requirements of the production line. It integrates technologies such as multi-modal data fusion, lightweight model design, adaptive feature adjustment, and process mapping association, effectively solving problems such as low efficiency, strong subjectivity, insufficient recognition ability for tiny and hidden defects, difficulty in adapting to complex and diverse defect features, and difficult deployment of deep learning models in edge computing environments in traditional PCB appearance defect detection methods, improving the accuracy, efficiency, and adaptability of PCB appearance defect detection, reducing the deployment cost, realizing real-time monitoring and optimization of the production process, and promoting the intelligent development of the PCB manufacturing industry.

[0061] In one embodiment, the lightweight convolutional neural network includes the following lightweight feature compression operators:

[0062]

[0063]

[0064] Among them, represents the input feature tensor, represents the depthwise separable convolution kernel, represents the depthwise separable convolution operator, represents the batch normalization function, ξ represents the non-linear activation function, P represents the channel rearrangement operator, η max =R + represents the upper bound of the Lipschitz constant.

[0065] Exemplarily, the input feature tensor F (1)It is usually a multi-dimensional array that contains the feature information extracted from the PCB surface image. It can be from a multi-modal feature extraction network, with the characteristics of multi-scale and multi-channel. The batch normalization function BN(x) is used to normalize the feature map. By normalizing each feature value to zero mean and unit variance, it accelerates the training process of the network and improves the stability and generalization ability of the model. The non-linear activation function ξ is used to introduce non-linearity, enabling the network to learn complex feature representations, setting negative-valued features to zero, retaining positive-valued features, and enhancing the expressive power and robustness of the model. The channel rearrangement operator P is used to rearrange the feature channels and optimize the feature representation. By rearranging the channels, the correlation between features can be better utilized, improving the efficiency and performance of the model. The upper bound η of the Lipschitz constant max is used to control the Lipschitz continuity of the network. Lipschitz continuity ensures that the change in the network output does not exceed a certain multiple of the input change, thereby improving the stability and robustness of the model. By using lightweight feature compression operators such as depthwise separable convolution kernels, batch normalization functions, non-linear activation functions, and channel rearrangement operators, the computational amount and memory occupancy can be significantly reduced while ensuring the feature expression ability. The combination of these operators not only improves the efficiency of the model but also ensures the real-time performance and stability of the model on resource-constrained edge devices. By controlling the upper bound of the Lipschitz constant, the robustness and noise resistance of the model are further enhanced, enabling the lightweight convolutional neural network to operate efficiently and accurately in PCB appearance defect detection.

[0066] In one of the embodiments, the adaptive attention algorithm includes the following formula:

[0067]

[0068] where, a ij represents the feature weight at position (i, j), Δs ij represents the local defect feature saliency measure, represents the gradient vector of the feature map at (i, j), ∈ = R + represents the numerical stability constant, g ij represents the fused representation of the global structural feature and the process prior, W g represents the trainable weight matrix, represents the process database query result, φ(x) = log(1 + e βx ) represents the non-linear enhancement function of the local defect feature, represents the normalization function of the global structural feature.

[0069] Specifically, through the adaptive attention mechanism, the feature weight a at each position (i, j) is dynamically calculated ij, for subsequent feature fusion, the weight value reflects the importance of this position in defect detection. Local defect feature saliency measure Δs ij Through the gradient vector Capture the edge or texture changes in the local area. For large-scale defects such as large-area stains: the gradient distribution range is wide, and the numerator of Δs ij is generally high as a whole, but the denominator will also increase, and finally Δs ij may tend to be flat. At this time, the model relies on the global process knowledge of ψ(g ij ) to suppress misjudgment. For small-scale defects such as fine scratches: the gradient is concentrated locally, and the numerator of Δs ij is significantly higher than the surrounding area. After the denominator is normalized by the global expectation, Δs ij still remains high. At this time, φ(Δs ij ) increases the weight through non-linear enhancement such as β amplification. The fusion representation g ij of the global structural feature and the process prior Fuses the gradient vector DB with the query result P DB (i, j) of the process database and suppresses outliers through the tanh activation function. For process-related defects, such as standard solder joint deviation, P ij (i, j) contains the prior features of similar defects. After fusion, the value of g ij is high, and the output of ψ(g DB ) approaches 1, enhancing the weight of this type of defect. For non-process defects such as random stains P DB (i, j), P ij (i, j) has no associated features, the value of g ij is low, and the output of ψ(g βx ) approaches 0, suppressing the weight of this type of defect. The non-linear enhancement function φ(x) = log(1 + e The normalization function representing the global structural feature can compress the fused feature g ij to the interval (-1, 1) to balance the contributions of local and global features. Distinguish defect scales according to the gradient intensity. This formula uses Δs ij to distinguish defect scales according to the gradient intensity; uses φ(x) and β to adjust the sensitivity of local features. Uses ψ(x) and P DB (i, j) to distinguish defect types in combination with process knowledge, enabling the model to efficiently process real-time data in edge computing scenarios and taking into account the complex characteristics of different defects at the same time.

[0070] In one embodiment, the process features of the third feature set are non-linearly mapped and associated with the process features included in the preset PCB manufacturing process feature library to generate a fourth feature set including process adaptation information, including:

[0071] The process features are associated using the following process parameter correlation function to generate a fourth feature set including process adaptation information:

[0072]

[0073] <x, y> M =(1 - γ)||X||1 + γ||X||2

[0074] where ρ(F (3) , p i ) represents the correlation degree between the third feature set F (3) and the i-th process parameter p i , F (3) =R d represents the feature vector after adaptive attention, p i =R d represents the i-th parameter vector in the process parameter database, <x, y> M represents the elastic norm inner product, M = R d×d , represents mapping the process parameters to a high-dimensional semantic space, γ represents the elastic norm mixing coefficient, ||X||1 represents the L1 norm, ||X||2 represents the L2 norm, β represents the temperature-sensitive coefficient, t represents the real-time collected PCB environmental temperature, t i represents the standard test temperature corresponding to the process parameter pi, and tanh(·) represents the normalization attenuation function of the temperature difference.

[0075] Exemplarily, the elastic norm inner product <x, y> M maps the feature vector X to a high-dimensional semantic space M, calculates the similarity by mixing the L1 and L2 norms, balances the sparsity and smoothness of the features, and converts the original features into more discriminative representations. For example, features such as PCB surface texture and color are mapped to a defect-sensitive semantic space. The normalization attenuation function of the temperature difference tanh(·) dynamically adjusts the correlation degree according to the difference between the real-time temperature t and the standard test temperature t i . The greater the temperature difference, the more significant the attenuation of the correlation degree. During the PCB manufacturing process, temperature changes will affect material properties such as solder fluidity and coating adhesion. Through the normalization attenuation function of the temperature difference, the model can automatically reduce the correlation confidence under non-standard temperatures and reduce false detections. Using to normalize the elastic norm inner product to ensure that the value range of the matching correlation degree is between (-1, 1), combining the feature similarity and temperature factors, the final correlation degree ρ(F(3) , p i ).

[0076] In one embodiment, a lightweight classifier is used to perform real-time detection on the fourth feature set to generate a defect detection result, including:

[0077] The following formula is used to perform real-time detection on the fourth feature set to generate a defect detection result:

[0078]

[0079] where, represents the final defect detection result, c ∈ C represents traversing all possible categories, F (4) represents the feature vector after process association, represents the weight coefficient of the m-th feature corresponding to the c-th type of defect, TopK γ = R d → R γ represents the dynamic feature selection operator, γ represents the number of features selected determined by the real-time available resources, κ(T edge ) represents the feature attenuation coefficient of the edge device temperature perception, k represents the temperature sensitivity coefficient, T0 represents the device rated operating temperature threshold, M free represents the real-time available memory of the edge device, and ⊙ represents the Hadamard product of the eigenvalue temperature compensation.

[0080] Exemplarily, the dynamic feature selection operator TopK γ dynamically selects the most important γ features according to the real-time available memory M free to reduce the computational complexity and improve the real-time performance. The feature attenuation coefficient κ(T edge ) attenuates the feature importance according to the real-time temperature, and suppresses the noise or misdetection caused by temperature changes. For example, in the PCB manufacturing process, the temperature fluctuation causes the change of sensor sensitivity or the deviation of material characteristics. Using this formula can reduce redundant calculations, adapt to memory fluctuations, suppress temperature interference, improve the robustness of defect classification, and trace the contribution of key features to classification using weights .

[0081] In one embodiment, the algorithm further includes:

[0082] Fusing the fourth feature set with the multi-modal sensing features, and using the residual connection network to jointly represent the minute defects and large-scale anomalies in the features to generate a fifth feature set including defect classification information;

[0083] Performing multi-class defect probability prediction on the fifth feature set, calculating the confidence scores of each defect category, and comparing them with the preset dynamic confidence threshold:

[0084] If the confidence score of any defect category is lower than the corresponding threshold, the attention weights of the fifth feature set are optimized using a range adjustment algorithm to generate a sixth feature set and input it into the lightweight classifier;

[0085] Otherwise, directly use the fifth feature set as the sixth feature set and input it into the lightweight classifier;

[0086] Use the lightweight classifier to perform real-time detection on the sixth feature set and output the defect detection result.

[0087] Specifically, combine the fourth feature set with multi-modal sensing features such as non-visual data like temperature and vibration, and use a residual connection network for joint representation. By retaining the original features and introducing auxiliary information, enhance the model's collaborative capture ability for micro-defects and large-scale anomalies, and avoid information loss in the deep network. Classify the fused fifth feature set and output the probability scores of each defect category such as scratches and holes. Adjust the threshold according to real-time production conditions such as temperature fluctuations and equipment status. For example, in a high-temperature environment, the threshold can be relaxed to reduce the possibility of missed detections and ensure the reliability of the classification results. If the confidence of any category is lower than the threshold, trigger the range adjustment algorithm to dynamically optimize the attention weights of the fifth feature set. For example, enhance the weights in high-gradient regions and suppress the weights of background noise or irrelevant features. The optimized features are re-input into the lightweight classifier to improve the classification accuracy. If the confidence of all categories meets the standard, directly adopt the output result of the fifth feature set to avoid redundant calculations and ensure real-time performance.

[0088] In one of the embodiments, the algorithm further includes:

[0089] Obtain misdetection and missed detection samples in real time and store them in a circular buffer;

[0090] When the number of samples in the buffer reaches the preset threshold:

[0091] Use the stochastic gradient descent algorithm with momentum to adjust the parameters of the cross-modal fusion layer in the multi-modal feature extraction network, and use the FocalLoss function to adjust the parameters of the output layer of the classifier.

[0092] Exemplarily, it can be to monitor the classification results in real time, store the misdetected samples such as normal samples being determined as defective and the undetected samples such as defective samples being determined as normal, along with their true labels, into a circular buffer. An FIFO (First In First Out) queue can be adopted to ensure controllable memory occupancy while retaining typical error samples that have occurred recently. When the number of samples in the buffer reaches a preset threshold, the model parameter update process is initiated, which can avoid the interference of single-sample noise and improve the stability and convergence speed of parameter update. The stochastic gradient descent algorithm with momentum is adopted to introduce historical gradient information, accelerate parameter convergence and suppress oscillations, and efficiently handle non-convex optimization problems. Based on the gradient of the loss function between the true labels and the model prediction results of the misdetected / undetected samples in the buffer, the parameters of the cross-modal fusion layer in the multi-modal feature extraction network are adjusted by backpropagation. The Focal Loss function is used to reduce the loss contribution of samples with high probability predictions, such as samples that are clearly defect-free, to avoid the model overfitting to simple samples; for samples that are difficult to distinguish, the loss weight is amplified to force the model to focus on error-prone samples. Through the dynamic capture and batch optimization of misdetected / undetected samples, combined with multi-modal feature fusion and the design of a robust loss function, the accuracy and stability of PCB defect detection are significantly improved.

[0093] A real-time PCB appearance defect detection algorithm based on edge computing in this application realizes the detection of PCB appearance defects in the edge computing environment by integrating the multi-scale analysis ability of the multi-modal feature extraction network and the resource constraint adaptability of the lightweight convolutional neural network. Through the adaptive attention mechanism, the weight distribution between local defect features and global structure features is dynamically optimized, and combined with the non-linear mapping correlation technology of the process parameter database, the saliency enhancement of micro-defects and the robust representation of complex industrial scenarios are realized. Further, through the residual connection network to fuse multi-modal sensing features, combined with the dynamic confidence threshold determination and the closed-loop feedback optimization mechanism, the adaptability under complex working conditions such as temperature fluctuations and equipment state changes is improved. On the premise of limited computing resources and ensuring real-time performance, the accuracy and generalization ability of defect classification are significantly improved, providing a low-latency and highly reliable visual detection solution for high-density PCB intelligent manufacturing.

[0094] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0095] Based on the same inventive concept, an embodiment of the present application also provides a computing device for implementing the real-time detection algorithm of PCB appearance defects based on edge computing described above. The implementation solution provided by this device to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more embodiments of the computing device for the real-time detection algorithm of PCB appearance defects based on edge computing provided below can refer to the limitations on the real-time detection algorithm of PCB appearance defects based on edge computing in the above text, and will not be repeated here.

[0096] In an exemplary embodiment, as Figure 2 shown, a computing device for the real-time detection algorithm of PCB appearance defects based on edge computing is provided, including:

[0097] A multi-modal feature extraction module 11, configured to process PCB surface image data by using a multi-modal feature extraction network to obtain a first feature set including multi-scale features;

[0098] A feature dimension compression module 12, configured to compress the feature dimension of the first feature set by using a lightweight convolutional neural network based on the edge computing resource constraint conditions to obtain a second feature set;

[0099] An adaptive attention calculation module 13, configured to extract local defect features and global structure features of the second feature set, and dynamically adjust the feature weights according to the defect scale and type through an adaptive attention algorithm to obtain a third feature set;

[0100] A process feature association module 14, configured to perform non-linear mapping association between the process features in the third feature set and a preset PCB manufacturing process feature library to generate a fourth feature set including process adaptation information;

[0101] A real-time classification and decision-making module 15, configured to perform real-time detection on the fourth feature set by using a lightweight classifier to generate a defect detection result.

[0102] In one of the embodiments, the lightweight convolutional neural network in the feature dimension compression module 12 further includes the following lightweight feature compression operators:

[0103]

[0104]

[0105] Among them, represents the input feature tensor, represents the depthwise separable convolution kernel, represents the depthwise separable convolution operator, represents the batch normalization function, ξ represents the non-linear activation function, P represents the channel rearrangement operator, η max =R + , represents the upper bound of the Lipschitz constant.

[0106] In one of the embodiments, the adaptive attention calculation module 13 further includes the following formula:

[0107]

[0108] Among them, a ij represents the feature weight at position (i, j), Δs ij represents the local defect feature saliency measure, represents the gradient vector of the feature map at (i, j), ∈ = R + represents the numerical stability constant, g ij represents the fusion representation of the global structural feature and the process prior, W g represents the trainable weight matrix, represents the process database query result, φ(x) = log(1 + e βx ), represents the non-linear enhancement function of the local defect feature, represents the normalization function of the global structural feature.

[0109] In one of the embodiments, the process feature association module is further used for:

[0110] Using the following process parameter association degree function to perform process feature association and generate a fourth feature set containing process adaptation information:

[0111]

[0112] <x, y> M =(1 - γ)||X||1 + γ||X||2

[0113] Among them, ρ(F (3) , p i ) represents the third feature set F(3) The correlation degree with the i-th process parameter p i , F (3) = R d represents the feature vector after adaptive attention, p i = R d represents the i-th parameter vector in the process parameter database, <x, y> M represents the elastic norm inner product, M = R d×d , represents mapping the process parameters to a high-dimensional semantic space, γ represents the elastic norm mixing coefficient, ||X||1 represents the L1 norm, ||X||2 represents the L2 norm, β represents the temperature sensitivity coefficient, t represents the real-time collected PCB environmental temperature, t i represents the standard test temperature corresponding to the process parameter pi, tanh(·) represents the normalized attenuation function of temperature difference.

[0114] In one embodiment, the real-time classification decision module 15 is further configured to:

[0115] Use the following formula to perform real-time detection on the fourth feature set and generate a defect detection result:

[0116]

[0117] Wherein, represents the final defect detection result, c ∈ C represents traversing all possible categories, F (4) represents the feature vector after process association, represents the weight coefficient of the m-th feature corresponding to the c-th type of defect, TopK γ = R d → R γ represents the dynamic feature selection operator, γ represents the number of features selected determined by the real-time available resources, κ(T edge ) represents the feature attenuation coefficient of edge device temperature perception, k represents the temperature sensitivity coefficient, T0 represents the device rated operating temperature threshold, M free represents the real-time available memory of the edge device, ⊙ represents the Hadamard product of eigenvalue temperature compensation.

[0118] In one embodiment, the device is further configured to:

[0119] Fuse the fourth feature set with the multi-modal sensing features, and use the residual connection network to jointly represent the micro defects and large-scale anomalies in the features to generate a fifth feature set containing defect classification information;

[0120] Perform multi-class defect probability prediction on the fifth feature set, calculate the confidence scores of each defect class, and compare them with the preset dynamic confidence threshold:

[0121] If the confidence score of any defect category is lower than the corresponding threshold, use the range adjustment algorithm to optimize the attention weights of the fifth feature set, generate the sixth feature set and input it into the lightweight classifier;

[0122] Otherwise, directly use the fifth feature set as the sixth feature set and input it into the lightweight classifier;

[0123] Use the lightweight classifier to perform real-time detection on the sixth feature set and output the defect detection result.

[0124] In one embodiment, the device is also used for:

[0125] Obtain misdetection and missed detection samples in real time and store them in the circular buffer;

[0126] When the number of samples in the buffer reaches the preset threshold:

[0127] Adopt the stochastic gradient descent algorithm with momentum to adjust the parameters of the cross-modal fusion layer in the multi-modal feature extraction network, and use the FocalLoss function to adjust the parameters of the output layer of the classifier.

[0128] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of a real-time PCB appearance defect detection algorithm based on edge computing as described above.

[0129] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps in the above algorithm embodiments.

[0130] For the device embodiments, since they basically correspond to the algorithm embodiments, the relevant parts can be referred to the partial descriptions of the algorithm embodiments. The device embodiments described above are only illustrative. The components described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0131] The above embodiments only represent several implementation manners of the embodiments of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the embodiments of the application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the embodiments of the present application, several deformations and improvements can be made, and these all belong to the protection scope of the embodiments of the present application.

Claims

1. A real-time detection algorithm for PCB appearance defects based on edge computing, characterized in that, The algorithm includes: Processing the PCB surface image data by using a multi-modal feature extraction network to obtain a first feature set containing multi-scale features; Based on the edge computing resource constraint conditions, using a lightweight convolutional neural network to compress the feature dimensions of the first feature set to obtain a second feature set; Extracting the local defect features and global structure features of the second feature set, and dynamically adjusting the feature weights according to the defect scale and type through an adaptive attention algorithm to obtain a third feature set; Performing a non-linear mapping association between the process features of the third feature set and the process features included in a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information; Using a lightweight classifier to perform real-time detection on the fourth feature set to generate a defect detection result.

2. The algorithm according to claim 1, characterized in that, The lightweight convolutional neural network includes the following lightweight feature compression operators: Among them, represents the input feature tensor, represents the depthwise separable convolution kernel, represents the depthwise separable convolution operator, represents the batch normalization function, ξ represents the non-linear activation function, P represents the channel rearrangement operator, η max = R + , representing the upper bound of the Lipschitz constant.

3. The algorithm according to claim 1, characterized in that, The adaptive attention algorithm includes the following formula: Among them, a ij represents the feature weight at position (i, j), and Δs ij represents the significance measure of local defect features, represents the gradient vector of the feature map at (i, j), ∈ = R + represents the numerical stability constant, g ij represents the fusion representation of global structural features and process priors, W g represents the trainable weight matrix, represents the process database query result, φ(x) = log(1 + e βx ), represents the non - linear enhancement function of local defect features, represents the normalization function of global structural features.

4. The algorithm according to claim 1, wherein Performing a non-linear mapping association between the process features of the third feature set and the process features included in a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information, including: Using the following process parameter correlation function to perform non-linear mapping association of process features to generate a fourth feature set containing process adaptation information: <x, y> M = (1 - γ)||X||1 + γ||X||2 Among them, ρ(F (3) , p i ) represents the correlation degree between the third feature set F (3) and the i-th process parameter p i . F (3) = R d represents the feature vector after adaptive attention. p i = R d represents the i-th parameter vector in the process parameter database. <x, y> M represents the elastic norm inner product. M = R d×d , which means mapping the process parameters to a high-dimensional semantic space. γ represents the elastic norm mixing coefficient. ||X||1 represents the L1 norm. ||X||2 represents the L2 norm. β represents the temperature sensitivity coefficient. t represents the PCB ambient temperature collected in real time. t i represents the standard test temperature corresponding to the process parameter pi. tanh(·) represents the normalized attenuation function of temperature difference.

5. The algorithm according to claim 1, characterized in that, The using a lightweight classifier to perform real-time detection on the fourth feature set to generate a defect detection result includes: Using the following formula to perform real-time detection on the fourth feature set to generate a defect detection result: Among them, represents the final defect detection result, c ∈ C represents traversing all possible categories, and F (4) represents the feature vector after process association. represents the weight coefficient of the m-th feature corresponding to the c-th type of defect, and TopK γ = R d → R γ represents the dynamic feature selection operator, γ represents the number of feature selections determined by the real-time available resources, κ(T edge ) represents the feature attenuation coefficient of the edge device temperature perception, k represents the temperature sensitivity coefficient, T0 represents the rated operating temperature threshold of the device, and M free represents the real-time available memory of the edge device, and ⊙ represents the Hadamard product of the eigenvalue temperature compensation.

6. The algorithm according to claim 1, characterized in that The algorithm further includes: Fusing the fourth feature set with multi-modal sensing features, and using a residual connection network to jointly represent the micro-defects and large-scale anomalies in the features to generate a fifth feature set containing defect classification information; Performing multi-class defect probability prediction on the fifth feature set, calculating the confidence scores of each defect class, and comparing them with a preset dynamic confidence threshold: If the confidence score of any defect class is lower than the corresponding threshold, using a range adjustment algorithm to optimize the attention weights of the fifth feature set to generate a sixth feature set and inputting it into the lightweight classifier; Otherwise, directly using the fifth feature set as the sixth feature set and inputting it into the lightweight classifier; Using the lightweight classifier to perform real-time detection on the sixth feature set and outputting a defect detection result.

7. The algorithm according to claim 6, wherein The algorithm further includes: Real-time obtaining mis-detection and missed-detection samples and storing them in a circular buffer; When the number of samples in the buffer reaches a preset threshold: Adopting a stochastic gradient descent algorithm with momentum to adjust the parameters of the cross-modal fusion layer in the multi-modal feature extraction network, and using the FocalLoss function to adjust the parameters of the output layer of the classifier.

8. A real-time detection and calculation device for PCB appearance defects based on edge computing, characterized in that, The device includes: A multi-modal feature extraction module, configured to process the PCB surface image data by using a multi-modal feature extraction network to obtain a first feature set containing multi-scale features; A feature dimension compression module, configured to compress the feature dimensions of the first feature set by using a lightweight convolutional neural network based on the edge computing resource constraint conditions to obtain a second feature set; An adaptive attention calculation module, which is used to extract the local defect features and global structure features of the second feature set, and dynamically adjust the feature weights according to the defect scale and type through an adaptive attention algorithm to obtain a third feature set; A process feature association module, which is used to perform non-linear mapping association between the process features of the third feature set and the process features included in a preset PCB manufacturing process feature library to generate a fourth feature set containing process adaptation information; A real-time classification and decision-making module, which is used to perform real-time detection on the fourth feature set by using a lightweight classifier to generate a defect detection result.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the algorithm described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the algorithm described in any one of claims 1 to 7 are implemented.

Images