PCB (Printed Circuit Board) defect detection method and equipment for edge MCU (Microprogrammed Control Unit) and storage medium

Abstract

The invention discloses a PCB (Printed Circuit Board) defect detection method and equipment for an edge MCU (Microprogrammed Control Unit) and a storage medium, and belongs to the technical field of PCB defect detection. The method comprises the following steps: acquiring a PCB image; carrying out down-sampling optimization on the YOLOv10 model by adopting deformable convolution and separation characteristics respectively; and the optimized YOLOv10 model is adopted to detect the defects of the PCB. According to the method, the requirements of the PCB defect detection model for storage and computing resources can be remarkably reduced, high reasoning speed and accuracy can be kept, the problem of real-time data processing in a weak network environment is solved, and optimization of the response speed and safety of a whole industrial internet system is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of PCB defect detection, and more specifically to a PCB board defect detection method, device and storage medium for edge MCU. Background Art

[0002] Printed circuit boards (PCBs) play a key role in many fields, including medical, military and consumer electronics, and their quality directly affects the performance and reliability of electronic devices. Traditional PCB defect detection methods, such as manual inspection and electrical testing, are not only costly and inefficient, but also fail to meet the needs of modern industrialization. For this reason, automatic optical inspection (AOI) technology combined with machine vision has been widely used in industrial production lines, especially using high-resolution optical sensors and advanced image processing models (including traditional machine learning and deep learning methods) to identify PCB surface defects.

[0003] Although deep learning methods have made remarkable achievements in the field of defect detection, the diverse shapes and low-resolution features of PCBs make end-to-end models based on deep learning face the problems of false detection and missed detection. In addition, with the development of cloud computing and Internet of Things (IoT) technology, although cloud platforms provide powerful data processing capabilities, the large bandwidth requirements and privacy leakage risks of data transmission remain challenges. Edge computing, as an emerging trend, can reduce latency, improve real-time performance, and better protect privacy by processing data locally.

[0004] Therefore, how to provide a PCB board defect detection method, device and storage medium for edge MCU that can solve the above problems is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a PCB board defect detection method, device and storage medium for edge MCU, which effectively solve the problems of missed detection and false detection in PCB defect detection.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A PCB board defect detection method for edge MCU, comprising:

[0008] Get the PCB board image;

[0009] The YOLOv10 model is optimized using deformable convolution and separable feature downsampling respectively;

[0010] The optimized YOLOv10 model is used to detect defects in PCB boards.

[0011] Optionally, optimize the YOLOv10 model using deformable convolutions including:

[0012] Construct the CSPLayer_2Conv convolution module in YOLOv10, and integrate the deformable convolution third-generation DCNv3 into the CSPLayer_2Conv convolution module to obtain the CSPLayer_2DCNv3 convolution module.

[0013] Optionally, the calculation formula of the deformable convolution third generation DCNv3 is as follows:

[0014]

[0015] Where G is the total number of aggregation groups. For the gth group, w g ∈R C×C′ represents the position-independent projection weight of the g-th group, where C′=C / G represents the group dimension; m gk ∈R represents the modulation scalar of the kth sampling point in the gth group, which is normalized by the softmax function along dimension K; x g ∈R C′×H×W represents the input feature map after slicing; Δp gk is the grid sampling position p corresponding to the gth group k , p0 represents the reference position on the input feature map.

[0016] Optionally, optimizing the YOLOv10 model using split feature downsampling includes:

[0017] Constructing a separation feature map downsampling convolution SPDConv module, wherein the separation feature map downsampling convolution SPDConv module includes: a space-to-depth SPD layer and a non-strided convolution layer;

[0018] The separated feature map downsampling convolution SPDConv module downsamples the CSPLayer_2DCNv3 output feature map of the backbone network in YOLOv10 through the original image conversion method.

[0019] Optionally, optimizing the YOLOv10 model also includes:

[0020] Remove the redundant downsampling convolutional layers in the YOLOv10 model backbone network.

[0021] Optionally, optimizing the YOLOv10 model also includes:

[0022] Convert the YOLOv10 model data format from 32-bit floating point to 8-bit integer.

[0023] Optionally, an optimized YOLOv10 model is used to detect PCB board defects, including:

[0024] The optimized YOLOv10 model is used as the PCB defect detection model. After quantization, the Tflite binary file is generated. The Tflite binary file is converted into an optimized C language PCB defect detection model through the STM32Cube.AI expansion package X-CUBE-AI-8.0.1, and deployed to the GD32H757MCU to detect PCB board defects.

[0025] A PCB board defect detection device for edge MCU includes: a memory, a processor, and a PCB board defect detection program stored in the memory and executable on the processor, wherein the PCB board defect detection program is configured to implement a PCB board defect detection method for edge MCU.

[0026] A storage medium stores a PCB board defect detection program, and when the PCB board defect detection program is executed by a processor, a PCB board defect detection method for an edge MCU is implemented.

[0027] It can be seen from the above technical solutions that compared with the prior art, the present invention discloses a PCB board defect detection method, device and storage medium for edge MCU. First, the PCB defect characteristics are optimized and the YOLOv10 model is optimized to design a high-precision PCB defect detection model. Secondly, the PCB defect detection model is optimized using model feature map visualization technology to adapt to the hardware limitations of the microcontroller unit MCU. The present invention not only significantly reduces the storage and computing resource requirements of the PCB defect detection model, but also maintains high reasoning speed and accuracy, solves the problem of real-time data processing in a weak network environment, and optimizes the response speed and security of the entire industrial Internet system. The specific beneficial effects are:

[0028] 1) Aiming at the business characteristics of PCB defect detection, such as high target complexity, diverse shapes, low resolution and similar background, the present invention proposes an optimization method, which combines deformable convolution technology and separation feature map downsampling method to effectively alleviate the problems of missed detection and false detection of target detection caused by these characteristics, thereby significantly improving the detection accuracy.

[0029] 2) In order to achieve the best balance between model parameters and detection accuracy, the present invention uses feature map visualization technology to simplify the backbone network of the model. This method effectively solves the problems faced by AI models when deployed on MCUs, such as limited flash memory capacity, limited computing resources, and slow model reasoning speed, ensuring that efficient reasoning performance and accuracy can be maintained in a resource-constrained hardware environment.

[0030] In summary, the present invention provides a PCB board defect detection method, device and storage medium for edge MCU. These improvements provide key technical support for realizing real-time and high-precision PCB defect detection in industrial applications, and are expected to be widely used in the fields of industrial automation and intelligent manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0032] Figure 1 A schematic diagram of the structure of the CSPLayer_2DCNv3 module provided in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of the structure of the SPDConv module provided in an embodiment of the present invention;

[0034] Figure 3 A visualization diagram of a backbone network feature graph provided by an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of the overall hardware framework of the system provided by an embodiment of the present invention;

[0036] Figure 5 A schematic flow chart of a PCB board defect detection method for edge MCU provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] See also Figure 5 As shown, an embodiment of the present invention discloses a PCB board defect detection method for edge MCU, including:

[0039] Get the PCB board image;

[0040] The YOLOv10 model is optimized using deformable convolution and separable feature downsampling respectively;

[0041] The optimized YOLOv10 model is used to detect defects in PCB boards.

[0042] In a specific embodiment, in order to implement the above method, this embodiment specifically includes the following four parts:

[0043] First-level optimization: Based on the three major characteristics of defect targets, deformable convolution and separation feature downsampling are used to optimize the YOLOv10 model to improve the accuracy of the PCB defect detection model and solve the problems of missed detection and false detection.

[0044] Step 1: PCB surface defects are complex and present in various forms, especially rat bite defects, which are rich in types and shapes. Based on the principle of deformable convolution, the CSPLayer_2Conv module (the convolution module in YOLOv10) was reconstructed, and the third generation of deformable convolution (DCNv3) was integrated into it to design CSPLayer_2DCNv3.

[0045] (1) DCNv3 convolutional neurons share weights, introduce a multi-grouping mechanism, and perform softmax normalization based on sample points. For the current pixel p0, DCNv3 is calculated as follows:

[0046]

[0047] Where G is the total number of aggregation groups. For the gth group, w g ∈R C×C′ represents the position-independent projection weight of the group, where C′=C / G represents the group dimension. gk ∈R represents the modulation scalar of the k-th sampling point in the g-th group, normalized by the softmax function along dimension K. g ∈R C′×H×W Represents the input feature map after slicing. Δp gk is the grid sampling position p corresponding to the gth group k , p0 represents the reference position on the input feature map.

[0048] Specifically, DCNv3 effectively overcomes the limitations of traditional convolution and attention-based methods, and enhances the ability to handle complex visual tasks by improving spatial aggregation and long-distance dependency processing. It inherits the inductive bias of convolution, allowing the model to learn efficiently even under conditions of limited data and short training time. In addition, DCNv3 uses sparse sampling and 3×3 cores to improve computational and memory efficiency, while simplifying the optimization process and reducing resource consumption, thus showing significant advantages in improving model performance. As a result, CSPLayer_2DCNv3 has the ability to adaptively extract feature information of PCB defects and long-range modeling capabilities while reducing model parameters and saving computing resources. The added deformable convolution module can flexibly handle the problem of insufficient receptive field of detection points corresponding to small targets, strengthen the focus on small targets, effectively improve missed detections and false detections, and improve detection accuracy. CSPLayer_2DCNv3 replaces Bottleneck in CSPLayer_2Conv, and integrates DCNv3 convolution into the new Bottleneck. The new Bottleneck structure inputs the feature map through deformable convolution, allowing the free deformation of the sampling grid. The offset learned from the feature map through the additional convolution layer fully learns the scale, posture, viewpoint and deformation of the target object, so that CSPLayer_2DCNv3 can output more effective feature information in gradient splitting. The detailed structure of CSPLayer_2DCNv3 is as follows: Figure 1 shown.

[0049] Step 2: In PCB surface defect detection, since the area occupied by the defect is very small relative to the entire PCB board, the defect feature information becomes sparse and low-resolution, and these features belong to shallow semantic features. In addition, the defect features are visually highly similar to the background features of the surrounding defect-free areas, especially the two defect types of copper and burrs. The use of convolution strides and / or pooling layers in convolutional neural networks leads to the loss of fine-grained information and the learning of less efficient feature representations. It is necessary to use an efficient downsampling method to replace the traditional convolution downsampling method. Separate Feature Map Downsampling Convolution (SPDConv) is designed to eliminate this existing defect of convolutional neural networks (CNNs) and consists of a space-to-depth (SPD) layer and a convolution-free stride (Conv) layer.

[0050] Specifically, SPDConv consists of a space-to-depth (SPD) layer and a non-strided convolution layer. SPDConv uses a raw image conversion technique to downsample the feature maps within and across CNNs, such as Figure 2 shown.

[0051] Consider an intermediate feature map X of arbitrary size S×S×C1, and slice the sub-feature map sequence into:

[0052]

[0053] Typically, given the original feature map X, the submap f x,y is composed of all entries X(i,j) such that (i+x) and (j+y) are divisible by the scale (scale). Therefore, each submap downsamples X by a scale factor. For example, when scale = 2, we get four submaps f 0,0 ,f 1,0 ,f 0,1 ,f 1,1 , the shape of each submap is And X is downsampled by a scale of 2.

[0054] In a specific embodiment, these sub-feature maps are then concatenated along the channel dimension to obtain a feature map X', which has a reduced scale scale times in the spatial dimension and an increased scale times in the channel dimension. 2 times.

[0055] After the SPD feature conversion layer, a non-strided (i.e., stride=1) convolutional layer is added, which has C2 filters, where C2 < scale 2 C1, and further map the feature Conversion The purpose of using non-strided convolution is to retain all discriminative feature information as much as possible.

[0056] Second-level optimization: Based on feature map visualization technology, the four downsampling convolutional layers after the backbone network of the YOLOv10 model are visualized and redundant downsampling convolutional layers are removed. While maintaining the model's ability to extract defect feature information, the number of model parameters and the amount of calculation are greatly reduced, achieving a good balance between accuracy and model complexity.

[0057] Specifically, the defect area of ​​PCB is very small relative to the area of ​​the entire PCB board, and the defect feature information is highly sparse and has a low resolution in space. When using the YOLOv10 model backbone network to extract PCB defect feature information, as the number of network downsampling increases, it becomes more difficult to capture useful information. This information loss is mainly due to repeated downsampling operations, which leads to the loss of high-frequency detail information, which is not conducive to the accurate identification of minor defects. In addition, repeated downsampling operations not only lead to information loss, but also unnecessarily increase the number of parameters and computational burden of the model. For this purpose, feature map visualization technology is used to visualize the feature maps of PCB defects extracted by the last four downsampling convolutional layers in the YOLOv10 model backbone network, as shown in Figure 2. Figure 3 shown.

[0058] Specifically, according to Figure 3 According to the analysis, as the number of network downsampling increases, it is found that the PCB defect feature information captured by the fourth and fifth downsampling convolution layers of the backbone network is significantly reduced. Therefore, these redundant downsampling convolution layers are removed, and only the first three downsampling convolution layers that can effectively capture the PCB defect feature information are retained. This strategy significantly reduces the number of model parameters and calculations while retaining the PCB defect detection accuracy to the maximum extent, thereby improving the model's operating efficiency and practicality.

[0059] Third-level optimization: Based on model quantization technology, the model data format is converted from 32-bit floating point (FP32) to 8-bit integer (INT8). Using the INT8 format for reasoning not only reduces the computational peak of model reasoning, but also optimizes resource utilization, allowing the model to be efficiently deployed on the edge microcontroller unit (MCU). This technology significantly improves the application potential of the model in resource-constrained environments and is suitable for scenarios that require high-speed and precise calculations on edge devices.

[0060] Specifically, INT8 quantization is used to reduce the storage requirements of the model and speed up the calculation, especially on resource-constrained hardware. The basic principle is to convert the model parameters from floating point numbers (such as FP32) to integers (such as INT8), which not only reduces the size of the model but also speeds up the inference process. The following are the steps of INT8 quantization:

[0061] (1) Quantization and dequantization:

[0062] Quantization is the process of converting continuous real values ​​into discrete integer values, expressed as:

[0063]

[0064] Dequantization is the process of converting a quantized integer value back to its original floating-point approximation:

[0065] x'≈s·(qz);

[0066] Where x is the original floating point value, q is the integer value after quantization, x' is the dequantized original floating point value, s is the scale factor used to adjust the value, and z is the zero point used to ensure that the representation of zero is accurate during the quantization process.

[0067] (2) Determine the scaling factor and zero point:

[0068] The scaling factor s can be determined based on the range of the data and the range of the integer. Using INT8 quantization, the range of integer values ​​is [-128, 127] (for signed integers), and the scaling factor can be:

[0069]

[0070] The zero point z is usually set to 0 if the data is strictly positive or if unsigned integers (0,255) are used.

[0071] (3) Model parameter quantification:

[0072] According to (1) and (2), the parameter w of the quantization model can be quantized by the following formula:

[0073]

[0074] And use dequantization at inference time:

[0075] w≈s·w int ;

[0076] Specifically, the model processed in this way can replace floating-point operations with integer operations, significantly improving the calculation speed, especially on hardware that does not support or has low efficiency in floating-point operations.

[0077] Specifically, the quantified PCB defect detection model performance is shown in Table 1.

[0078] Table 1 Defect performance after quantification of PCB detection model

[0079]

[0080]

[0081] Fourth level optimization: Convert the quantized Tflite binary file of the PCB defect detection model into an optimized C language PCB defect detection model through the STM32Cube.AI expansion package X-CUBE-AI-8.0.1, and successfully deploy it to the GD32H757MCU.

[0082] Specifically, GD32H75MCU adopts The core, the processor frequency is up to 600MHz, and the working performance at the highest frequency can reach 1552DMIPS. The test achieved an outstanding performance of 2888 points. The large built-in storage space of GD32H7 can support complex operating systems and embedded AI, machine learning (ML), etc.

[0083] Specifically, the PCB defect detection model is deployed in the GD32H757 MCU. The image resolution processed by the model is 256×256×3 as shown in Table 2. Under this configuration, the model achieves an average average precision (mAP) of 0.8194, requires 57.24 million multiplication-addition operations (MACCs), and occupies 372.51KB of read-only memory (ROM). The model uses X-CUBE-AI 8.0.1 as the inference framework to optimize the execution efficiency and resource management on the microcontroller unit (MCU).

[0084] Table 2 Model performance indicators

[0085]

[0086] The overall hardware framework diagram of the system is as follows Figure 4 , the written overall system code is burned into the GD32H757MCU. From Table 3, it is found that the system inference speed is 237.72 milliseconds. The MCU can complete the inference processing of an image in about 0.24 seconds, showing good performance and response speed in actual application scenarios. This configuration makes the system particularly suitable for industrial applications that require fast and real-time processing, such as online PCB defect detection.

[0087] Table 3 MCU overall system operating speed

[0088] Data Types Image resolution Chip Model Chip Type frequency Inference speed INT8 256×256×3 GD32H757 MCU 600HZ 237.72ms

[0089] On the other hand, an embodiment of the present invention discloses a PCB board defect detection device for an edge MCU, including: a memory, a processor, and a PCB board defect detection program stored in the memory and executable on the processor, the PCB board defect detection program being configured to implement a PCB board defect detection method for an edge MCU as described above.

[0090] On the other hand, an embodiment of the present invention discloses a storage medium, on which a PCB board defect detection program is stored. When the PCB board defect detection program is executed by a processor, a PCB board defect detection method for an edge MCU as described above is implemented.

[0091] In summary, in order to solve the problems of missed detection and false detection in PCB defect detection and overcome the limitations of edge MCU devices, the present invention proposes an optimization scheme, which optimizes YOLOv10 to propose a PCB defect detection model that can be deployed on edge MCU devices and has high real-time and high precision. This scheme can not only run in a resource-constrained hardware environment, but also ensure reliable detection performance in practical applications. This optimization scheme provides strong technical support for real-time PCB defect detection in the field of industrial automation.

[0092] In a specific embodiment, the present invention proposes a cascaded PCB defect detection model optimization scheme to improve the YOLOv10 model by mining the three major defect characteristics that lead to missed detection and false detection in PCB defect detection (complex targets and diverse shapes, low resolution and similar background) and the effective downsampling convolution layer that captures defect information features in the backbone network, thereby adopting deformable convolution technology, separation feature map downsampling and feature map visualization to propose a PCB defect detection model that can be deployed on edge MCU devices and has high real-time and high precision. First, in view of the complex and diverse shapes of PCB defects, the deformable convolution technology is introduced to effectively deal with the complexity and diversity of PCB surface defects. Secondly, in view of the low resolution and similar background of printed circuit board defect feature information, the separation feature map downsampling is adopted to retain the rich fine-grained information in the shallow features. Then, the feature map extracted by the backbone network of the visualization model is removed, and the redundant downsampling convolution layer is removed, while maintaining the model's ability to extract defect feature information, the model parameters and calculation amount are greatly reduced.

[0093] In a specific embodiment, the present invention firstly targets the business characteristics of the detection target (complex and diverse shapes, low resolution and similar background), and adopts (deformable convolution technology and separation feature map downsampling method) targeted technical methods to effectively alleviate the situation of missed detection and misdetection of target detection caused by the business characteristics of the detection target, without increasing the model parameters and calculation amount, and improves the accuracy of the PCB defect detection model. Secondly, in order to better balance the relationship between model parameters and accuracy, feature map visualization technology is used to simplify the backbone network, which effectively solves the problems of limited flash memory capacity, limited computing resources and slow model reasoning speed faced by AI models when deployed on MCU. Finally, the PCB defect detection model is deployed in the edge MCU, and the balance accuracy reaches 81.94%, and the reasoning time is about 237.72ms, which is lower than similar model libraries.

[0094] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0095] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A PCB board defect detection method for edge MCU, characterized in that: include: Get the PCB board image; The YOLOv10 model is optimized using deformable convolution and separable feature downsampling respectively; The optimized YOLOv10 model is used to detect defects in PCB boards.

2. A PCB board defect detection method for edge MCU according to claim 1, characterized in that: Optimizing the YOLOv10 model using deformable convolutions includes: Construct the CSPLayer_2Conv convolution module in YOLOv10, and integrate the deformable convolution third-generation DCNv3 into the CSPLayer_2Conv convolution module to obtain the CSPLayer_2DCNv3 convolution module.

3. A PCB board defect detection method for edge MCU according to claim 2, characterized in that: The calculation formula of the third generation of deformable convolution DCNv3 is: Where G is the total number of aggregation groups, for the gth group, w g ∈R C×C′ represents the position-independent projection weight of the g-th group, where C′=C / G represents the group dimension; m gk ∈R represents the modulation scalar of the kth sampling point in the gth group, which is normalized by the softmax function along dimension K; x g ∈R C′×H×W represents the input feature map after slicing; Δp gk is the grid sampling position p corresponding to the gth group k , p0 represents the reference position on the input feature map.

4. A PCB board defect detection method for edge MCU according to claim 1, characterized in that: Optimizing the YOLOv10 model using split feature downsampling includes: Constructing a separation feature map downsampling convolution SPDConv module, wherein the separation feature map downsampling convolution SPDConv module includes: a space-to-depth SPD layer and a non-strided convolution layer; The separated feature map downsampling convolution SPDConv module downsamples the CSPLayer_2DCNv3 output feature map of the backbone network in YOLOv10 through the original image conversion method.

5. A PCB board defect detection method for edge MCU according to claim 1, characterized in that: Optimizing the YOLOv10 model also includes: Remove the redundant downsampling convolutional layers in the YOLOv10 model backbone network.

6. A PCB board defect detection method for edge MCU according to claim 1, characterized in that: Optimizing the YOLOv10 model also includes: Convert the YOLOv10 model data format from 32-bit floating point to 8-bit integer.

7. A PCB board defect detection method for edge MCU according to claim 1, characterized in that: Use the optimized YOLOv10 model to detect PCB board defects, including: The optimized YOLOv10 model is used as the PCB defect detection model. After quantization, the Tflite binary file is generated. The Tflite binary file is converted into an optimized C language PCB defect detection model through the STM32Cube.AI expansion package X-CUBE-AI-8.0.1, and deployed to the GD32H757MCU to detect PCB board defects.

8. A PCB board defect detection device for edge MCU, characterized in that: include: A memory, a processor, and a PCB board defect detection program stored in the memory and executable on the processor, wherein the PCB board defect detection program is configured to implement a PCB board defect detection method for an edge MCU as described in any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium stores a PCB board defect detection program, and when the PCB board defect detection program is executed by the processor, the PCB board defect detection method for the edge MCU according to any one of claims 1 to 7 is implemented.

Images