PCB defect detection method, system, equipment and storage medium

Abstract

The present application discloses a PCB defect detection method, system, device and storage medium, which belongs to the technical field of industrial product defect detection. The method comprises: obtaining a PCB data set, performing image preprocessing and defect annotation on the PCB data set; creating a virtual operating environment, building a YOLOv10 model in the virtual operating environment, and performing lightweight processing on the YOLOv10 model; optimizing a spatial to depth convolution SPD_conv module to a DSPD module, and configuring the DSPD module to the YOLOv10 model to obtain a DSPD_YOLOv10 model; building a bidirectional feature pyramid network BiFPN, and configuring it to the neck network of the DSPD_YOLOv10 model; training the DSPD_YOLOv10 model according to the PCB data set, and detecting a sample to be detected according to the trained DSPD_YOLOv10 model; and using the DSPD_YOLOv10 model to detect PCB defects, high-precision and high-efficiency detection of PCB defects is achieved.

Description

Technical Field

[0001] The present application belongs to the technical field of industrial product defect detection, and specifically relates to a PCB defect detection method, system, equipment and storage medium. Background Art

[0002] As an important component of electronic products, printed circuit boards (PCBs) have circuit components and dense wire layouts on their surfaces. The quality of PCBs is related to the performance and life of electronic products. The development of integrated circuit (IC) packaging technology is driving electronic products to be lighter, thinner and more compact, which also requires PCB circuit layouts to be more compact. Therefore, the industry's requirements for PCB quality will become higher and higher.

[0003] Defects are inevitable in the PCB production process; defect detection is a key link to ensure PCB quality. PCB defect detection faces some difficulties that need to be overcome urgently. The size of defects such as missing hole, mouse bite, open circuit, short circuit, spur, spurious copper, etc. accounts for a relatively small proportion of the entire PCB image. The background image of the PCB also affects the detection. These will have a great impact on the accuracy of PCB detection; therefore, how to solve these problems to improve the detection accuracy of PCB is the current difficulty. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a PCB defect detection method, system, device and storage medium, which can solve the technical problem of low PCB defect detection accuracy in the prior art.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a PCB defect detection method, the method comprising:

[0007] Acquire a PCB data set, and perform image preprocessing and defect annotation on the PCB data set;

[0008] Creating a virtual operating environment, constructing a YOLOv10 model in the virtual operating environment, and performing lightweight processing on the YOLOv10 model;

[0009] Optimize the spatial-to-depth convolution SPD_conv module into a DSPD module, and configure the DSPD module into the YOLOv10 model to obtain a DSPD_YOLOv10 model;

[0010] Construct a bidirectional feature pyramid network BiFPN, and configure the bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model;

[0011] Input the PCB data set into the DSPD_YOLOv10 model for training, and evaluate the training results;

[0012] The sample to be detected is detected according to the trained DSPD_YOLOv10 model to obtain the detection result.

[0013] As an optional implementation of the first aspect of the present application, the PCB data set is obtained, and image preprocessing and defect annotation are performed on the PCB data set; specifically as follows:

[0014] Industrially produced PCB boards are imaged and sorted to obtain a PCB dataset containing defects, including missing holes, mouse bites, open circuits, short circuits, burrs and fake copper; the PCB dataset is expanded by data enhancement; the expanded PCB dataset is subjected to adaptive linear interpolation super-resolution reconstruction processing, and the defects are marked, and finally the PCB dataset is divided into a training set, a test set and a validation set.

[0015] According to the above optional implementation, the diversity and clarity of the data set are increased through data enhancement and adaptive linear interpolation super-resolution reconstruction processing, thereby improving the model's ability to recognize tiny PCB defects; accurate defect labeling and reasonable data set division help the model learn more accurate feature representation, thereby improving the accuracy of defect detection; although defect labeling requires a certain amount of manpower investment, this burden can be greatly reduced through automated and semi-automated tools; at the same time, once the model training is completed, it can efficiently detect a large number of PCB images, thereby significantly reducing the time and cost of manual inspection.

[0016] As an optional implementation of the first aspect of the present application, the process of lightweight processing the YOLOv10 model is:

[0017] Replace the ordinary convolution in the YOLOv10 model with a lightweight ghost convolution GhostConv.

[0018] According to the above optional implementation, by introducing GhostConv, the computational complexity of the YOLOv10 model can be significantly reduced; the generation process of the ghost feature map is more efficient than the traditional convolution operation, thereby reducing the computational complexity of the model and reducing memory usage; the YOLOv10 model after lightweight processing has a smaller memory usage; this helps to deploy the model in resource-constrained environments (such as embedded devices or mobile devices) and improve its practicality; due to the reduction in computational complexity and memory usage, the running speed of the YOLOv10 model after lightweight processing will be improved. This is particularly important for real-time defect detection tasks, which can shorten the detection time and improve production efficiency; although lightweight processing will reduce the computational complexity and memory usage of the model, through reasonable network design and parameter adjustment, it can be ensured that the performance of the model will not be greatly affected, which means that the lightweight YOLOv10 model can still maintain a high defect detection accuracy.

[0019] As an optional implementation of the first aspect of the present application, the space-to-depth convolution SPD_conv module includes a space-to-depth layer and a non-strided convolution layer; the space-to-depth convolution SPD_conv module is optimized to a DSPD module, and the DSPD module is configured in the YOLOv10 model, specifically:

[0020] The non-strided convolution layer in the space-to-depth convolution SPD_conv module is deleted, and the space-to-depth layer is optimized by adaptive weighting processing and depth sorting to obtain the DSPD module;

[0021] The step size of the first convolution of the backbone network in the YOLOv10 model is kept unchanged, and the DSPD module is configured after the remaining convolutional layers.

[0022] According to the above optional implementation, the traditional non-strided convolution layer is deleted to avoid information loss caused by the compression channel. By deleting the non-strided convolution layer, the detection accuracy of the YOLOv10 model can be improved; the defect feature proportion of each feature sub-graph can be obtained by adaptive weighting processing, and then the corresponding weight is assigned, so that the model can process the defects more targeted, reduce unnecessary resource waste, and further improve the detection accuracy of the model; the spatial dimension of each feature sub-graph is ensured to remain the same through spatial dimension processing; the depth sorting of the space-to-depth layer is optimized, which helps to improve the operation efficiency of the YOLOv10 model, especially in a resource-constrained environment; the DSPD module can better extract spatial features and convert the spatial dimension of the input image into the depth dimension. Compared with the compression channel of the non-strided convolution layer, the channel processing is more targeted and the loss of feature information is reduced; when processing low-resolution images and small objects, the DSPD module needs to retain as much spatial information as possible. The DSPD module avoids information loss in traditional stride convolution and pooling operations by converting the information of the spatial dimension into the depth dimension; by reordering the depth dimension of the features, the detection speed of the model can be accelerated.

[0023] As an optional implementation of the first aspect of the present application, the process of the DSPD module processing an image is as follows:

[0024] The input feature map is sliced ​​to obtain several feature sub-maps. The specific formula is as follows:

[0025] ;

[0026] The feature subgraphs are subjected to adaptive weighting processing and spatial dimension processing to make the spatial dimensions of the feature subgraphs consistent. The specific formula is as follows:

[0027] ;

[0028] Then, the feature sub-graphs after adaptive weighting processing and spatial dimension processing are spliced. The specific formula is as follows:

[0029] ;

[0030] in, represents the sampling slicing function, X represents the input feature map, s represents the scale, e represents the sampling step size, X s-1，s-1 represents the feature at position (s-1, s-1) in the input feature information, P s-1，s-1 X represents the adaptive weighting and spatial dimension processing s-1，s-1 , , , and Respectively represent X s-1，s-1 , X s-2，s-1 , X s-1，s-2 and X 0，0 The corresponding feature weight value, Indicates the spatial dimension processing of features. It represents the output feature information corresponding to the scale s, and cat is a function that concatenates the feature maps.

[0031] According to the above optional implementation, by setting the step length e and the ratio s, the sampling and slicing function samples and slices the feature information into a number of slices of the same size, and selects the required slice containing defects according to the preset conditions; for example, X s-2,s-1 , which is a slice among several slices, and the subscripts s-2 and s-1 represent the horizontal position and vertical position of the slice respectively; after the slice selection is completed, each feature map is adaptively weighted and processed in spatial dimension; the model is more targeted in dealing with defects, which reduces unnecessary waste of resources and further improves the detection accuracy of the model; the spatial dimension of each feature sub-map is ensured to remain the same; the cat function splices the slices selected by the sampling slice function that meet the preset features in a preset order, so as to obtain a new feature map containing a large number of slices containing defect features; the DSPD module generates a new feature representation by slicing and splicing the slices in depth dimension, which helps the model capture spatial features more effectively; in the processing process, the selection of scale s, the processing of step size e, the adjustment of index and offset, and the order of feature splicing are all factors that affect the final new features; by reasonably designing the DSPD module, it can be effectively integrated into deep learning models such as YOLOv10 to improve the performance and efficiency of the model.

[0032] As an optional implementation of the first aspect of the present application, configuring the bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model is specifically:

[0033] Part of the Concat layer in the neck network of the YOLOv10 model is replaced with the bidirectional feature pyramid network BiFPN.

[0034] According to the above optional implementation manner, the BiFPN module can generate more expressive feature maps through cross-scale feature fusion, thereby improving the model's detection ability for targets of different scales; configuring BiFPN to the neck network of the DSPD_YOLOv10 model can improve the detection accuracy and efficiency of the model; the selected partial Concat layer replacement layer is more conducive to capturing small target information than the remaining Concat layers, and the partial replacement is because the present invention selects an optimal balance between accuracy and inference speed; it will neither ignore the requirements of meeting industrial speed in pursuit of accuracy, nor give up accuracy in pursuit of speed. When replacing the Concat layer with the BiFPN module, it is necessary to ensure that the input and output sizes of the model and the number of feature maps remain unchanged to maintain the compatibility of the model.

[0035] As an optional implementation manner of the first aspect of the present application, in the evaluation of the training results, the evaluation indicators include recall rate, accuracy, average accuracy, average accuracy mean and frame rate;

[0036] The recall rate is calculated by the following formula:

[0037] ;

[0038] The accuracy is calculated by the following formula:

[0039] ;

[0040] The average accuracy is calculated by the following formula:

[0041] ;

[0042] The mean average precision is calculated by the following formula:

[0043] ;

[0044] The frame rate is calculated by the following formula:

[0045] ;

[0046] Among them, Recall represents the recall rate, Precision represents the accuracy, AP represents the average accuracy, mAP represents the mean of the average accuracy, FPS represents the frame rate, and F N Indicates the number of samples predicted as negative samples which are actually positive samples, F P Indicates the number of samples predicted as positive which are actually negative samples, T P represents the number of positive samples predicted to be positive, P(r) represents the function image surrounded by recall and precision, n represents the number of defect categories, i represents the i-th category, F nRepresents the number of images to be detected, and T represents the time used to detect all the images to be detected.

[0047] According to the above optional implementation, the average precision is the area value obtained by integrating the precision-recall (PR) curve under different confidence thresholds; it provides a single performance indicator for measuring the accuracy of the model at different recall levels; the mean average precision is one of the key indicators for measuring the performance of the target detection algorithm, which takes into account the average detection accuracy of all categories; by calculating the accuracy of each category and taking the average value, the mean average precision can comprehensively reflect the detection performance of the algorithm on different categories; the higher the mean average precision, the more accurately the algorithm can identify the target object and the more stable its performance on different categories; the precision reflects the proportion of instances predicted by the algorithm as positive samples that are actually positive samples; a high precision means It means that the results predicted by the algorithm are more reliable and reduce false alarms. In PCB defect detection, high precision can ensure that the detected defects are more accurate, reducing unnecessary rework and cost waste. The recall rate measures the proportion of positive samples that the algorithm can correctly identify to the total positive samples. A high recall rate means that the algorithm can detect more actual defects and reduce missed alarms. In practical applications, a high recall rate is crucial to ensuring product quality and safety. The frame rate indicates the number of images that the algorithm can process per second. The higher the frame rate value, the faster the algorithm can process input images in real time or near real time. In PCB defect detection, a high frame rate value can ensure efficient operation of the production line and reduce waiting time and cost.

[0048] In a second aspect, an embodiment of the present application provides a PCB defect detection system, the system comprising:

[0049] A first acquisition module is used to establish a YOLOv10 model and obtain a PCB data set required by the network model, wherein the PCB data set includes a training set, a verification set, and a test set;

[0050] A first processing module, configured to preprocess the PCB data set acquired by the first acquisition module to obtain the preprocessed PCB data set;

[0051] A second processing module, used for performing lightweight processing on the YOLOv10 model established by the first acquisition module;

[0052] The first improvement module optimizes the space-to-depth convolution DSPD_conv module to a DSPD module, and configures the DSPD module to the YOLOv10 model after lightweight processing to obtain a DSPD_YOLOv10 model;

[0053] The second improvement module configures the preset bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model;

[0054] A first training module, training the DSPD_YOLOv10 model improved by the second improvement module through the PCB data set preprocessed by the first processing module;

[0055] The first evaluation module is used to evaluate the training result of the first training module.

[0056] According to a PCB defect detection system of the second aspect of the present application, the first acquisition module can establish a YOLOv10 model, and effectively acquire and divide the PCB data set into a training set, a validation set and a test set, which ensures that there is sufficient and diverse data support during the model training process, thereby improving the generalization ability of the model; the first processing module preprocesses the PCB data set, which usually includes operations such as image enhancement, which helps to improve the model's recognition accuracy of PCB defects and reduce the impact of noise on model training; the second processing module performs lightweight processing on the YOLOv10 model, which can significantly reduce the computational complexity and storage requirements of the model, making the model more suitable for running on resource-constrained hardware, thereby improving the practical applicability of the system; the first improvement module can further improve the detection accuracy of the model by optimizing the space-to-depth convolution SPD_conv module to a DSPD module and configuring it in the lightweight YOLOv10 model. The DSPD module may enhance the model's ability to identify PCB defects through more effective feature extraction and conversion; the second improvement module configures the preset bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model, which helps the model to better integrate multi-scale features and improve the detection ability of PCB defects of different sizes and shapes; the first training module uses the preprocessed PCB dataset to train the improved DSPD_YOLOv10 model to ensure that the model can learn the effective features of PCB defects; the first evaluation module evaluates the training results, which can quantify the performance of the model, including detection accuracy, recall rate and other indicators, and provide data support for further optimization of the model.

[0057] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0058] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0059] In the embodiment of the present application, by optimizing the SPD_conv module to the DSPD module and configuring the BiFPN network, the DSPD_YOLOv10 model can more effectively capture and utilize the feature information in the image, thereby improving the accuracy of defect detection; lightweight processing of the YOLOv10 model can reduce the computational complexity and memory usage of the model, thereby improving the operating efficiency of the model and making it more suitable for deployment in practical applications; by creating a virtual operating environment, the dependencies in the model development process can be easily managed to ensure the consistency and repeatability of the model in different environments, which helps to enhance the scalability and maintainability of the model; in summary, the PCB defect detection method achieves high-precision and high-efficiency detection of PCB defects through a series of technical optimizations and innovations, and provides strong technical support for the automation and intelligence of the PCB manufacturing industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flow chart of a PCB defect detection method provided by some embodiments of the present application;

[0061] Figure 2 It is a schematic diagram of a DSPD_YOLOv10 model in a PCB defect detection method provided in some embodiments of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0064] In the following, in combination with the accompanying drawings, a PCB defect detection method, system, device and storage medium provided by the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0065] Example

[0066] A PCB defect detection method comprises the following steps:

[0067] S100: Acquire a PCB data set, perform image preprocessing and defect annotation on the PCB data set;

[0068] Furthermore, the specific process is as follows: imaging industrially produced PCB boards, and organizing them to obtain a PCB dataset containing defects, including missing holes, rat bites, open circuits, short circuits, burrs and fake copper; expanding the PCB dataset by means of data enhancement; performing adaptive linear interpolation super-resolution reconstruction on the expanded PCB dataset, marking the defects, and finally dividing the PCB dataset into a training set, a test set and a validation set.

[0069] It should be understood that, based on the above-mentioned feature definition, first, the produced PCB boards are imaged by industrial cameras or other imaging devices. These images should clearly show various details on the PCB boards, including possible defects; the PCB images obtained by imaging are sorted to form a PCB data set containing various defects (such as missing holes, rat bites, open circuits, short circuits, burrs and fake copper). This step is the basis for subsequent processing; in order to increase the diversity and robustness of the data set, the PCB data set is expanded by data enhancement. Data enhancement techniques may include rotation, scaling, flipping, color transformation, etc. These operations can generate more training samples without changing the essential content of the image; the expanded PCB data set is subjected to adaptive linear interpolation super-resolution reconstruction processing; this step aims to improve the resolution and clarity of the image, thereby helping the model to more accurately identify defects; adaptive linear interpolation is an image interpolation method that can adaptively select interpolation weights according to local features of the image to obtain better reconstruction effects; after image preprocessing, the defects in the image need to be accurately labeled; usually involving manual or semi-automatic tools to mark the location and type of defects. The accuracy of annotation is crucial to the training effect of the model. Finally, the processed PCB data set is divided into training set, test set and validation set. The training set is used for the model training process; the test set is used to evaluate the performance of the model, that is, the performance on unseen data; the validation set is used to adjust the model parameters during the training process to avoid overfitting. This division method helps to ensure the generalization ability and stability of the model.

[0070] It should be noted that data enhancement and adaptive linear interpolation super-resolution reconstruction processing increase the diversity and clarity of the data set, thereby improving the model's ability to recognize tiny PCB defects; accurate defect annotation and reasonable data set division help the model learn more accurate feature representation, thereby improving the accuracy of defect detection; although defect annotation requires a certain amount of manpower investment, this burden can be greatly reduced through automated and semi-automated tools; at the same time, once the model training is completed, it can efficiently detect a large number of PCB images, thereby significantly reducing the time and cost of manual inspection.

[0071] S200: creating a virtual operating environment, building a YOLOv10 model in the virtual operating environment, and performing lightweight processing on the YOLOv10 model;

[0072] Furthermore, the process of lightweight processing of the YOLOv10 model is as follows:

[0073] Replace the ordinary convolution in the YOLOv10 model with a lightweight ghost convolution GhostConv.

[0074] It should be understood that, based on the above-mentioned feature limitations, in the PCB defect detection method, lightweight processing of the YOLOv10 model is a key step, which aims to reduce the computational complexity and memory usage of the model and improve its operating efficiency in resource-constrained environments; specifically, the lightweight processing process is to replace the ordinary convolution in the YOLOv10 model with a lightweight ghost convolution (GhostConv); ordinary convolution directly generates all feature maps (feature maps) in the convolution operation, while GhostConv generates feature maps in a more efficient way; it first generates a part of the feature map (called the basic feature map) through a small number of traditional convolution operations, and then generates the remaining feature maps (called ghost feature maps) through linear transformations (such as cheap operations or depth-separable convolutions); these ghost feature maps have similar feature information to the basic feature maps, but the computational complexity is greatly reduced;

[0075] It should be noted that by introducing GhostConv, the computational complexity of the YOLOv10 model can be significantly reduced; the generation process of the ghost feature map is more efficient than the traditional convolution operation, thereby reducing the computational complexity of the model and reducing memory usage: the lightweight YOLOv10 model has a smaller memory usage; this helps to deploy the model in resource-constrained environments (such as embedded devices or mobile devices) and improve its practicality; due to the reduction in computational complexity and memory usage, the lightweight YOLOv10 model will be improved in running speed. This is especially important for real-time defect detection tasks, which can shorten the detection time and improve production efficiency; although lightweight processing will reduce the computational complexity and memory usage of the model, through reasonable network design and parameter adjustment, it can be ensured that the performance of the model will not be greatly affected, which means that the lightweight YOLOv10 model can still maintain a high defect detection accuracy.

[0076] S300: Optimize the spatial-to-depth convolution SPD_conv module to a DSPD module, and configure the DSPD module into the YOLOv10 model to obtain the DSPD_YOLOv10 model;

[0077] Furthermore, the space-to-depth convolution SPD_conv module includes a space-to-depth layer and a non-strided convolution layer; the space-to-depth convolution SPD_conv module is optimized to a DSPD module, and the DSPD module is configured into the YOLOv10 model, specifically:

[0078] The non-strided convolution layer in the space-to-depth convolution SPD_conv module is deleted, and the space-to-depth layer is optimized by adaptive weighting and depth sorting to obtain the DSPD module;

[0079] Keep the stride of the first convolution of the backbone network in the YOLOv10 model unchanged, and configure the DSPD module after the remaining convolutional layers;

[0080] Specifically, in the YOLOv10 model, the DSPD module is configured after GhostConv2, GhostConv3, SCDown1 and SCDown2 of the backbone network, and GhostConv1 and SCDown1 of the neck network; at the same time, the convolution step size of GhostConv2, GhostConv3, SCDown1 and SCDown2 in the backbone network is set to 1, the step size of GhostConv1 and SCDown1 in the neck network is set to 1, and the upsampling structure in the neck network remains unchanged.

[0081] It should be understood that, based on the above feature restrictions, the original space-to-depth convolution (SPD_conv) module includes a space-to-depth layer and a non-strided convolution layer; in the optimization process, the non-strided convolution layer in the SPD_conv module is first deleted. This step aims to improve the module's ability to obtain information, because the non-strided convolution layer will perform channel compression on the concatenated features, which will cause some information loss; after deleting the non-strided After the convolution layer, the depth sorting of the space-to-depth layer is optimized; depth sorting refers to the rearrangement of the channel dimensions of the input data to better utilize the characteristics of the space-to-depth layer; the optimized depth sorting can make the space-to-depth layer extract spatial features more effectively while reducing the amount of computation; finally, the optimized DSPD module is configured in the YOLOv10 model. In order to maintain the structural integrity and performance of the model, the step size of the first convolution of the backbone network in the YOLOv10 model is kept unchanged, and the DSPD module is configured after the remaining convolution layers; this configuration method can ensure that the DSPD module can make full use of the feature information extracted by the backbone network while reducing the amount of computation and memory usage of the model.

[0082] It should be noted that the traditional non-strided convolution layer is deleted to avoid information loss caused by the compression channel. By deleting the non-strided convolution layer, the detection accuracy of the YOLOv10 model can be improved; the defect feature proportion of each feature sub-graph can be obtained through adaptive weighting processing, and then the corresponding weight is assigned, so that the model can handle defects more targeted, reduce unnecessary resource waste, and further improve the detection accuracy of the model; the spatial dimension of each feature sub-graph is ensured to remain the same through spatial dimension processing; the depth sorting of the space-to-depth layer is optimized, which helps to improve the operating efficiency of the YOLOv10 model, especially in resource-constrained environments; the DSPD module can better extract spatial features and convert the spatial dimension of the input image into the depth dimension. Compared with the compression channel of the non-strided convolution layer, the channel processing is more targeted and the loss of feature information is reduced; when processing low-resolution images and small objects, the DSPD module needs to retain as much spatial information as possible. The DSPD module avoids information loss in traditional stride convolution and pooling operations by converting the information of the spatial dimension into the depth dimension; by reordering the depth dimension of the features, the detection speed of the model can be accelerated.

[0083] Furthermore, the process of image processing by the DSPD module is as follows:

[0084] The input feature map is sliced ​​to obtain several feature sub-maps. The specific formula is as follows:

[0085] ;

[0086] The feature subgraphs are adaptively weighted and processed in terms of spatial dimension to make the spatial dimensions of the feature subgraphs consistent. The specific formula is as follows:

[0087] ;

[0088] Then, the feature sub-graphs after adaptive weighting processing and spatial dimension processing are spliced. The specific formula is as follows:

[0089] ;

[0090] in, represents the sampling slicing function, X represents the input feature map, s represents the scale, e represents the sampling step size, X s-1，s-1 represents the feature at position (s-1, s-1) in the input feature information, P s-1，s-1 X represents the adaptive weighting and spatial dimension processing s-1，s-1 , , , and Respectively represent X s-1，s-1 , X s-2，s-1 , X s-1，s-2 and X 0，0 The corresponding feature weight value, Indicates the spatial dimension processing of features. It represents the output feature information corresponding to the scale s, and cat is a function that concatenates the feature maps.

[0091] Specifically, based on the above formula, let X be the input feature map with a size of C×H×W, where C, H, and W represent the number of channels, height, and width of the feature map X respectively; the initial feature map X is sliced ​​by the DSPD module with a scale of 2 and a step size of 1 to obtain X 0，0 , X 1，0 , X 0，1 , X 1，1 Four feature sub-graphs, each with 1 / 4 times the number of channels and 1 / 2 times the length and width of the initial feature graph X, are then processed by adaptive weighting and spatial dimension processing to obtain P 0，0 , P 1，0 , P0，1 , P 1，1 Finally, the channel dimension is optimized and merged to obtain a new feature map with the depth of the initial feature map. The depth of the new feature map is the original ; The splicing process is as follows:

[0092] .

[0093] It should be understood that, based on the above feature definition, by setting the step size e and the ratio s, the sampling slicing function samples and slices the feature information into several slices of the same size; for example, X s-2,s-1 , which is a slice among several slices, and the subscripts s-2 and s-1 represent the horizontal position and vertical position of the slice respectively; after the slicing operation is completed, the slice is adaptively weighted and processed in spatial dimension; the cat function stacks and splices the slices that have been adaptively weighted and processed in spatial dimension in sequence, thereby improving the model's ability to capture features.

[0094] It should be noted that by setting the step size e and the ratio s, the sampling and slicing function samples and slices the feature information into several slices of the same size, and selects the required slice containing defects according to the preset conditions; for example, X s-2,s-1 , which is a slice among several slices, and the subscripts s-2 and s-1 represent the horizontal position and vertical position of the slice respectively; after the slice selection is completed, each feature map is adaptively weighted and processed in spatial dimension; the model is more targeted in dealing with defects, which reduces unnecessary waste of resources and further improves the detection accuracy of the model; the spatial dimension of each feature sub-map is ensured to remain the same; the cat function splices the slices selected by the sampling slice function that meet the preset features in a preset order, so as to obtain a new feature map containing a large number of slices containing defect features; the DSPD module generates a new feature representation by slicing and splicing the slices in depth dimension, which helps the model capture spatial features more effectively; in the processing process, the selection of scale s, the processing of step size e, the adjustment of index and offset, and the order of feature splicing are all factors that affect the final new features; by reasonably designing the DSPD module, it can be effectively integrated into deep learning models such as YOLOv10 to improve the performance and efficiency of the model.

[0095] S400: Build a bidirectional feature pyramid network BiFPN and configure the bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model;

[0096] Further, the configuration process is as follows:

[0097] Replace some Concat layers in the neck network of the YOLOv10 model with the bidirectional feature pyramid network BiFPN;

[0098] Specifically, the twelfth layer Concat0 and the fifteenth layer Concat1 in the original neck network of YOLOv10 are replaced with BiFPN, and the rest of the Concat layers remain unchanged.

[0099] It should be understood that based on the above feature restrictions, in the neck network of the YOLOv10 model, the Concat layer is used to splice feature maps of different levels; these feature maps usually have different resolutions and numbers of channels, and it is necessary to identify these Concat layers and determine which layers can be replaced by the BiFPN module; the selected twelfth layer Concat0 and the fifteenth layer Concat1 is replaced with a BiFPN module, which fuses features through top-down and bottom-up paths, and uses weighted features and cross-scale connections to enhance the effect of feature fusion; during the replacement process, it is necessary to ensure that the input and output of the BiFPN module are consistent with those of the original Concat layer to maintain the integrity and performance of the model; the BiFPN module contains multiple learnable parameters, such as weights and biases, which need to be optimized during the training process; according to the requirements of the DSPD_YOLOv10 model and the characteristics of the dataset, the parameters of the BiFPN module are adjusted to achieve the best performance; and, after replacing and adjusting the parameters, the DSPD_YOLOv10 model needs to be retrained to ensure that the BiFPN module can work effectively with the rest of the model; during the training process, the performance of the model needs to be monitored and adjusted and optimized as needed.

[0100] It should be noted that the BiFPN module can generate more expressive feature maps through cross-scale feature fusion, thereby improving the model's detection ability for targets of different scales; configuring BiFPN to the neck network of the DSPD_YOLOv10 model can improve the detection accuracy and efficiency of the model; the selected partial Concat layer replacement layer is more conducive to the capture of small target information than the remaining Concat layers, and the partial replacement is because the present invention selects an optimal balance between accuracy and inference speed; it will neither ignore the requirements of meeting industrial speed in pursuit of accuracy, nor give up accuracy in pursuit of speed. When replacing the Concat layer with the BiFPN module, it is necessary to ensure that the input and output sizes of the model and the number of feature maps remain unchanged to maintain the compatibility of the model.

[0101] S500: Input the PCB data set into the DSPD_YOLOv10 model for training, and evaluate the training results;

[0102] Furthermore, in evaluating the training results, the evaluation indicators include recall rate, precision, average precision, mean average precision and frame rate;

[0103] The recall rate is calculated by the following formula:

[0104] ;

[0105] The accuracy is calculated by the following formula:

[0106] ;

[0107] The average accuracy is calculated by the following formula:

[0108] ;

[0109] The mean average precision is calculated by the following formula:

[0110] ;

[0111] The frame rate is calculated using the following formula:

[0112] ;

[0113] Among them, Recall represents the recall rate, Precision represents the accuracy, AP represents the average accuracy, mAP represents the mean of the average accuracy, FPS represents the frame rate, and F N Indicates the number of samples predicted as negative samples which are actually positive samples, F P Indicates the number of samples predicted as positive which are actually negative samples, T P represents the number of positive samples predicted to be positive, P(r) represents the function image surrounded by recall and precision, n represents the number of defect categories, i represents the i-th category, F n Represents the number of images to be detected, and T represents the time used to detect all the images to be detected.

[0114] It should be understood that, based on the above feature definitions, the average precision is the area value obtained by integrating the precision-recall (PR) curve under different confidence thresholds; it provides a single performance indicator for measuring the accuracy of the model at different recall levels; the mean average precision is one of the key indicators for measuring the performance of target detection algorithms, which takes into account the average detection accuracy of all categories; by calculating the accuracy of each category and taking the average, the mean average precision can comprehensively reflect the detection performance of the algorithm in different categories; the higher the mean average precision, the more accurately the algorithm can identify the target object and the more stable its performance in different categories; the precision rate reflects the proportion of instances predicted by the algorithm as positive samples that are actually positive samples; high precision Accuracy means that the results predicted by the algorithm are more reliable and reduce false alarms. In PCB defect detection, high precision can ensure that the detected defects are more accurate, reducing unnecessary rework and cost waste. Recall measures the proportion of positive samples that the algorithm can correctly identify to the total positive samples. A high recall means that the algorithm can detect more actual defects and reduce missed alarms. In practical applications, a high recall is crucial to ensuring product quality and safety. Frame rate refers to the number of images that the algorithm can process per second. The higher the frame rate value, the faster the algorithm can process input images in real time or near real time. In PCB defect detection, a high frame rate value can ensure efficient operation of the production line and reduce waiting time and cost.

[0115] S600: Detect the sample to be detected according to the trained DSPD_YOLOv10 model to obtain the detection result;

[0116] According to a PCB defect detection method of this embodiment, first, a large amount of PCB (printed circuit board) image data needs to be collected, and these data should contain various possible defect types, such as breakage, short circuit, missing components, etc.; the collected PCB images are preprocessed, such as denoising, contrast enhancement, etc., to improve image quality. At the same time, the defects in the image are accurately annotated, which is the basis for training the model to identify defects; in order to isolate and manage dependencies in the model development process, a virtual operating environment is created, which helps to ensure the consistency and repeatability of the model in different environments; in the virtual operating environment, a YOLOv10 model is constructed, which is an advanced target detection model with high accuracy and real-time performance. In order to further improve the operating efficiency of the model, it is lightweight, such as pruning and quantization. The spatial-to-depth convolution (SPD_conv) module is optimized to a more efficient DSPD module and configured in the YOLOv10 model. This step aims to improve the model's ability to process spatial features while reducing the amount of calculation. Configure the bidirectional feature pyramid network BiFPN: Configure the bidirectional feature pyramid network (BiFPN) in the neck network of the DSPD_YOLOv10 model. BiFPN can more effectively fuse feature information of different scales, thereby enhancing the model's ability to capture shallow feature information and improve the accuracy of defect detection. The DSPD_YOLOv10 model is trained using the preprocessed PCB dataset. During the training process, the model learns how to identify and locate defects in the image. After the training is completed, the main performance indicators of the model are analyzed, such as accuracy and recall. These indicators can reflect the performance of the model in the defect detection task. Based on the analysis results, the performance of the model is evaluated. If the model performs poorly, it may be necessary to adjust the model structure, optimize parameters, or increase training data. If the model performs well, the next application can be carried out.

[0117] It should be noted that by optimizing the SPD_conv module to the DSPD module and configuring the BiFPN network, the DSPD_YOLOv10 model can more effectively capture and utilize the feature information in the image, thereby improving the accuracy of defect detection; lightweight processing of the YOLOv10 model can reduce the computational complexity and memory usage of the model, thereby improving the operating efficiency of the model and making it more suitable for deployment in practical applications; by creating a virtual operating environment, the dependencies in the model development process can be easily managed to ensure the consistency and repeatability of the model in different environments, which helps to enhance the scalability and maintainability of the model; in summary, the PCB defect detection method has achieved high-precision and high-efficiency detection of PCB defects through a series of technical optimizations and innovations, providing strong technical support for the automation and intelligence of the PCB manufacturing industry.

[0118] It should be noted that the PCB defect detection method provided in the embodiment of the present application can be executed by a PCB defect detection system, or a control module in the PCB defect detection system for executing and loading a PCB defect detection method. In the embodiment of the present application, a PCB defect detection method provided in the embodiment of the present application is described by taking a PCB defect detection system executing and loading a PCB defect detection method as an example.

[0119] A PCB defect detection system includes the following modules:

[0120] The first acquisition module is used to establish a YOLOv10 model and obtain the PCB data set required by the network model. The PCB data set includes a training set, a validation set, and a test set;

[0121] A first processing module is used to preprocess the PCB data set acquired by the first acquisition module to obtain a preprocessed PCB data set;

[0122] A second processing module is used to perform lightweight processing on the YOLOv10 model established by the first acquisition module;

[0123] The first improvement module optimizes the spatial-to-depth convolution SPD_conv module to a DSPD module, and configures the DSPD module to the lightweight YOLOv10 model to obtain the DSPD_YOLOv10 model;

[0124] The second improvement module configures the preset bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model;

[0125] The first training module trains the DSPD_YOLOv10 model improved by the second improvement module through the PCB data set preprocessed by the first processing module;

[0126] The first evaluation module is used to evaluate the training result of the first training module.

[0127] A PCB defect detection system in an embodiment of the present application may be a device, or a component, integrated circuit, or chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), etc., which is not specifically limited in the embodiment of the present application.

[0128] A PCB defect detection system in the embodiment of the present application is a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0129] The PCB defect detection system provided in the embodiment of the present application can realize Figure 1 to Figure 2 The various processes of implementing a PCB defect detection method in the method embodiment will not be described again here to avoid repetition.

[0130] According to a PCB defect detection system of this embodiment, the first acquisition module can establish a YOLOv10 model, and effectively acquire and divide the PCB data set into a training set, a verification set and a test set, which ensures that there is sufficient and diverse data support in the model training process, thereby improving the generalization ability of the model; the first processing module preprocesses the PCB data set, which usually includes operations such as image enhancement, which helps to improve the model's recognition accuracy of PCB defects and reduce the impact of noise on model training; the second processing module performs lightweight processing on the YOLOv10 model, which can significantly reduce the computational complexity and storage requirements of the model, making the model more suitable for running on resource-constrained hardware, and improving the practical applicability of the system; the first improvement module optimizes the space-to-depth convolution SPD_conv module to a DSPD module and configures it in the lightweight YOLOv10 model, which can further improve the detection accuracy and speed of the model. The DSPD module may enhance the model's ability to identify PCB defects through more effective feature extraction and conversion; the second improvement module configures the preset bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model, which helps the model to better integrate multi-scale features and improve the detection ability of PCB defects of different sizes and shapes; the first training module uses the preprocessed PCB dataset to train the improved DSPD_YOLOv10 model to ensure that the model can learn the effective features of PCB defects; the first evaluation module evaluates the training results, which can quantify the performance of the model, including detection accuracy, recall rate and other indicators, and provide data support for further optimization of the model.

[0131] Optionally, an embodiment of the present application further provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, each process of the above-mentioned PCB defect detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, an embodiment of the present application further provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by the processor, each process of the above-mentioned PCB defect detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0132] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0133] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0135] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A PCB defect detection method, characterized in that: The method comprises: Acquire a PCB data set, and perform image preprocessing and defect annotation on the PCB data set; Creating a virtual operating environment, constructing a YOLOv10 model in the virtual operating environment, and performing lightweight processing on the YOLOv10 model; Optimize the spatial-to-depth convolution SPD_conv module into a DSPD module, and configure the DSPD module into the YOLOv10 model to obtain a DSPD_YOLOv10 model; Construct a bidirectional feature pyramid network BiFPN, and configure the bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model; Input the PCB data set into the DSPD_YOLOv10 model for training, and evaluate the training results; The sample to be tested is tested according to the trained DSPD_YOLOv10 model to obtain a test result; The space-to-depth convolution SPD_conv module includes a space-to-depth layer and a non-stridedconvolution layer; the space-to-depth convolution SPD_conv module is optimized to a DSPD module, and the DSPD module is configured in the YOLOv10 model, specifically: The non-strided convolution layer in the space-to-depth convolution SPD_conv module is deleted, and the space-to-depth layer is optimized by adaptive weighting processing and depth sorting to obtain the DSPD module; Keep the step size of the first convolution of the backbone network in the YOLOv10 model unchanged, and configure the DSPD module after the remaining convolution layers; The process of image processing by the DSPD module is as follows: The input feature map is sliced ​​to obtain several feature sub-maps. The specific formula is as follows: ； The feature subgraphs are subjected to adaptive weighting processing and spatial dimension processing to make the spatial dimensions of the feature subgraphs consistent. The specific formula is as follows: ； Then, the feature sub-graphs after adaptive weighting processing and spatial dimension processing are spliced. The specific formula is as follows: ； in, represents the sampling slicing function, X represents the input feature map, s represents the scale, e represents the sampling step size, X s-1，s-1 represents the feature at position (s-1, s-1) in the input feature information, P s-1，s-1 X represents the adaptive weighting and spatial dimension processing s-1，s-1 , , , and Respectively represent X s-1，s-1 , X s-2，s-1 , X s-1，s-2 and X 0，0 The corresponding feature weight value, Indicates the spatial dimension processing of features. It represents the output feature information corresponding to the scale s, and cat is a function that concatenates the feature maps.

2. A PCB defect detection method according to claim 1, characterized in that: The PCB data set is obtained, and image preprocessing and defect annotation are performed on the PCB data set; specifically, as follows: Industrially produced PCB boards are imaged and sorted to obtain a PCB dataset containing defects, including missing holes, mouse bites, open circuits, short circuits, burrs and fake copper; the PCB dataset is expanded by data enhancement; the expanded PCB dataset is subjected to adaptive linear interpolation super-resolution reconstruction processing, and the defects are marked, and finally the PCB dataset is divided into a training set, a test set and a validation set.

3. A PCB defect detection method according to claim 1, characterized in that: The process of lightweight processing the YOLOv10 model is as follows: Replace the ordinary convolution in the YOLOv10 model with a lightweight ghost convolution GhostConv.

4. A PCB defect detection method according to claim 1, characterized in that: Configure the bidirectional feature pyramid network BiFPN to the neck network of the DSPD_YOLOv10 model, specifically: Part of the Concat layer in the neck network of the YOLOv10 model is replaced with the bidirectional feature pyramid network BiFPN.

5. A PCB defect detection method according to claim 1, characterized in that: In the evaluation of training results, the evaluation indicators include recall rate, precision, average precision, average precision mean and frame rate; The recall rate is calculated by the following formula: ； The accuracy is calculated by the following formula: ； The average accuracy is calculated by the following formula: ； The mean average precision is calculated by the following formula: ； The frame rate is calculated by the following formula: ； Among them, Recall represents the recall rate, Precision represents the accuracy, AP represents the average accuracy, mAP represents the mean of the average accuracy, FPS represents the frame rate, and F N Indicates the number of samples predicted as negative samples which are actually positive samples, F P Indicates the number of samples predicted as positive which are actually negative samples, T P represents the number of positive samples predicted to be positive, P(r) represents the function image surrounded by recall and precision, n represents the number of defect categories, i represents the i-th category, F n Represents the number of images to be detected, and T represents the time used to detect all the images to be detected.

6. A PCB defect detection system capable of implementing a PCB defect detection method according to any one of claims 1 to 5, characterized in that: The system comprises: The first acquisition module is used to establish a YOLOv10 model and obtain a PCB data set required by the network model, wherein the PCB data set includes a training set, a validation set, and a test set; A first processing module, configured to preprocess the PCB data set acquired by the first acquisition module to obtain the preprocessed PCB data set; A second processing module, used for performing lightweight processing on the YOLOv10 model established by the first acquisition module; The first improvement module optimizes the space-to-depth convolution SPD_conv module into a DSPD module, and configures the DSPD module into the YOLOv10 model after lightweight processing to obtain a DSPD_YOLOv10 model; The second improvement module configures the preset bidirectional feature pyramid network BiFPN into the neck network of the DSPD_YOLOv10 model; A first training module, training the DSPD_YOLOv10 model improved by the second improvement module through the PCB data set preprocessed by the first processing module; The first evaluation module is used to evaluate the training result of the first training module.

7. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of a PCB defect detection method as described in claims 1-5 are implemented.

8. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of a PCB defect detection method as described in claims 1-5 are implemented.

Images