A flexible circuit board defect detection model training method and defect detection method

Abstract

The application discloses a flexible circuit board defect detection model training method and a defect detection method. The training method of the flexible circuit board defect detection model comprises the following steps: acquiring a flexible circuit board original image, labeling a target defect in the flexible circuit board original image, and obtaining one or more data sets; adopting a mosaic data enhancement method and / or a random clipping data enhancement method to perform data enhancement expansion on the data sets to obtain a sample set; based on a YOLOv4 algorithm, a YOLOv4 structure is constructed and improved to obtain an improved YOLOv4 model; the improved YOLOv4 model is trained by using the sample set to obtain a flexible circuit board defect detection model; and the detection model obtained by the training method is used to detect defects of the flexible circuit board, and the detection is more accurate and faster.

Description

Technical Field

[0001] This invention relates to the field of flexible circuit board defect detection technology, and in particular to a method for training a flexible circuit board defect detection model and a defect detection method. background

[0002] Flexible printed circuit boards (FPCBs) are characterized by their ability to be bent, rolled, and folded freely, meeting various spatial layout requirements. When used in electronic products, they effectively reduce the size and weight of these products, satisfying the demand for high-density, miniaturization, and high reliability. However, during the production of FPCBs, defects such as air bubbles, solder mask in holes, slag, surface contamination, and uneven surface can occur due to process flaws, environmental influences, and improper human operation, resulting in a low yield rate. Therefore, the detection of defects in FPCBs is crucial and a prerequisite for improving the pass rate of electronic products.

[0003] The YOLOv4 algorithm, based on the YOLO (You Only Live Once) object detection architecture, optimizes various aspects such as data processing, backbone network, network training, activation function, and loss function to varying degrees. Applying YOLOv4 to object detection yields good classification and detection accuracy. However, due to the diverse types, random locations, and small defects in flexible circuit board images, traditional object detection methods struggle to accurately detect defects on flexible circuit boards. Even with the YOLOv4 network, it is difficult to accurately locate and classify small defects on the surface of flexible circuit boards.

[0004] The above background information is provided only to assist in understanding the inventive concept and technical solution of this invention. It does not necessarily belong to the prior art of this patent application, nor does it necessarily provide technical teaching. In the absence of clear evidence that the above information was disclosed before the filing date of this patent application, the above background information should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0005] The purpose of this invention is to provide a training method and a defect detection method for a flexible circuit board defect detection model based on YoloV4-Ghost, which can accurately locate and classify small defect targets on the surface of flexible circuit boards, improve the detection accuracy and efficiency of surface defects, and save labor costs.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A training method for a flexible circuit board defect detection model based on YOLOv4-Ghost includes the following steps:

[0008] Obtain the original image of the flexible circuit board, annotate the target defects in the original image of the flexible circuit board, and obtain one or more datasets;

[0009] The dataset is augmented using a mosaic data augmentation method and / or a random cropping data augmentation method to obtain a sample set.

[0010] Based on the YOLOv4 algorithm, a YOLOv4 structure is constructed, and the GhostNet network structure is used as the feature extraction backbone network. The feature pyramid CSPP module is used, and the neck part of the YOLOv4 structure is improved to a SPANet structure to obtain an improved YOLOv4 model.

[0011] The improved YOLOv4 model is trained using the sample set to obtain a flexible circuit board defect detection model.

[0012] Furthermore, based on any one or a combination of the aforementioned technical solutions, the improved YOLOv4 model includes a feature extraction backbone network, a feature pyramid CSPP module, a SPANet path aggregation network module, and a multi-classifier module.

[0013] The GhostNet network structure is configured to generate and output four feature maps A1, A2, A3 and A4;

[0014] The feature maps A1, A2, A3 and A4 are transmitted to the feature pyramid CSPP module to generate and output feature maps C1, C2, C3 and C4;

[0015] The feature maps C1, C2, C3, and C4 are transmitted to the SPANet path aggregation network module to generate valid feature maps.

[0016] Furthermore, following any one or a combination of the aforementioned technical solutions, the processing steps of the SPANet path aggregation network module for the feature maps C1, C2, C3, and C4 are as follows:

[0017] Convolution operations are performed on the four feature maps C1, C2, C3, and C4 respectively to generate C 1_0 C 2_0 C 3_0 C 4_0 ;

[0018] For C 4_0 Perform an upsampling operation to obtain C 4_1 ; will C 3_0 and C 4_1 Perform feature fusion to form C 3_1 ; For C 3_1The effective feature layers undergo multiple convolution operations to form C. 3_2 ; For C 3_2 Perform upsampling and compare the upsampling result with C. 2_0 Perform feature fusion to form C 2_1 ; will C 1_0 and C 2_1 Perform feature fusion to form C 1_1 ; For C 1_1 The effective feature layers undergo multiple convolution operations to form C. 1_2 ; For C 1_2 Perform downsampling and compare the downsampling result with C. 2_1 By splicing, a C is formed. 2_2 ; For C 2_2 Perform downsampling and convolution operations, and then compare the results with C. 3_2 By splicing, a C is formed. 3_3 ; will C 3_3 With C 4_1 By splicing, a C is formed. 4_2 ;

[0019] For C 1_2 Perform convolution operations to generate a valid feature map P1;

[0020] After performing max pooling on the effective feature map P1, it is then compared with C. 2_2 Add elements one by one, then perform convolution to generate an effective feature map P2;

[0021] After performing convolution and max pooling operations on the effective feature map P2, it is then compared with C. 3_2 Add elements one by one and perform convolution on the results to generate an effective feature map P3;

[0022] The effective feature map P3 and C 4_2 Add elements one by one and perform convolution on the results to generate an effective feature map P4.

[0023] Furthermore, following any one or a combination of the aforementioned technical solutions, the steps by which the Feature Pyramid CSPP module operates on feature maps A1, A2, A3, and A4 are as follows:

[0024] The Feature Pyramid CSPP module transmits the feature maps A1, A2, A3, and A4 through pooling layers of 5, 9, and 13, respectively. Then, it performs convolutions with kernel sizes of 5, 7, and 9 to concatenate the six feature maps obtained through pooling and convolution. Finally, it performs a convolution operation with a kernel size of 1 to generate and output the feature maps C1, C2, C3, and C4.

[0025] Furthermore, following any one or a combination of the aforementioned technical solutions, the multi-classifier module includes multiple YOLO Head classifiers, which are used to receive fused features of different sizes.

[0026] Furthermore, following any one or a combination of the aforementioned technical solutions, the operation of data augmentation and expansion of the dataset includes:

[0027] Data augmentation and / or expansion can be achieved by applying one or more of the following methods to the dataset: adding noise, translation, rotation, cropping, affine transformation, increasing grayscale, and blending with a specific background.

[0028] Furthermore, based on any one or a combination of the aforementioned technical solutions, the target defect includes creases, indentations, foreign matter, and oxidation.

[0029] Furthermore, following any one or a combination of the aforementioned technical solutions, the sample set is divided into a training set and a validation set, wherein the number of samples in the training set is greater than the number of samples in the validation set.

[0030] The improved YOLOv4 model is trained using the training set to obtain a flexible circuit board defect detection model.

[0031] The detection capability of the obtained flexible circuit board defect detection model is verified using the validation set, and the performance of the flexible circuit board defect detection model is evaluated.

[0032] Verify and evaluate whether the performance of the flexible circuit board defect detection model meets the requirements. If it does, the current flexible circuit board defect detection model is used for defect detection of flexible circuit boards. Otherwise, it is necessary to fine-tune the network parameters of the model and / or increase the number of samples in the training set to optimize the flexible circuit board defect detection model until a validated flexible circuit board defect detection model is obtained.

[0033] Furthermore, following any one or a combination of the aforementioned technical solutions, the performance evaluation of the flexible circuit board defect detection model includes defect classification evaluation, the defect classification evaluation steps of which are as follows:

[0034] The average accuracy mAP of the flexible circuit board defect detection model is calculated using the following formula:

[0035]

[0036] Among them, AP i Let k be the precision of the i-th defect classification, and k be the number of defect classifications.

[0037] If the average accuracy mAP ≥ 90%, the flexible circuit board defect detection model is considered to be able to accurately detect various defects; otherwise, the flexible circuit board defect detection model is optimized.

[0038] According to another aspect of the present invention, a method for detecting defects in flexible printed circuit boards is provided, comprising the following steps:

[0039] Acquire an image of the flexible circuit board to be inspected;

[0040] The dataset is augmented and expanded using a mosaic data augmentation method and / or a random cropping data augmentation method to obtain a sample set.

[0041] The sample set is input into a pre-trained flexible circuit board defect detection model for detection;

[0042] The flexible circuit board defect detection model outputs detection results, including the location of the target defect, the type of the target defect, and the confidence level of the target defect.

[0043] The flexible circuit board defect detection model is trained through the following steps:

[0044] Obtain the original image of the flexible circuit board, annotate the target defects in the original image of the flexible circuit board, and obtain one or more datasets;

[0045] The dataset is augmented and expanded using mosaic data augmentation and / or random cropping data augmentation methods to obtain a sample set.

[0046] Based on the YOLOv4 algorithm, we construct and improve the YOLOv4 structure to obtain an improved YOLOv4 model;

[0047] The improved YOLOv4 model is trained using the sample set to obtain a flexible circuit board defect detection model.

[0048] According to another aspect of the present invention, another method for training a flexible circuit board defect detection model based on YoloV4-Ghost is provided, comprising the following steps:

[0049] Acquire flexible circuit board images and perform preprocessing, label target defects in the images, form one or more datasets, and divide each dataset into a training set and a validation set, wherein the number of samples in the training set is greater than the number of samples in the validation set;

[0050] Based on the YOLOv4 algorithm, we construct and improve the YOLOv4 structure to obtain an improved YOLOv4 model;

[0051] The training set is input into the improved YOLOv4 model for training to obtain a flexible circuit board defect detection model.

[0052] The validation set is input into the flexible circuit board defect detection model to evaluate its performance. If the model meets the requirements, it is used as the flexible circuit board defect detection model. Otherwise, the parameters of the flexible circuit board defect detection model are optimized and adjusted, and / or the number of training set samples is increased to optimize the model until a validated flexible circuit board defect detection model is obtained.

[0053] Furthermore, following any one or a combination of the aforementioned technical solutions, the number of data in the training set is X, and the number of data in the validation set is Y, where 4Y≤X≤8Y.

[0054] Furthermore, following any one or a combination of the aforementioned technical solutions, the dataset is augmented and expanded using a mosaic data augmentation method and / or a random cropping data augmentation method to obtain a sample set, which is then divided into the training set and the validation set.

[0055] Furthermore, following any one or a combination of the aforementioned technical solutions, the operation of data augmentation and expansion of the dataset includes:

[0056] The dataset can be augmented and / or expanded by performing one or more of the following methods: adding noise, translation, rotation, cropping, affine transformation, increasing grayscale, and blending with a specific background.

[0057] Furthermore, based on any or a combination of the aforementioned technical solutions, the improved YOLOv4 model includes a feature extraction backbone network module, a feature pyramid CSPP module, a SPANet path aggregation network module, and a multi-classifier module;

[0058] The GhostNet network structure is configured to generate and output four feature maps A1, A2, A3 and A4;

[0059] The feature maps A1, A2, A3 and A4 are transmitted to the feature pyramid CSPP module to generate and output feature maps C1, C2, C3 and C4;

[0060] The feature maps C1, C2, C3, and C4 are transmitted to the SPANet path aggregation network module to generate valid feature maps.

[0061] Furthermore, following any one or a combination of the aforementioned technical solutions, the processing steps of the SPANet path aggregation network module for the feature maps C1, C2, C3, and C4 are as follows:

[0062] Convolution operations are performed on the four feature maps C1, C2, C3, and C4 respectively to generate C1_0, C2_0, C3_0, and C4_0;

[0063] Upsampling is performed on C4_0 to obtain C4_1; features from C3_0 and C4_1 are fused to form C3_1; multiple convolution operations are performed on the effective feature layers of C3_1 to form C3_2; C3_2 is upsampled, and its upsampled result is fused with features from C2_0 to form C2_1; features from C1_0 and C2_1 are fused to form C1_1; multiple convolution operations are performed on the effective feature layers of C1_1 to form C1_2; C1_2 is downsampled, and its downsampled result is concatenated with C2_1 to form C2_2; C2_2 is downsampled and convolved, and its result is concatenated with C3_2 to form C3_3; C3_3 is concatenated with C4_1 to form C4_2;

[0064] Perform a convolution operation on C1_2 to generate an effective feature map P1;

[0065] After performing max pooling on the effective feature map P1, it is added element-wise to C2_2, and then convolution is performed to generate the effective feature map P2.

[0066] After performing convolution and max pooling operations on the effective feature map P2, it is added element-wise to C3_2, and then convolution is performed again to generate the effective feature map P3.

[0067] Add P3 and C4_2 element by element, and then perform a convolution operation to generate an effective feature map P4.

[0068] Furthermore, following any one or a combination of the aforementioned technical solutions, the steps by which the Feature Pyramid CSPP module operates on feature maps A1, A2, A3, and A4 are as follows:

[0069] The Feature Pyramid CSPP module transmits the feature maps A1, A2, A3, and A4 through pooling layers of 5, 9, and 13, respectively. Then, it performs convolutions with kernel sizes of 5, 7, and 9 to concatenate the six feature maps obtained through pooling and convolution. Finally, it performs a convolution operation with a kernel size of 1 to generate and output the feature maps C1, C2, C3, and C4.

[0070] Furthermore, following any one or a combination of the aforementioned technical solutions, the multi-classifier module includes multiple YOLO Head classifiers, which are used to receive fused features of different sizes.

[0071] Furthermore, following any one or a combination of the aforementioned technical solutions, the performance evaluation of the flexible circuit board defect detection model includes defect classification evaluation, the defect classification evaluation steps of which are as follows:

[0072] The average accuracy mAP of the flexible circuit board defect detection model is calculated using the following formula:

[0073]

[0074] Among them, AP i Let k be the precision of the i-th defect classification, and k be the number of defect classifications.

[0075] If the average accuracy mAP ≥ 90%, the flexible circuit board defect detection model is considered to be able to accurately detect various defects; otherwise, the flexible circuit board defect detection model is optimized.

[0076] Furthermore, based on any one or a combination of the aforementioned technical solutions, the target defect includes creases, indentations, foreign matter, and oxidation.

[0077] The beneficial effects of the technical solution provided by this invention are as follows:

[0078] a. The flexible circuit board defect detection model can detect multiple target defects, solving the problem that traditional algorithms can only detect specific defects, improving detection accuracy and efficiency, and saving labor costs;

[0079] b. By adopting the GhostNet network structure as the backbone network for feature extraction, the number of network parameters is reduced by nearly one-third compared to the original network, making it suitable for industrial machines with low configuration.

[0080] c. The CSPP network structure effectively avoids image distortion caused by image region cropping and scaling operations, solves the problem of graph-related repetitive feature extraction in convolutional neural networks, and improves detection speed and the ability to identify small target defects;

[0081] d. This invention proposes the SPANet path aggregation network, which expands and aggregates paths to improve performance. Bottom-up paths are enhanced to make lower-level information easier to propagate. Attached Figure Description

[0082] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0083] Figure 1 A schematic diagram of the training process for a flexible circuit board defect detection model provided in an exemplary embodiment of the present invention;

[0084] Figure 2 A YoloV4-Ghost network architecture diagram provided as an exemplary embodiment of the present invention;

[0085] Figure 3 A schematic diagram of a CSPP module for a feature pyramid provided as an exemplary embodiment of the present invention;

[0086] Figure 4 A schematic diagram of a SPANet path aggregation network module provided as an exemplary embodiment of the present invention;

[0087] Figure 5 A schematic diagram of the process of feature maps by the SPANet path aggregation network module provided as an exemplary embodiment of the present invention;

[0088] Figure 6 A flowchart illustrating the verification of a flexible circuit board defect detection model provided as an exemplary embodiment of the present invention;

[0089] Figure 7 This is a flowchart illustrating a method for detecting defects in flexible circuit boards, provided as an exemplary embodiment of the present invention. Detailed Implementation

[0090] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0091] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0092] Due to the diverse types, random locations, and small defects in flexible circuit board (PCB) images, traditional target detection methods struggle to accurately detect PCB defects. Even using the YOLOv4 network, it's difficult to precisely locate and classify small defects on the PCB surface. This invention combines YOLOv4 and GhostNet (a lightweight network module) for PCB defect detection, enabling faster and more accurate detection of target defects on the PCB surface. Based on the YOLOv4 algorithm, this invention constructs and improves the YOLOv4 structure to obtain a model suitable for PCB defect detection. To this end, this invention provides a training method for it, such as... Figure 1 As shown, the training method for the flexible circuit board defect detection model includes the following steps:

[0093] Obtain the original image of the flexible circuit board, annotate the target defects in the original image of the flexible circuit board, and obtain one or more datasets;

[0094] The dataset is augmented using a mosaic data augmentation method and / or a random cropping data augmentation method to obtain a sample set.

[0095] Based on the YOLOv4 algorithm, a YOLOv4 structure is constructed, and the GhostNet network structure is used as the feature extraction backbone network. The feature pyramid CSPP module is used, and the neck part of the YOLOv4 structure is improved to a SPANet structure to obtain an improved YOLOv4 model.

[0096] The improved YOLOv4 model is trained using the sample set to obtain a flexible circuit board defect detection model.

[0097] In one embodiment of the present invention, images of flexible printed circuit boards (FPCBs) in an industrial setting are acquired. Defects in the FPCB images are manually labeled to obtain a dataset. The dataset is then augmented to obtain a sample set. Mosaic data augmentation and / or random cropping data augmentation methods can be used to augment the dataset. In this embodiment, 50% of the dataset is randomly selected for augmentation using the mosaic data augmentation method, and another 50% is randomly selected for augmentation using the random cropping data augmentation method. These two data augmentation methods are combined to apply noise, translation, rotation, cropping, affine transformation, grayscale enhancement, and background fusion to the dataset, achieving data augmentation and / or augmentation to obtain the sample set. Using two data augmentation methods to augment the dataset and obtain the sample set enhances the network's robustness and prevents overfitting.

[0098] After obtaining the sample set, the network configuration is parsed, and the pixel area of ​​the network anchor is set to {12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401}, and the ratio of the anchor_mask is set to [[6,7,8],[3,4,5],[0,1,2]]. After starting the data loading thread, the network training begins. The sample set is input into the improved YOLOv4 model for multiple iterations of training, with N+M iterations. The learning rate for the first N generations is set to 0.0005-0.005, and the learning rate for the last M generations is set to 0.00005-0.0005, where N and M are both not less than 50. In this embodiment, the learning rate for the first N generations is set to 0.001, and the learning rate for the last M generations is set to 0.0001, with N = M = 100. The weights are saved every 10 iterations, and the best and last weights are updated until the number of iterations reaches the maximum value, which can be an empirical value or a set value.

[0099] In one embodiment of the present invention, GhostNet is applied to the YOLOv4 algorithm to propose an improved YOLOv4 model. The improved YOLOv4 model includes a feature extraction backbone network module, a feature pyramid CSPP module, a SPANet path aggregation network module, and a multi-classifier module. See also... Figure 2 The feature extraction backbone network is a lightweight GhostNet network structure. During training, the GhostNet network structure is configured to generate and output four feature maps A1, A2, A3, and A4, each with a size of 13*13. Feature maps A1, A2, A3, and A4 are transmitted to the Feature Pyramid CSPP module to generate and output feature maps C1, C2, C3, and C4; feature maps C1, C2, C3, and C4 are then transmitted to the SPANet path aggregation network module to generate valid feature maps. After training, a flexible circuit board defect detection model is obtained.

[0100] The use of GhostNet as the feature extraction backbone network in this invention significantly reduces the computational and parameter requirements, making it more lightweight for industrial applications. The calculation formulas for computational and parameter requirements are as follows:

[0101] Computational complexity:

[0102]

[0103] Where n is the number of output channels, h is the height of the image, w is the width of the image, c is the number of input channels, k is the kernel size, s is the speedup ratio, and d is the kernel size for each linear operation.

[0104] Number of parameters:

[0105]

[0106] Where n is the number of output channels, c is the number of input channels, k is the kernel size, s is the speedup ratio, and d is the kernel size for each linear operation.

[0107] Refer to Figure 3, which provides a schematic diagram of the CSPP module of the feature pyramid according to an exemplary embodiment of the present invention. Based on the existing SPP module, three convolutional layers are added. The four feature maps A1, A2, A3, and A4 enter the feature pyramid CSPP module, first passing through pooling layers of size 5, 9, and 13, and then through convolutions with kernel sizes of 5, 7, and 9. The six feature maps obtained through pooling and convolution are concatenated, and then a convolution operation with a kernel size of 1 is performed to generate and output feature maps C1, C2, C3, and C4, thereby reducing the number of channels. Using the newly proposed CSPP network structure as the Neck part of this network makes the network more accurate in recognizing small targets and enhances the ability to extract features of small target defects.

[0108] See Figure 4, a schematic diagram of a SPANet path aggregation network module provided in an exemplary embodiment of the present invention; see also Figure 5 The feature maps C1, C2, C3, and C4 are input into the SPANet path aggregation network module. The operation steps performed by this module on the four feature maps C1, C2, C3, and C4 are as follows:

[0109] Perform a 1x1 convolution operation on the four feature maps C1, C2, C3, and C4 to generate C 1_0 C 2_0 C 3_0 C 4_0 ;

[0110] For C 4_0 Perform an upsampling operation to obtain C 4_1 ; will C 3_0 and C 4_1 Feature fusion to form C 3_1 ; For C 3_1 The effective feature layer undergoes five 3x3 convolution operations to form C. 3_2 ; For C 3_2 Perform upsampling and compare the sampling results with C 2_0 Perform feature fusion to form C 2_1 ; will C 1_0 and C 2_1 Perform feature fusion to form C 1_1 ; For C 1_1 The effective feature layer is subjected to five 3x3 convolutions to form C. 1_2 ; For C 1_2Perform downsampling and compare the downsampling result with C. 2_1 By splicing, a C is formed. 2_2 ; For C 2_2 After performing downsampling and convolution operations, the result is compared with C. 3_2 By splicing them together, a C is formed. 3_3 ; will C 3_3 With C 4_1 To form C 4_2 And repeat the stacking N times.

[0111] For C 1_2 Perform a 3x3 convolution operation to generate an effective feature map P1;

[0112] After max pooling the effective feature map P1, and then performing C... 2_2 Add elements one by one and perform a 3*3 convolution operation on the result to generate an effective feature map P2;

[0113] After performing 3x3 convolution and max pooling operations on the effective feature map P2, it is then compared with C. 3_2 Add elements one by one and perform a 3*3 convolution operation on the result to generate an effective feature map P3;

[0114] Connect P3 and C 4_2 The elements are added one by one, and the result is subjected to a 3*3 convolution operation to generate an effective feature map P4, which is then input into the multi-classifier module.

[0115] In this embodiment, the input layer of the SPANet path aggregation network structure adds a top-down path, making it easier for top-level information to propagate. Moreover, the spatial and semantic information is richer than that of the original YOLOv4 model, and the important features of the feature map are more prominent, which can effectively improve the accuracy of detection and recognition.

[0116] In one embodiment of the present invention, the multi-classifier module includes four YOLO Head classifiers and four multi-classifier modules receive fused features of different sizes output by the feature fusion module, including fused features of 13*13, 26*26, 52*52, and 104*104.

[0117] After training, the semantic information of the low-level feature maps P1, P2, P3, and P4 and the high-level feature maps C1, C2, C3, and C4 is combined to create a feature map block from bottom to top and from left to right. When detecting an image, there will be many predicted boxes, and there may be high redundancy among the predicted results. Non-maximum suppression can filter out other predicted boxes that exceed the threshold with the highest confidence score of the same detected object. That is, among multiple prediction results of the same predicted box, only the prediction result with the highest confidence score is taken.

[0118] The training process for the flexible circuit board defect detection model also involves model acceptance and evaluation: see [link to relevant documentation]. Figure 6 The sample set is used to construct a training set and a validation set. Specifically, the sample set is divided into a training set and a validation set according to a certain ratio, and the number of samples in the training set is greater than the number of samples in the validation set, i.e., X>Y, where X is the number of samples in the training set and Y is the number of samples in the validation set. Preferably, 4Y≤X≤8Y. The improved YOLOv4 model is trained iteratively multiple times using the training set to obtain a flexible circuit board defect detection model.

[0119] The detection capability of the obtained flexible circuit board defect detection model is verified using the validation set, and the performance of the flexible circuit board defect detection model is evaluated.

[0120] Verify and evaluate whether the performance of the flexible circuit board defect detection model meets the requirements. If it does, the current flexible circuit board defect detection model is used for defect detection of flexible circuit boards. Otherwise, it is necessary to fine-tune the network parameters of the model (including but not limited to one or more combinations of the number of previous iterations N, the number of subsequent iterations M, the learning rate of the first N generations, the learning rate of the last M generations, the number of channels, the kernel size of the convolution, the speedup ratio, the kernel size of each linear operation, and the number of convolution operations) and / or increase the number of samples in the training set to optimize the flexible circuit board defect detection model until a validated flexible circuit board defect detection model is obtained.

[0121] The performance evaluation of the flexible circuit board defect detection model mainly includes defect classification evaluation, and the defect classification evaluation steps are as follows:

[0122] The average accuracy mAP of the flexible circuit board defect detection model is calculated using the following formula:

[0123]

[0124] Among them, AP i Let be the accuracy of the i-th defect classification, and k be the number of defect classifications.

[0125] If the average accuracy mAP ≥ 90%, the flexible circuit board defect detection model is considered to be able to accurately detect various defects. Otherwise, the flexible circuit board defect detection model needs to be optimized, mainly by increasing the number of training set samples, adjusting the learning rate, and increasing the number of iterations, or a combination thereof.

[0126] In one embodiment of the present invention, the training set and the validation set are divided into a dataset in a ratio of 8.5:1.5 (X:Y = 8.5:1.5). The image data in the training set is used for training the model, and the image data in the validation set is used to verify the detection capability of the model. Through extensive training, the model is optimized by selecting one or more combinations of increasing the number of training set samples, adjusting the learning rate, and increasing the number of iterations based on the validation results.

[0127] In one embodiment of the present invention, a method for detecting defects in flexible circuit boards is provided, such as... Figure 7 As shown, the detection method includes the following steps:

[0128] Images of flexible printed circuit boards (FPCBs) were collected on-site, and target defects in the FPCB images were labeled to obtain a dataset.

[0129] The dataset is augmented and expanded using a mosaic data augmentation method and / or a random cropping data augmentation method to obtain a sample set.

[0130] The sample set is input into the pre-trained flexible circuit board defect detection model;

[0131] The flexible circuit board defect detection model outputs detection results, including the location of the target defect, the type of the target defect, and the confidence level of the target defect.

[0132] The flexible circuit board defect detection model is trained through the following steps:

[0133] Obtain the original image of the flexible circuit board, annotate the target defects in the original image of the flexible circuit board, and obtain one or more datasets;

[0134] The dataset is augmented and expanded using mosaic data augmentation and / or random cropping data augmentation methods to obtain a sample set.

[0135] Based on the YOLOv4 algorithm, we construct and improve the YOLOv4 structure to obtain an improved YOLOv4 model;

[0136] Using the aforementioned sample set, the improved YOLOv4 model is trained to obtain a flexible circuit board defect detection model.

[0137] The obtained flexible circuit board defect detection model was verified and evaluated, and the model was selectively optimized based on the verification and evaluation results.

[0138] The implementation of this flexible circuit board defect detection method involves using the flexible circuit board defect detection model trained by the above training method to detect the input image of the flexible circuit board to be detected, and then outputting the defect detection results, which include: the location of the target defect, the type of the target defect, and the confidence level of the target defect.

[0139] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0140] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A training method for a flexible circuit board defect detection model based on YOLOv4-Ghost, characterized in that, Includes the following steps: Obtain the original image of the flexible circuit board, annotate the target defects in the original image of the flexible circuit board, and obtain one or more datasets; The dataset is augmented using a mosaic data augmentation method and / or a random cropping data augmentation method to obtain a sample set. Based on the YOLOv4 algorithm, a YOLOv4 structure is constructed, and the GhostNet network structure is used as the feature extraction backbone network. The Feature Pyramid CSPP module is employed, and the neck part of the YOLOv4 structure is improved to a SPANet structure, resulting in an improved YOLOv4 model. The improved YOLOv4 model includes a feature extraction backbone network, a Feature Pyramid CSPP module, a SPANet path aggregation network module, and a multi-classifier module. The GhostNet network structure is configured to generate and output four feature maps A1, A2, A3, and A4. A1, A2, A3, and A4 are transmitted to the Feature Pyramid CSPP module, generating and outputting feature maps C1, C2, C3, and C4. The steps of the Feature Pyramid CSPP module in operating on the feature maps A1, A2, A3, and A4 are as follows: The Feature Pyramid CSPP module transmits the feature maps A1, A2, A3, and A4 through pooling layers of 5, 9, and 13 respectively, and convolves them with convolutions of kernel sizes of 5, 7, and 9. The six feature maps obtained through pooling and convolution are then concatenated, and a convolution operation with a kernel size of 1 is performed again to generate and output the feature maps C1, C2, C3, and C4 accordingly. The feature maps C1, C2, C3, and C4 are transmitted to the SPANet path aggregation network module to generate valid feature maps; the valid feature maps are then transmitted to the multi-classifier module; wherein, the processing steps of the SPANet path aggregation network module for the feature maps C1, C2, C3, and C4 are as follows: Convolution operations are performed on the four feature maps C1, C2, C3, and C4 respectively to generate C 1_0 C 2_0 C 3_0 C 4_0 ; For C 4_0 Perform an upsampling operation to obtain C 4_1 ; will C 3_0 and C 4_1 Perform feature fusion to form C 3_1 ; For C 3_1 The effective feature layers undergo multiple convolution operations to form C. 3_2 ; For C 3_2 Perform upsampling and compare the upsampling result with C. 2_0 Perform feature fusion to form C 2_1 ; will C 1_0 and C 2_1 Perform feature fusion to form C 1_1 ; For C 1_1 The effective feature layers undergo multiple convolution operations to form C. 1_2 ; For C 1_2 Perform downsampling and compare the downsampling result with C. 2_1 By splicing, a C is formed. 2_2 ; For C 2_2 Perform downsampling and convolution operations, and then compare the results with C. 3_2 By splicing, a C is formed. 3_3 ; will C 3_3 With C 4_1 By splicing, a C is formed. 4_2 ; For C 1_2 Perform convolution operations to generate an effective feature map P1; then perform max pooling on the effective feature map P1 and then concatenate it with C. 2_2 Element-wise addition is performed, followed by convolution to generate an effective feature map P2; the effective feature map P2 is then subjected to convolution and max pooling operations and then convolved with C. 3_2 Element-wise addition is performed, and the result is convolved to generate an effective feature map P3; the effective feature map P3 is then compared with C. 4_2 Add elements one by one and perform convolution on the results to generate an effective feature map P4; Using the aforementioned sample set, the improved YOLOv4 model is trained to obtain a flexible circuit board defect detection model. After training, the semantic information of feature maps P1, P2, P3 and P4 and feature maps C1, C2, C3 and C4 are combined to create a feature map block from bottom to top and from left to right. When detecting multiple bounding boxes in an image, only the prediction with the highest confidence score is taken from the multiple predictions for the same bounding box.

2. The training method for the flexible circuit board defect detection model based on YoloV4-Ghost according to claim 1, characterized in that, The multi-classifier module includes multiple YOLO Head classifiers, which are used to receive fused features of different sizes.

3. The training method for the flexible circuit board defect detection model based on YOLOV4-Ghost according to claim 1, characterized in that, The data augmentation and expansion operations performed on the dataset include: Data augmentation and / or expansion can be achieved by applying one or more of the following methods to the dataset: adding noise, translation, rotation, cropping, affine transformation, increasing grayscale, and blending with a specific background.

4. The training method for the flexible circuit board defect detection model based on YOLOv4-Ghost according to claim 1, characterized in that, The target defects include creases, indentations, foreign matter, and oxidation.

5. The training method for a flexible circuit board defect detection model based on YoloV4-Ghost according to any one of claims 1-4, characterized in that, The sample set is divided into a training set and a validation set, wherein the number of samples in the training set is greater than the number of samples in the validation set; The improved YOLOv4 model is trained using the training set to obtain a flexible circuit board defect detection model. The detection capability of the obtained flexible circuit board defect detection model is verified using the validation set, and the performance of the flexible circuit board defect detection model is evaluated. Verify and evaluate whether the performance of the flexible circuit board defect detection model meets the requirements. If it does, the current flexible circuit board defect detection model is used for defect detection of flexible circuit boards. Otherwise, it is necessary to fine-tune the network parameters of the model and / or increase the number of samples in the training set to optimize the flexible circuit board defect detection model until a validated flexible circuit board defect detection model is obtained.

6. The training method for the flexible circuit board defect detection model based on YoloV4-Ghost according to claim 5, characterized in that, Validating and evaluating the performance of the flexible circuit board defect detection model includes defect classification evaluation, the steps of which are as follows: The average accuracy mAP of the flexible circuit board defect detection model is calculated using the following formula: Among them, AP i Let k be the precision of the i-th defect classification, and k be the number of defect classifications. If the average accuracy mAP ≥ 90%, the flexible circuit board defect detection model is considered to be able to accurately detect various defects; otherwise, the flexible circuit board defect detection model is optimized.

7. A method for detecting defects in flexible printed circuit boards, characterized in that, Includes the following steps: Acquire an image of the flexible circuit board to be inspected; The dataset was augmented and expanded using mosaic data augmentation and / or random cropping data augmentation methods to obtain a sample set. The sample set is input into a pre-trained flexible circuit board defect detection model for detection; The flexible circuit board defect detection model outputs detection results, including the location of the target defect, the type of the target defect, and the confidence level of the target defect. The flexible circuit board defect detection model is trained using the training method described in any one of claims 1 to 6.

Images