Flexible circuit board defect detection method and device, medium and equipment

Abstract

The invention discloses a flexible circuit board defect detection method and device, a medium and equipment, and relates to the technical field of defect detection. In order to solve the problem of inaccurate detection of small defects of a flexible circuit board, an attention mechanism module is added to the output end of a backbone network SPPF module in an original YOLOv8 network, and a convolution module of a c2f module connected with a minimum-size detection head in a neck network is replaced with a deformable convolution module. Therefore, the detection accuracy of the tiny defects of the flexible circuit board is improved. The attention mechanism enhances semantic information related to defects and transmits accurate and prominent defect semantic information to the neck network; the neck network carries out fusion based on the enhanced defect semantic information so as to transmit more comprehensive scale information of the tiny defects to the head network; the deformable convolution module assists the anchor frame generated by the minimum-size detection head to be matched with the position and the scale of the tiny defect more accurately, and a more accurate detection result is obtained.

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology, and in particular to a method, device, medium and equipment for detecting defects in a flexible circuit board. Background Art

[0002] FPC (Flexible Printed Circuit) is an electronic component that has been widely used in recent years. It was originally developed to meet the needs of aerospace rocket technology. It uses flexible materials such as polyester film or polyimide as the substrate, and embeds the circuit design in it to form a printed circuit with extremely high reliability and excellent flexibility. This design allows various precision components to be flexibly arranged in a very small and limited space, so it can be flexibly used in various environments. Compared with traditional rigid circuit boards, flexible circuit boards can be bent and folded at will, are light in weight, small in size, have good heat dissipation performance, and are easy to install, breaking the traditional interconnection technology. In recent years, flexible circuit boards have become standard in various devices such as mobile phones, computers, cameras, LED displays, and play an important role in automotive electronics, medical equipment, aerospace and other fields. With the improvement of people's living standards, the demand for electronic products is also growing steadily. Therefore, the demand for flexible circuit boards in China's electronics manufacturing industry continues to rise, and the total output value of flexible circuit boards is also increasing year by year. Flexible circuit boards have many advantages such as being light, thin, and resistant to bending. However, with the increase in the wiring density of circuit boards, the line size has reached the micron level. However, the limitations of manufacturing equipment and processes have led to an increase in the types and number of micro defects in flexible circuit boards, and the production qualification rate is difficult to guarantee. If defective flexible circuit boards are used to produce subsequent products, product performance will be seriously affected, and ultimately lead to non-human failures in the products. The losses incurred will be borne by the manufacturer, which will not only bring huge economic losses to the manufacturer, but also affect the manufacturer's reputation. In order to avoid the frequent occurrence of such situations, manufacturers of flexible circuit boards will conduct defect detection and eliminate the products produced.

[0003] At present, deep learning networks can be used for defect detection of flexible circuit boards. The introduction of this technology has greatly improved the detection efficiency and automation level. However, the defects that may appear in the manufacturing process of flexible circuit boards are often small in size and complex in shape, which poses significant challenges to defect detection. Since these tiny defects are difficult to identify visually and may appear similar to normal textures in images, the existing deep learning detection technology has certain limitations in accuracy, and the accuracy of the detection results still needs to be further improved. Summary of the invention

[0004] Based on this, in order to solve the technical problems in the prior art, the present invention provides a flexible circuit board defect detection method, device, medium and equipment.

[0005] The present invention provides a method for detecting defects in a flexible circuit board, comprising: An attention mechanism module is added to the output end of the backbone network SPPF module in the original YOLOv8 network to form an improved backbone network; wherein the attention mechanism module is an MHSA attention mechanism module or an SA attention mechanism module, and the MHSA attention mechanism captures information of different dimensions of the feature map output by the SPPF module through multiple attention heads; the SA attention mechanism divides the feature map output by the SPPF module into multiple groups according to the channel dimension, and calculates the attention weight after shuffling the channels of each group; and the attention weight is used to weight the features in each group.

[0006] The convolution module of the c2f module connected to the minimum size detection head of the head network in the neck network in the original YOLOv8 network is replaced with a deformable convolution module to form an improved neck network; wherein the deformable convolution module is implemented by the DCNv2 deformable convolution, which replaces conv3 to conv5 in the c2f module with deformable convolutions and adds weights to each sampling point in the deformable convolution to reduce the probability of processing areas in the feature map that are not related to defects.

[0007] Construct an improved YOLOv8 network that includes an improved backbone network, an improved neck network, and a head network in the original YOLOv8 network.

[0008] An electron microscope is used to collect defect images of flexible circuit boards, including exposed copper, foreign matter, wrinkles, poor adhesive tape, bruises, poor gold surface, tears, scratches, coverage and poor development. Different types of flexible circuit board defect images are enhanced by Gaussian blur, random rotation, random scaling and brightness change. The flexible circuit board defect images before and after enhancement are integrated and labeled to obtain a data set. The data set is used to train the improved YOLOv8 network to obtain a flexible circuit board defect detection model.

[0009] The image of the flexible circuit board to be detected is input into the flexible circuit board defect detection model. The defects in the image of the flexible circuit board to be detected are identified at different scales by improving the backbone network, and the first defect feature map, the second defect feature map and the third defect feature map with decreasing scales and increasing defect semantic information are obtained, wherein the third defect feature map is obtained by weighting the defect semantic information in the feature map output by the SPPF module through the attention mechanism module; feature fusion is performed by improving the neck network. During the feature fusion process, the second defect feature map and the first defect feature map are based on the defect semantic information provided by the third defect feature map, and the neighborhood space size information of the defect semantic information is gradually enhanced to obtain the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map with decreasing sizes; wherein the third defect enhancement feature map is obtained by performing a deformable convolution operation on the feature map of the input c2f module through the deformable convolution module; the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map are respectively detected by the detection head with decreasing scales in the head network, and the defect position and defect type in the image of the flexible circuit board to be detected are output.

[0010] The present invention provides a flexible circuit board defect detection device, comprising: A model building module is used to add an attention mechanism module at the output end of the SPPF module of the backbone network in the original YOLOv8 network to form an improved backbone network; replace the convolution module of the c2f module in the neck network of the original YOLOv8 network connected to the minimum size detection head of the head network with a deformable convolution module to form an improved neck network; and build an improved YOLOv8 network including the improved backbone network, the improved neck network, and the head network in the original YOLOv8 network; The model training module is used to collect defect images of flexible circuit boards to construct a data set, and use the data set to train the improved YOLOv8 network to obtain a flexible circuit board defect detection model; The defect detection module is used to input the image of a flexible circuit board to be detected into a flexible circuit board defect detection model, and identify defects in the image of the flexible circuit board to be detected at different scales by improving the backbone network, so as to obtain a first defect feature map, a second defect feature map and a third defect feature map with successively decreasing scales and successively increasing defect semantic information, wherein the third defect feature map is obtained by weighting the defect semantic information in the feature map output by the SPPF module through the attention mechanism module; feature fusion is performed by improving the neck network, and during the feature fusion process, the second defect feature map and the first defect feature map are based on the defect semantic information provided by the third defect feature map, and the neighborhood space size information of the defect semantic information is gradually enhanced to obtain the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map with successively decreasing sizes; wherein the third defect enhancement feature map is obtained by performing a deformable convolution operation on the feature map of the input c2f module through the deformable convolution module; the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map are respectively detected by the detection head with successively decreasing scales in the head network, and the defect position and defect type in the image of the flexible circuit board to be detected are output.

[0011] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned flexible circuit board defect detection method when executing the program.

[0012] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects: This invention proposes an improved solution based on the basic architecture of YOLOv8, which improves the accuracy of detecting small defects in flexible circuit boards while maintaining the fast performance of YOLOv8. Specifically:

[0013] The present invention adds an attention mechanism at the output end of the backbone network, performs weighted processing on the feature map with the strongest semantic information, so as to enhance the semantic information related to the defect therein, thereby transmitting more accurate and prominent defect semantic information to the neck network; the neck network fuses the multi-scale features transmitted from the backbone network, starting from the feature with the smallest scale and the strongest semantic information output by the backbone network, and gradually fuses information to the features of larger scales, and the enhanced defect semantic features provide more accurate spatial perception for feature fusion, so that other scale features transmitted from the backbone network can be targetedly fused according to the enhanced defect semantic information, ensuring the complete transmission and representation of defect features of different scales, thereby transmitting more comprehensive defect scale information including tiny defects to the head network; the convolution layer in the c2f module connected to the minimum size detection head of the head network is replaced with a deformable convolution module, so that the minimum size detection head can adaptively adjust the anchor frame generation strategy of the minimum size detection head according to the size of different defects, so that the generated anchor frame more accurately matches the position and scale of the tiny defect, improves the flexibility and accuracy under complex defect shapes, especially when processing extremely small flexible circuit board defects, it can generate more accurate anchor frames and obtain more accurate detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0015] Figure 1 A schematic diagram of a process flow of a flexible circuit board defect detection method provided by the present invention; Figure 2 A schematic diagram of the defect detection principle provided by the present invention, Figure 2 (a) is the original defect image of the flexible circuit board. Figure 2 (b) is the segmented defect image of the flexible circuit board. Figure 2 (c) in the figure is a number of defect detection anchor boxes. Figure 2 (d) in the figure is the defect detection result; Figure 3 A schematic diagram of the improved YOLOv8 micro-defect detection network structure provided by the present invention; Figure 4 This is a schematic diagram of the flexible circuit board image acquisition device provided by the present invention, Figure 4 (a) is an electron microscope for collecting micro defects. Figure 4 (b) in the figure is the ImageView micro-defect collection interface; Figure 5 A schematic diagram of a defect sample provided by the present invention, Figure 5(a) is a copper exposure defect. Figure 5 (b) in the figure is a foreign body defect. Figure 5 (c) is a wrinkle defect. Figure 5 (d) is a defect of adhesive tape. Figure 5 (e) is a crush defect. Figure 5 (f) is a defect in the gold surface. Figure 5 (g) is a tear defect. Figure 5 (h) is a scratch defect. Figure 5 (i) in the equation is a coverage defect. Figure 5 (j) in the figure is a poor development defect; Figure 6 This is a schematic diagram of defect sample enhancement provided by the present invention. Figure 6 (a1) is the original crushed sample. Figure 6 (a2) is the right Figure 6 The result obtained by Gaussian blurring (a1) in Figure 6 (b1) is the original scratched sample. Figure 6 (b2) is the right Figure 6 The result obtained by randomly rotating (b1) in Figure 6 (c1) in is the original torn sample, Figure 6 (c2) is the right Figure 6 The result obtained by randomly scaling (c1) in Figure 6 (d1) in is the original overburden sample, Figure 6 (d2) is the right Figure 6 The result obtained by changing the brightness of (d1) in the figure; Figure 7 A defect marking schematic diagram provided by the present invention, Figure 7 (a) in the figure is a rectangular box defect annotation. Figure 7 (b) in the figure is a polygonal rectangle annotation; Figure 8 This is a schematic diagram of the convergence of training accuracy during the training process provided by the present invention. Figure 8 (a) in the figure is the convergence of accuracy. Figure 8 (b) in the figure is the convergence of recall rate. Figure 8 (c) in the figure shows the convergence of average detection accuracy; Fig. 9 This is a schematic diagram of the change trend of the loss function during the training process provided by the present invention. Fig. 9 (a) in the figure is the changing trend of the bounding box loss function. Fig. 9 (b) in the figure is the trend of the classification loss function. Fig. 9 (c) in the figure is the changing trend of the feature loss function; Fig.10The robustness test results and comparisons of different models provided by the present invention are shown. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in combination with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Automatic optical inspection has obvious advantages in the detection of micro-defects in flexible circuit boards. It can greatly improve the detection speed and reduce labor costs. Therefore, automatic optical inspection technology has become a hot area for domestic and foreign scholars and manufacturers to study. Its key technologies include image processing and computer vision. Many companies abroad have obtained mature technologies and experience. However, due to various reasons, domestic manufacturers have to pay high fees to purchase foreign equipment, and this type of equipment is mostly used for traditional circuit boards, which cannot meet the needs of automatic detection of micro-defects in flexible circuit boards. Due to the technology monopoly, domestic manufacturers cannot use existing equipment for research and development. Due to the late start of domestically developed equipment, its efficiency and accuracy cannot be compared with mature foreign products. At present, most flexible circuit board manufacturers still use algorithms based on traditional digital image features for defect detection. Although such algorithms can effectively replace part of manual visual inspection, in the detection of multiple types of defects, a single model is often not competent for all tasks, and multiple models are required to identify different types of defects. With the application of deep learning technology in various fields, many scholars have begun to apply deep learning technology to product defect detection.

[0018] Existing defect detection technologies can be divided into two categories: defect detection based on traditional digital image features and defect detection based on deep learning. Among them, defect detection methods based on traditional digital image features can be divided into three categories: reference method, non-reference method and hybrid method. The reference method is a method for detecting tiny defects in flexible circuit boards by comparing the image to be tested with the reference image. This method relies on comparing the feature points of the two to determine whether there is a defect. The first method uses the grayscale values ​​of the overlapping areas of the pixels of the image to be tested and the reference image to create a two-dimensional graph, and measures their similarity by calculating the smaller eigenvalue of the covariance matrix of the data points, thereby identifying abnormal points for local defect detection. The second method is a defect matching algorithm based on the QQ graph, which plots the quantiles of the image to be tested and the reference image respectively, and then uses the p-value of the chi-square test to evaluate the similarity between the images. Although the reference method is relatively simple and has fewer steps, it is slow and has low accuracy. The non-reference method directly operates on the image to be tested and analyzes the results to locate the defect position and identify the defect type. The algorithm proposed by Ma identifies defects on the circuit board by threshold segmentation after plotting the grayscale histogram of the image to be tested. Yuan Zhenfang used a self-designed CCD camera to capture images, and then realized defect recognition based on the difference between defects and background imaging in different color channels of the RGB color space, and used a mask algorithm to extract the area of ​​interest and extract the contour. Then a multi-information tone detection algorithm was used for defect recognition, and finally a clustering algorithm was used for classification. The hybrid method combines the advantages of the reference method and the non-reference method to achieve defect detection. Based on the idea of ​​template matching, Yao Shuang studied the local range of flexible circuit board defect detection methods, and proposed a detection method based on skeleton endpoints and node features based on line features. Finally, the two methods were combined to achieve rapid identification of micro defects in flexible circuit boards and improve the detection speed. In general, defect detection technology based on traditional digital image features has the disadvantages of few defect detection types, low detection efficiency and success rate, and it is difficult to achieve the expected detection efficiency.

[0019] Object detection has been developing over the past few decades, from traditional manual features to the rise of deep learning, which has continuously promoted the development of this field. Deep learning models have made important progress in defect detection tasks. For example, the ResNet model has made a major breakthrough in image classification tasks, providing a powerful network structure for subsequent defect detection. With the emergence of single-stage detectors YOLO and SSD, the efficiency of industrial defect detection has been significantly improved. The SSD model achieves end-to-end object detection through a single feedforward network, which is suitable for real-time defect detection scenarios in industry. The instance segmentation model Mask R-CNN introduces an instance segmentation branch to generate accurate segmentation masks for each defect instance, improving the accuracy of defect detection. With the popularization of deep learning technology, some scholars have begun to introduce it into the field of circuit board micro-defect detection to achieve accurate and efficient detection of defects. For example, Wei et al. directly used deep convolutional neural networks to automatically detect a variety of different printed circuit board defects. Zhang et al. proposed a multi-task CNN model based on deep neural networks. By modifying the loss function, the model training of each label data is defined as a binary classification task to solve the problem of different defect classification in printed circuit board detection. The training of deep learning models usually requires a large amount of data support to enhance their generalization ability and better complete the task in practical applications. However, the actual frequency of micro-defects in flexible circuit boards is low, resulting in relatively few samples of such defects and prone to overfitting problems. To solve this problem, Xu Xiyan proposed a method that combines empirical mode decomposition and sparse representation algorithms. Based on reducing image noise, conditional generative models are used to expand samples of a small number of samples, thereby improving the accuracy of the model in small sample detection. In addition, in order to solve the problem of difficult samples, Xu introduced a mining algorithm and combined it with a cross-optimization strategy to make the network pay more attention to difficult samples during training, thereby improving the performance of the model. In general, the above-mentioned deep learning-based defect detection technology is mainly proposed for the task of printed circuit board defect detection. Due to the different characteristics of micro-defects in printed circuit boards and flexible circuit boards, and the large differences in the materials of the two, the characteristics of micro-defects in flexible circuit boards are very complex. There is a lack of data sets for the detection of multiple types of micro-defects. Therefore, the above-mentioned algorithm performs poorly in the detection of micro-defects in flexible circuit boards.

[0020] Based on this, the present invention introduces deep learning technology into the detection of micro-defects in flexible circuit boards, and uses its advantages of fast speed and high precision to train the collected flexible circuit board micro-defect sample data, and finally obtains a detection model. Since there are many types of micro-defects in flexible circuit boards, and some defects are too small to detect, the model training process faces many challenges. In view of the above situation, the present invention will improve the existing defect deep learning model and study a variety of micro-defect detection methods specifically for flexible circuit boards. The present invention aims to apply deep learning technology to the detection of various types of micro-defects in flexible circuit boards, and perform detection based on the improved YOLOv8 algorithm to solve the problem that the existing technology cannot be directly applied to this field, and finally realize a flexible circuit board micro-defect detection algorithm suitable for actual detection.

[0021] Example 1 Figure 1 The process of the flexible circuit board defect detection method of this embodiment is shown. Figure 1 The method is described in detail and specifically comprises the following steps:

[0022] S1: Add an attention mechanism module at the output end of the SPPF module of the backbone network in the original YOLOv8 network to form an improved backbone network; replace the convolution module of the c2f module in the neck network of the original YOLOv8 network connected to the minimum size detection head of the head network with a deformable convolution module to form an improved neck network; construct an improved YOLOv8 network including the improved backbone network, the improved neck network and the head network in the original YOLOv8 network.

[0023] The target detection algorithm of two-dimensional images usually includes two main tasks. The first is the positioning of the target, which aims to identify the position of the detectable target in the image and delimit the target with a bounding box or other form. The second is the classification of the target, that is, to determine which category the identified detectable target belongs to. By comparing the target in the image with the predefined categories, the model can assign the target to the corresponding category. These two tasks are usually the core components of the target detection algorithm, and the two work together to enable the algorithm to accurately detect and identify targets in the image. The present invention intends to adopt a single-stage target detection algorithm, also known as a target detection algorithm based on regression methods, because this type of algorithm can combine the two tasks of target positioning and target category distinction in the same convolutional neural network, and has the advantage of fast target detection speed. Among them, the most commonly used single-stage target detection algorithm is YOLO, and its basic idea is mainly to detect single defects by dividing cells, such as Figure 2As shown in the figure, however, each divided cell is responsible for detecting one object, which may lead to missed detection problems, especially when two objects of different categories appear in the same cell, which may reduce the detection accuracy of small-scale objects. The micro-defects of flexible circuit boards are small in scale, and there may be multiple defects in one detection frame, which may lead to missed detection. Therefore, the present invention replaces the convolution of the original model with deformable convolution and adds an attention mechanism to improve the model's detection performance for small-scale defects. The improved YOLOv8 micro-defect detection network structure is constructed as shown in the figure. Figure 3 Specifically including:

[0024] S101: Introducing deformable convolution.

[0025] The present invention uses the mature YOLOv8 as the basic backbone network and makes improvements, because YOLOv8 provides a brand new SOTA model. After inputting the micro-defect image data to be detected, the backbone network extracts features from the image data. Compared with YOLOv5, the first convolution kernel of the backbone network is changed from 6×6 to 3×3, and the C3 module is also changed to a C2f module, so more skip-layer connections and additional segmentation operations are added. At the same time, the SPPF module is compared with SPP, and the simple parallel maximum pooling method is changed to a serial + parallel method; then the neck network extracts multiple feature maps for target detection, and the effective feature maps are passed to the detection head for prediction results. In terms of loss function calculation, YOLOv8 uses VFL Loss to calculate the classification loss, and uses DFL Loss and CIOU Loss to calculate the regression loss, which helps to improve the accuracy and stability of the model in target detection tasks. The workflow of deformable convolution is as follows: define a standard convolution kernel with a fixed size and step size; generate a set of offsets before performing deformable convolution; for each sampling point in the standard convolution, offset it according to the generated offset; use bilinear interpolation to calculate the eigenvalues ​​of the offset sampling points that are not at integer positions; use the offset sampling points and the eigenvalues ​​obtained by bilinear interpolation to perform weighted summation according to the rules of standard convolution to obtain a pixel value on the output feature map to obtain the output feature map.

[0026] In order to capture more details in the image, the present invention introduces deformable convolution to replace the terminal convolution layer in the YOLOv8 structure. Deformable convolution is also a convolution operation, but it is not limited to the traditional fixed-shape convolution kernel, but can flexibly adjust the shape in space according to specific needs. This flexibility enables deformable convolution to capture the features in the image more accurately. Compared with the traditional fixed-shape convolution kernel, deformable convolution is more suitable for processing feature areas of various shapes and sizes. Therefore, the introduction of this convolution layer enables the detection network to adapt to different target shapes and postures more flexibly, improving the accuracy and robustness of the model to the target. The present invention adopts DCNv2 (Deformable ConvNets v2) deformable convolution, and adds weights for each sampling point. These weights learn how to emphasize or weaken the contribution of each sampling point to the convolution result, and design the features after ROI pooling of DCNv2 to be closer to the features in R-CNN, avoiding the problem that DCNv1 (Deformable ConvNets v1) may spend time detecting irrelevant areas, thereby significantly improving the performance of the model in micro-defect detection tasks.

[0027] S102: Introduce attention mechanism.

[0028] The current attention mechanism can be mainly divided into two categories, one is the spatial attention mechanism and the other is the channel attention mechanism. The simultaneous use of the two attention mechanisms will increase the amount of calculation. The SA (Shuffle Attention) module combines group convolution, spatial attention mechanism, channel attention mechanism and ShuffleNetv2, which can effectively reflect the above two attention mechanisms. Therefore, the present invention selects the SA module to add an attention mechanism to the YOLOv8 model to enhance the feature representation ability of the network. Considering that MHSA (Multi-Header Self-Attention) can flexibly adapt to different tasks and scenarios by adjusting the number and weight of attention, it has certain versatility and adjustability. Therefore, the present invention introduces the multi-head self-attention mechanism MHSA to compare with SA. The multi-head self-attention mechanism expands Self-Attention into multiple attention heads and captures more sequence dependencies through parallel calculation, thereby improving the representation ability and generalization ability of the model; at the same time, because the multi-head self-attention mechanism can simultaneously consider the dependencies between different positions, it may have better performance when processing long sequences or sequences with long-distance dependencies.

[0029] The workflow of the MHSA attention mechanism is as follows: divide the feature map output by the SPPF module into multiple subsets, each subset corresponds to a head; for each head, calculate the self-attention score based on the query Query, key Key and value Value respectively; apply the softmax function to the attention score of each head, and use the converted attention score as the weight to perform weighted summation on the value Value; concatenate or average the output feature maps from different heads to obtain the final output feature map.

[0030] The workflow of the SA attention mechanism is as follows: divide the channel dimension of the feature map into multiple groups; shuffle the channels within each group to increase information exchange between channels; calculate the attention weight for each shuffled channel group; use the calculated attention weight to weight the features within each channel group; and recombine the weighted channel groups into a complete feature map.

[0031] By fusing the attention mechanism and the deformable convolution module, that is, pre-locating the spatial position information of micro-defects and then using the deformable convolution module to capture more details in the defect image, the attention mechanism is mainly used to accurately locate the spatial position information of micro-defects, while the deformable convolution is used to process various types of defect shapes. The two are linked to each other to achieve high-precision multi-type micro-defect detection.

[0032] S2: Collect defect images of flexible circuit boards to build a dataset, use the dataset to train the improved YOLOv8 network, and obtain a flexible circuit board defect detection model.

[0033] S201: Collect image data.

[0034] The dataset is the basis of flexible circuit board micro-defect detection based on deep learning, and plays a vital role in the training and performance improvement of machine learning algorithms. In view of the current problems such as the small number of flexible circuit board micro-defect samples and the difficulty in constructing multi-type datasets, the present invention first performs microscopic image data acquisition, data enhancement, and data labeling, and builds the development environment required for the experiment.

[0035] In this embodiment, an Ausmicro A0-3M180-3018 electron microscope is used to collect micro defects of flexible circuit boards. Figure 4As shown in (a), the device consists of four parts, namely industrial camera, optical lens, workbench and LED light source. There are two LED light sources, which are high-angle light source and low-angle light source. The device is used to collect images of defective parts from samples of flexible circuit boards of different styles and functions provided by the manufacturer of flexible circuit boards. Then, the ImageView software is used to drive and control the electron microscope to collect images, and the clarity of the captured images is adjusted by white balance, etc. For example, the shooting interface of the "poor development" defect is shown in the figure. Figure 4 as shown in (b).

[0036] There are many types of defects in flexible circuit boards. In this constructed data set, 10 types of defects with a frequency of more than 10 in the samples were selected, involving 10 types of micro defects such as "exposed copper, foreign matter, wrinkles, poor adhesive tape, pressure damage, poor gold surface, tearing, scratches, covering, and poor development". This is to improve the detection accuracy and speed of the model for common defects and meet the expected detection standards in the industry. The types of micro defects of flexible circuit boards and the number of specific defect samples in the constructed data set are shown in Tables 1 and Figure 5 shown.

[0037] Table 1. Defect sample category table Since there are few samples of micro-defects in flexible circuit boards, positive sample data is introduced, that is, images of flexible circuit board samples without defects. Defective samples are used as negative samples, and data enhancement technology is used to enhance several types of micro-defect negative samples to expand the data set. The positive-negative sample ratio is 10:1. At the same time, the introduction of positive samples can better help the model learn the data distribution under normal circumstances, thereby improving the generalization ability of the model. Defective data sets tend to tend to defective samples, which may cause the model to be more inclined to predict that the samples are defective. At the same time, in actual applications, samples without defects are often more common than samples with defects. Therefore, the introduction of positive samples can make the model better adapt to real scenarios and improve robustness and practicality.

[0038] S202: Data enhancement.

[0039] As shown in Table 1, there are too few data images of micro-defect samples of flexible circuit boards. If these data are used for model training, the effect will inevitably not meet expectations and overfitting is likely to occur. Therefore, data enhancement technology is used to expand the existing data set. Data enhancement can increase the diversity of samples and improve the robustness of the model. By randomly changing the samples, the model's dependence on certain attributes is reduced, thereby enhancing the generalization ability of the model. The image data of defective samples is processed in four ways: Gaussian blur, random rotation, random scaling, and brightness change. Figure 6As shown in the figure, (1) is the original sample, and (2) is the processed sample, where each defect sample is subjected to 1:4 data enhancement. After data enhancement, the 294 flexible circuit board defect image data are expanded to 1176, and the data set contains a total of 2940 positive samples and 1176 negative samples, thus ensuring the robustness and generalization performance of model training.

[0040] S203: Data mark.

[0041] The negative samples in the data set are labeled using LabelMe. When labeling, the defect features are clustered in a small range and labeled using a rectangular box, such as Figure 7 As shown in (a) in the figure, for data with wide defect distribution, polygons are used for fine annotation, such as Figure 7 As shown in (b), by labeling with LabelMe software, the defect instance information on each defect image can be obtained. The above information is stored in a text file in json format. By parsing the json file with commands, the annotated mask image corresponding to the original image can be obtained. The json file generated by the original image after labeling cannot be directly used in the subsequent deep learning algorithm, so the json file is converted into a txt file supported by YOLOv8.

[0042] S204: Setting model evaluation indicators.

[0043] The experimental evaluation method uses the detection speed and detection accuracy in target detection as indicators to evaluate the detection effect of the algorithm on micro-defects of flexible circuit boards. The average detection time of each image data is used as the indicator of detection speed, and the average detection accuracy (mAP) is used as the indicator of detection accuracy. In the target detection algorithm, each category corresponds to an accuracy rate (precision) and a recall rate (recall), as shown in formula (1):

[0044] (1); Among them, TP is the true positive rate, FP is the false positive rate, FN is the false negative rate, and mAP is the mean of the areas under the precision-recall curves of multiple categories, which can directly reflect the positioning and classification of micro-defects of flexible circuit boards by the target detection algorithm.

[0045] S205: Training experiment environment setting.

[0046] The expanded and labeled dataset is divided into training set, validation set and test set in a ratio of 8:1:1. The dataset and related deep learning model system development environment information are shown in Table 2.

[0047] Table 2. Experimental environment information table S206: Model training results.

[0048] (1) Loss function

[0049] After adding the SA attention mechanism and deformable convolution DCNv2 to the original YOLOv8 algorithm, the convergence of the accuracy, recall and average detection accuracy mAP obtained by the training model is shown in the figure below: Figure 8 As shown in the figure, it can be seen that the improved model can effectively converge on all three indicators, and the average detection accuracy mAP is close to 90%, thus ensuring the overall detection accuracy. At the same time, we also tested the convergence of three loss functions on the test set. The experimental results are as follows: Fig. 9 As shown in the figure, good convergence effects can be achieved using bounding box loss, classification loss, and feature loss functions, indicating that the dataset constructed by this model has good generalization performance.

[0050] (2) Ablation experiment

[0051] In order to verify the effectiveness of the improved YOLOv8 for detecting various types of micro-defects, the present invention compares another single-stage target detection model SSD, the original YOLOv8 model, the YOLOv8 model with deformable convolution, the YOLOv8 model with SA attention mechanism and deformable convolution, and the YOLOv8 model with MHSA attention mechanism and deformable convolution on 10 flexible circuit board micro-defect datasets. The ablation experiment results are shown in Table 3. After adding SA attention mechanism and DCNv2 deformable convolution on the basis of the original YOLOv8, the average detection accuracy mAP of the model reaches 91.2%. The improved method improves the accuracy by 2.2% on the basis of the original YOLOv8 model, indicating that after adding the attention mechanism and deformable convolution, the model's detection ability for some small-size defects of flexible circuit boards is improved. Although it takes 0.052s more time, the overall detection time is still around 0.1s, which can meet the needs of real-time detection in factories.

[0052] Table 3. Ablation experiment of improved YOLOv8 flexible circuit board micro-defect detection model (3) Robustness analysis.

[0053] In the actual image acquisition process, due to the limited automatic zoom capability of the microscope, blurry images are inevitable. In order to verify whether the improved method can have good robustness in the real environment, Gaussian noise is introduced to perform anti-noise detection on the above five models. The final detection results are as follows: Fig.10As shown in the figure, the horizontal axis is the intensity of Gaussian noise, and the vertical axis corresponds to the average detection accuracy of the model. The results show that the YOLOv8 model with the addition of SA attention mechanism and DCNv2 deformable convolution corresponds to Fig.10 The broken line in the figure has better robustness under different noise conditions than the other four models, indicating that the improved method of the present invention has strong anti-interference ability, and the accuracy decreases relatively little when interfered, which can better complete the task of efficient detection of various types of micro defects in flexible circuit boards.

[0054] S3: Input the image of the flexible circuit board to be detected into the flexible circuit board defect detection model, and identify the defects in the image of the flexible circuit board to be detected at different scales by improving the backbone network, and obtain the first defect feature map, the second defect feature map and the third defect feature map with decreasing scales and increasing defect semantic information, wherein the third defect feature map is obtained by weighting the defect semantic information in the feature map output by the SPPF module through the attention mechanism module; perform feature fusion by improving the neck network, and in the feature fusion process, the second defect feature map and the first defect feature map gradually enhance the neighborhood space size information of the defect semantic information based on the defect semantic information provided by the third defect feature map, and obtain the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map with decreasing scales; wherein the third defect enhancement feature map is obtained by performing a deformable convolution operation on the feature map of the input c2f module through the deformable convolution module; perform defect detection on the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map respectively through the detection head with decreasing scales in the head network, and output the defect position and defect type in the image of the flexible circuit board to be detected.

[0055] The present invention aims to apply deep learning technology to flexible circuit board defect detection, and perform detection based on the improved YOLOv8 algorithm to solve the problem that the existing technology cannot be directly applied to this field, and finally realize a flexible circuit board micro-defect detection algorithm suitable for actual detection. In summary, the purposes of the present invention include:

[0056] (1) To address the problems of insufficient micro-defect samples in flexible circuit boards and difficulty in constructing data sets, an electron microscope was used to collect images of 10 types of micro-defects, including "exposed copper, foreign matter, wrinkles, poor adhesive tape, bruises, poor gold surface, tears, scratches, coverage, and poor development". The data set was expanded to 4116 microscopic images based on four enhancement methods, including Gaussian blur, random rotation, random scaling, and brightness change, thus solving the problem of insufficient defect samples.

[0057] (2) In view of the problems of low detection efficiency and few identification types in the current micro-defect detection algorithm for flexible circuit boards, an improved YOLOv8 detection method for micro-defects in flexible circuit boards is proposed. In the improved algorithm, the original convolution in YOLOv8 is replaced by deformable convolution DCNv2. After using DCNv2, the accuracy of the model reaches 89.3%, which is 0.3% higher than the original convolution. Then, the SA attention mechanism and MHSA attention mechanism are introduced. The detection accuracy of the model is improved from 89% to 91.2%, and the average detection time is 0.105s, which can meet the requirements of real-time and efficient detection of micro-defects in flexible circuit boards. The detection accuracy of the model is improved from 89% to 91.2%, and the average detection time is 0.105s, which can meet the requirements of real-time and efficient detection of micro-defects in flexible circuit boards.

[0058] The present invention constructs a dataset of micro-defect samples of flexible circuit boards, and uses the improved YOLOv8 network to train and test a variety of micro-defect datasets of flexible circuit boards. The results are as follows: As the latest version of the YOLO algorithm, YOLOv8 is widely recognized for its accuracy and speed in various detection tasks, but it still has shortcomings in the detection of small-scale defects. Based on YOLOv8, this study introduces two attention mechanisms, SA and MHSA, and replaces the convolution of YOLOv8 with deformable convolution. After adding the SA attention mechanism and deformable convolution, the accuracy of the model is improved from 89% to 91.2%, and the average detection speed is 0.105s; after introducing the MHSA attention mechanism and deformable convolution, the accuracy of the model is improved from 89% to 89.6%, and the average detection speed is 0.093s. The results show that the model trained by the YOLOv8 algorithm after introducing the SA attention mechanism and deformable convolution has better performance and stronger generalization ability, and is more suitable for the task of micro-defect detection of various types of flexible circuit boards.

[0059] In summary, the objectives of the present invention include: (1) In order to solve the problems of insufficient micro-defect samples of flexible circuit boards and difficulty in constructing data sets, an electron microscope was used to collect 10 types of micro-defect images, including "copper exposure, foreign matter, wrinkles, poor adhesive tape, bruises, poor gold surface, tears, scratches, coverage, and poor development", and based on four enhancement methods, including Gaussian blur, random rotation, random scaling, and brightness change, the data set was expanded to 4116 microscopic images, solving the problem of insufficient defect samples. (2) In view of the problems of low detection efficiency and few identification types in the current flexible circuit board micro-defect detection algorithm, an improved YOLOv8 detection method for flexible circuit board micro-defects is proposed. In the improved algorithm, the original convolution in YOLOv8 is replaced by deformable convolution DCNv2. After using DCNv2, the accuracy of the model reaches 89.3%, which is 0.3% higher than the original convolution. Then, the SA attention mechanism and MHSA attention mechanism are introduced, and the detection accuracy of the model is improved from 89% to 91.2%. The average detection time is 0.105s, which can meet the requirements of real-time and efficient detection of micro-defects in flexible circuit boards.

[0060] The main innovations of the present invention are: (1) A dataset of micro-defect samples of various types of flexible circuit boards was constructed. Using an Ausmicro A0-3M180-3018 electron microscope, 294 images of 10 micro-defect images (negative samples) and 2940 non-defect images (positive samples) were collected. The defect images were annotated using LabelMe, and the dataset was expanded to 4116 images based on four enhancement methods: Gaussian blur, random rotation, random scaling, and brightness change. (2) A flexible circuit board defect detection method based on improved YOLOv8 was proposed. Two attention mechanisms, ShuffleAttention (SA) attention mechanism and Multi-header Self-attention (MHSA) attention mechanism, were introduced into the algorithm. The detection accuracy of the model was improved from 89% to 91.2%, and the average detection time was 0.105s, which can meet the requirements of efficient detection of micro-defects in flexible circuit boards.

[0061] The above is a flexible circuit board defect detection method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding flexible circuit board defect detection device, including: A model building module is used to add an attention mechanism module at the output end of the SPPF of the backbone network in the original YOLOv8 network; and replace the convolution module in the original YOLOv8 network with a deformable convolution module; and build an improved YOLOv8 network including the backbone network, the neck network, and the head network.

[0062] The model training module is used to collect flexible circuit board defect images to build a data set, and use the data set to train the improved YOLOv8 network to obtain a flexible circuit board defect detection model.

[0063] The detection module is used to input the flexible circuit board image to be detected into the flexible circuit board defect detection model, identify defects in the flexible circuit board image to be detected at different scales by improving the deformable convolution module in the backbone network, and weight the defects in the flexible circuit board image to be detected by the attention mechanism module to obtain an initial flexible circuit board defect feature map; perform feature fusion on the initial flexible circuit board defect feature map through the neck network to obtain a flexible circuit board defect enhanced feature map; detect the flexible circuit board defect enhanced feature map through the head network to obtain the flexible circuit board position in the flexible circuit board image to be detected.

[0064] The specific definition of the flexible circuit board defect detection device can be found in the definition of the flexible circuit board defect detection method above, which will not be repeated here. Each module in the above-mentioned flexible circuit board defect detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0065] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A flexible circuit board defect detection method is provided.

[0066] The present invention also provides a computer device structure. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 A flexible circuit board defect detection method is provided.

[0067] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0068] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A method for detecting defects in a flexible circuit board, characterized in that: include: An attention mechanism module is added to the output end of the SPPF module of the backbone network in the original YOLOv8 network to form an improved backbone network; the convolution module of the c2f module in the neck network of the original YOLOv8 network connected to the minimum size detection head of the head network is replaced with a deformable convolution module to form an improved neck network; an improved YOLOv8 network including the improved backbone network, the improved neck network and the head network in the original YOLOv8 network is constructed; Collect defect images of flexible circuit boards to build a data set, use the data set to train the improved YOLOv8 network, and obtain a flexible circuit board defect detection model; The image of the flexible circuit board to be detected is input into the flexible circuit board defect detection model. The defects in the image of the flexible circuit board to be detected are identified at different scales by improving the backbone network, and the first defect feature map, the second defect feature map and the third defect feature map with decreasing scales and increasing defect semantic information are obtained, wherein the third defect feature map is obtained by weighting the defect semantic information in the feature map output by the SPPF module through the attention mechanism module; feature fusion is performed by improving the neck network. During the feature fusion process, the second defect feature map and the first defect feature map are based on the defect semantic information provided by the third defect feature map, and the neighborhood space size information of the defect semantic information is gradually enhanced to obtain the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map with decreasing sizes; wherein the third defect enhancement feature map is obtained by performing a deformable convolution operation on the feature map of the input c2f module through the deformable convolution module; the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map are respectively detected by the detection head with decreasing scales in the head network, and the defect position and defect type in the image of the flexible circuit board to be detected are output.

2. The flexible circuit board defect detection method according to claim 1, characterized in that: The attention mechanism module is an MHSA attention mechanism module, which extracts defect semantic information in the feature map output by the SPPF module from different dimensions through multiple attention heads and performs weighting.

3. The method for detecting defects in a flexible circuit board according to claim 1, wherein: The attention mechanism module is an SA attention mechanism module, which divides the feature map output by the SPPF module into multiple groups according to the channel dimension, and calculates the attention weight after shuffling the channels of each group; The features within each group are weighted using attention weights.

4. The method for detecting defects in a flexible circuit board according to claim 1, wherein: The deformable convolution module is a DCNv2 deformable convolution module, which replaces conv3 to conv5 in the c2f module with deformable convolutions and adds weights to each sampling point in the deformable convolution.

5. The method for detecting defects in a flexible circuit board according to claim 1, wherein: The collecting of defect images of the flexible circuit board to construct a data set specifically includes: Use electron microscope to collect images of different types of defects in flexible circuit boards; Different types of flexible circuit board defect images are enhanced by Gaussian blur, random rotation, random scaling and brightness change; The defective images of the flexible circuit board before and after the enhancement process are integrated and annotated to obtain a data set.

6. The method for detecting defects in a flexible circuit board according to claim 5, characterized in that: The different types of flexible circuit board defect images include exposed copper, foreign matter, wrinkles, poor adhesive tape, pressure damage, poor gold surface, tearing, scratches, coverage and poor development.

7. A flexible circuit board defect detection device, characterized in that: include: A model building module is used to add an attention mechanism module at the output end of the SPPF module of the backbone network in the original YOLOv8 network to form an improved backbone network; replace the convolution module of the c2f module in the neck network of the original YOLOv8 network connected to the minimum size detection head of the head network with a deformable convolution module to form an improved neck network; and build an improved YOLOv8 network including the improved backbone network, the improved neck network, and the head network in the original YOLOv8 network; The model training module is used to collect defect images of flexible circuit boards to construct a data set, and use the data set to train the improved YOLOv8 network to obtain a flexible circuit board defect detection model; The defect detection module is used to input the image of a flexible circuit board to be detected into a flexible circuit board defect detection model, and identify defects in the image of the flexible circuit board to be detected at different scales by improving the backbone network, so as to obtain a first defect feature map, a second defect feature map and a third defect feature map with successively decreasing scales and successively increasing defect semantic information, wherein the third defect feature map is obtained by weighting the defect semantic information in the feature map output by the SPPF module through the attention mechanism module; feature fusion is performed by improving the neck network, and during the feature fusion process, the second defect feature map and the first defect feature map are based on the defect semantic information provided by the third defect feature map, and the neighborhood space size information of the defect semantic information is gradually enhanced to obtain the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map with successively decreasing sizes; wherein the third defect enhancement feature map is obtained by performing a deformable convolution operation on the feature map of the input c2f module through the deformable convolution module; the first defect enhancement feature map, the second defect enhancement feature map and the third defect enhancement feature map are respectively detected by the detection head with successively decreasing scales in the head network, and the defect position and defect type in the image of the flexible circuit board to be detected are output.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 6 is implemented.

Images