A flexible circuit board defect detection method based on skeleton generation and fusion configuration

Abstract

The present invention discloses a method for defect detection of flexible circuit boards based on skeleton generation and fusion configuration. First, a real defect-free template image of a flexible circuit board is collected; then, a virtual defect-free template image is formed by parsing and converting the Gerber file of the flexible circuit board production file. Then, surface defects of the flexible circuit board with different structural features and sizes are detected through skeleton generation and defect configuration. Then, the defect-free template image is subjected to offline color enhancement and then a defect data image fusion configuration is performed to obtain a defect data image. Finally, a data set is constructed in combination with the defect information, and a deep neural network model is trained to perform defect detection on the flexible circuit board. Compared with existing methods, the present invention circumvents the problems of few defect samples and long acquisition cycle in deep learning automatic optical detection of flexible circuit boards, is easy to operate, and has high efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to a circuit board defect detection method in the field of flexible circuit boards, and in particular to a flexible circuit board defect detection method based on skeleton generation and fusion configuration. Background Art

[0002] Nowadays, flexible printed circuits (FPCs) meet the demand for electronic products to develop in the direction of high density, lightweight, miniaturization, and high reliability, and are widely used in various industries, such as aerospace, military, consumer electronics, and many other fields. Due to the complex production process of FPCs, the types and forms of defects are diverse. For the problem of FPC defect detection, there are currently two main methods: manual visual inspection and automatic optical inspection. Automatic optical inspection collects surface images of the FPC and performs defect detection through image processing and computer graphics detection algorithms. Compared with manual visual inspection, automatic optical inspection has high efficiency, high precision, and low cost, and is divided into two categories: methods based on traditional digital image processing and methods based on deep learning.

[0003] Manual visual inspection is a popular method among domestic manufacturers. Because FPC defects are often as small as 0.1mm, workers must inspect the FPC surface with the aid of magnifying glasses and microscopes. For very small FPCs with dense circuitry, workers may even need to use a microscope. This narrow field of view requires frequent focusing and movement to inspect all FPC surfaces, requiring high concentration and easily leading to missed defects and over-inspections. Therefore, manual visual inspection is costly, inefficient, and lacks accuracy.

[0004] Traditional digital image processing-based inspection methods are the mainstream in automated optical inspection. They primarily use template matching and feature analysis to categorize different types of defects. However, FPC defects exhibit complex characteristics, making them difficult to classify and manually design. Even defects caused by the same cause can vary significantly in scale and morphology. Traditional methods are not adaptable to the inspection of FPC surface defects, which exhibit large variations in scale, a wide variety of defect types, and complex texture structures.

[0005] Deep learning-based detection methods use deep convolutional neural networks to autonomously learn defect features from datasets, and then train deep learning models for detection. They perform well against the complex features of FPC defects and can achieve automated AI detection. However, in actual production scenarios, production lines periodically change product models based on customer requirements. Different products have different defect features and background textures. Models trained on older FPC defect datasets often have poor generalization capabilities, and detection of newer FPC images cannot meet industry-required accuracy. Furthermore, current factory yields often reach 98%-99%, and the slow accumulation of new defect data leads to a severe shortage of training data and poor model training results. Summary of the Invention

[0006] In response to the above-mentioned problems existing in the existing FPC defect detection, the present invention proposes a flexible circuit board defect detection method based on skeleton generation and fusion configuration. Different from the existing deep learning method, the present invention uses virtual defect simulation, defect data generation and other data enhancement methods to amplify FPC defect data, and simulates real defects by rendering the generated virtual data through digital image processing methods to construct virtual defect samples; and forms a defect image construction data set through image fusion method, avoiding the problems of data scarcity and slow accumulation in deep learning detection, and improving the accuracy and versatility of the FPC defect detection method.

[0007] The technical solution adopted by the present invention comprises the following steps:

[0008] Step 1: Collect a real defect-free template image of the flexible circuit board;

[0009] Step 2: Generate a virtual defect-free template image of the flexible circuit board according to the production file of the flexible circuit board;

[0010] Step 3: Construct multiple types of surface defect images of the flexible circuit board and obtain the defect feature set;

[0011] Step 4: Perform image enhancement on the real defect-free template image and the virtual defect-free template image respectively, thereby obtaining a set of defect-free template images of the circuit board;

[0012] Step 5: After fusing the defect-free template image set of the circuit board and the surface defect images in the defect feature set, a target detection image set is obtained;

[0013] Step 6: Obtain information about various defects in the target detection image by finding the minimum enclosing contour method and form a training data set information. The target detection data set is formed by the target detection image set, data set information and corresponding surface defect images. The target detection data set is input into the deep convolutional neural network for training to obtain a defect detection model, and the defect detection model is used to perform defect detection on the flexible circuit board.

[0014] The step 2 is specifically as follows:

[0015] First, the production file of the flexible circuit board is parsed and converted to obtain the part type information of the flexible circuit board; then, the part type information of the flexible circuit board is matched and parsed to obtain the circuit board pattern matching data; then, the circuit board pattern matching data is rendered and pattern synthesized to obtain a virtual defect-free template image of the flexible circuit board.

[0016] In the second step, after rendering and pattern synthesis of the circuit board pattern matching data, a virtual defect-free template image of the flexible circuit board is obtained, specifically:

[0017] The circuit board pattern matching data is rendered according to the process effects of different process layers and the materials used, with reference to the surface effects of real products, to obtain the rendered circuit board pattern data. The rendered circuit board pattern data is then superimposed according to the upper and lower covering relationship of different process layers to obtain a virtual defect-free template image of the flexible circuit board.

[0018] The step three is specifically as follows:

[0019] Firstly, different skeleton generation methods are used to generate linear skeletons and regional skeletons respectively. Linear defects, curved defects and geometric defects are constructed based on the linear skeleton, and irregular regional defects are constructed based on the regional skeleton. Then, morphological processing is performed on linear defects, curved defects, geometric defects and irregular regional defects respectively to obtain various defect features and use them to synthesize defect images, thereby forming a defect feature set for flexible circuit boards.

[0020] In step 3, for linear defects, first, randomly generate the direction angle θ and determine the basic length l base , and then determine the starting point of the linear defect (x s ,y s ) and the end point (x e ,y e ), the formula is as follows:

[0021]

[0022] Among them, random[] represents the function of generating random numbers, normal() represents the normal distribution function, and x s ,y s Indicates the x and y axis coordinate values ​​of the starting point of the linear defect, x e ,y e Indicates the x- and y-axis coordinates of the end point of the linear defect; l is the length of the linear defect;

[0023] According to the starting point of the linear defect (x s ,y s ) and the end point (x e ,y e ) Determine the coordinates of the remaining n-2 structural points using the following formula:

[0024]

[0025] Where i = 1, 2, ..., n, n is the total number of structure points, sort(·) represents the sorting function, t1, t2, ..., t n-2 are the 1st, 2nd, ..., n-2th structural points, p i is the i-th structural point t i The position coefficient, x i ,y i Represents the i-th structural point t i The x, y axis coordinate values.

[0026] For curve defects, the curve shape is determined according to the linear skeleton, and the smooth curve control points corresponding to the structure points arranged in sequence are generated through the curve parameter equation, and then all the structure points on the curve are generated. Finally, a closed or open curve is drawn. The structure point A i The generation formula of the forward and backward control points is as follows:

[0027]

[0028] Where i=2,...,n-2, α is the proportional coefficient, ||||2 represents the second norm, A i-1 Indicates structural point A i The forward structure point, A i+1 Indicates structural point A i The backward structure point, P i，1 is the structural point P i The forward control point, P i，2 is the structural point P i The backward control point;

[0029] For the nth structural point A n , when the generated curve is a closed curve, the structural point A n is the forward structure point of structure point A1, structure point A1 is the forward structure point of structure point A n The backward structure point, structure point A n The above formula is satisfied between the structure point A1;

[0030] When the generated curve is an open curve, the structure point A1 has only backward control points, and the structure point A n There is only a forward control point, and the formula is as follows:

[0031]

[0032] Finally, a series of discrete points are interpolated between two structural points using the discrete position coefficient t. The discrete points are connected to obtain the corresponding unit pixel areas, and the curve-shaped defects are formed by each unit pixel area.

[0033] For geometric defects, the defect shape is determined according to the linear skeleton. Each structural point is used as the center point of the geometric figure. Each structural point randomly selects a geometric figure to construct a point block feature. The geometric areas of the point block features generated by two adjacent structural points must intersect or be tangent. Finally, the point block features corresponding to each structural point constitute a geometric defect.

[0034] For irregular regional defects, the shape is determined according to the regional skeleton, and the smooth curve control points corresponding to the structural points arranged in sequence are generated through the curve parameter equation, and then all the structural points on the curve are generated. Finally, a closed curve is drawn to form a closed area. After filling the closed area, the irregular regional defect is obtained.

[0035] In the step 4, for the image enhancement of the real defect-free template image, the color of the real defect-free template image is enhanced in two color spaces respectively to obtain corresponding template images after multiple image enhancements.

[0036] In step 5, for each circuit board defect-free template image in the circuit board defect-free template image set, the corresponding target detection image specifically performs the following steps:

[0037] S1: Perform image segmentation on the defect-free template image of the circuit board to obtain multiple segmented template images;

[0038] S2: Determine the corresponding defect area to be placed in each segmented image, perform dilation and separation operations on each defect area to be placed, and obtain separated dilated defect areas; blur the texture of the separated dilated defect areas through Gaussian filtering and then fuse them with the corresponding segmented image through weighted superposition to obtain defect-free images with blurred color difference in the local area around the defect;

[0039] S3: blurring the edges of each surface defect image in the defect feature set by using a Gaussian filtering method to obtain a blurred edge surface defect image;

[0040] S4: After fusing the defect-free image with blurred color in the local area around the defect and at least one type of surface defect image with blurred edges through weighted superposition, a corresponding target detection image is obtained.

[0041] Compared with the existing flexible circuit board defect detection method, the present invention has the following beneficial effects:

[0042] The present invention proposes a set of flexible circuit board defect detection methods based on the actual production process and defect characteristics of flexible circuit boards. Compared with manual visual inspection and defect detection methods based on traditional digital image processing, the present invention is more adaptable to the characteristics of flexible circuit board defects with diverse types and forms and large size differences, and has higher efficiency and higher accuracy. General defect detection methods based on deep learning have high requirements for data sets and require a large amount of defect data for training. The present invention forms a data set through skeleton generation and fusion configuration, avoiding the problem that conventional deep learning methods need to accumulate a large amount of valid data when used and are not suitable for frequent product replacement in flexible circuit board production and the problem of slow data accumulation with high yield. Therefore, compared with existing flexible circuit board defect detection methods, the present invention has higher efficiency and better versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the present invention for realizing flexible circuit board defect detection based on skeleton generation and fusion configuration;

[0044] Figure 2 1 is a planar schematic diagram of a detection platform for capturing images to be detected in an embodiment;

[0045] Figure 3 It is a flow chart of the present invention for realizing the formation of a defect-free template image from a Gerber file;

[0046] Figure 4 It is a schematic diagram of the present invention for realizing skeleton generation and defect configuration construction of virtual defects;

[0047] Figure 5 is a flow chart of the present invention for realizing color enhancement and fusion configuration;

[0048] Figure 6 is a schematic diagram of a defect-free template image described in the present invention;

[0049] Figure 7 It is a schematic diagram of a defect image formed by skeleton generation and fusion configuration in the present invention;

[0050] Figure 8 This is an embodiment of the method proposed in the present invention applied to deep learning network defect detection.

[0051] In the picture: 1. Workbench, 2. CCD camera, 3. Objective lens, 4. Focusing handwheel. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. In the detailed description below, many specific details are provided to facilitate a comprehensive understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be implemented without some of these specific details. The following description of the embodiments is intended only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0053] like Figure 1 As shown, the present invention includes the following steps:

[0054] Step 1: Collect a real defect-free template image of the flexible circuit board;

[0055] In specific implementation, Figure 2 As shown, the flexible circuit board is placed on the workbench 1, and the CCD camera 2 equipped with a microscope is perpendicular to the workbench 1 and set directly opposite the object to be inspected. The CCD camera 2 is connected to the computer, and the magnification of the objective lens 3 is adjusted. The focus distance is adjusted by the focusing hand wheel 4 until the focus is clear. In this embodiment, the electronic magnification is set to 47 times, and the area to be inspected of the flexible circuit board is photographed. There is no defect in the current area to be inspected, and the CCD camera 2 stores the captured image in the computer as a true defect-free template image.

[0056] Step 2: Generate a virtual defect-free template image of the flexible circuit board according to the production file of the flexible circuit board;

[0057] Step 2 is as follows:

[0058] like Figure 3 As shown, first, the flexible circuit board production file (i.e., Gerber file) is parsed and converted to obtain the flexible circuit board's part information. In this embodiment, the flexible circuit board Gerber file format is RS-274X and contains a D-code file. Next, the flexible circuit board's part information is parsed and matched to obtain circuit board pattern matching data. Specifically, the data matching and parsing process involves extracting quality parameters, drawing commands, and control commands from each data block within the part information to obtain entities such as points, lines, ellipses, and arcs in process layers such as pads, traces, silk screens, and surface mounts on the flexible circuit board, thereby obtaining circuit board pattern matching data. In this embodiment, data matching and parsing of the file, including function codes and coordinate values, is performed. By matching function codes such as G-codes, D-codes, and M-codes in each line of data, drawing instructions are obtained, such as controlling switches and selecting aperture size, thereby obtaining drawing objects such as points, lines, arcs, and ellipses. The circuit board pattern matching data is then rendered and pattern synthesized to obtain a virtual defect-free template image of the flexible circuit board.

[0059] In step 2, after rendering and pattern synthesis of the circuit board pattern matching data, a virtual defect-free template image of the flexible circuit board is obtained, specifically:

[0060] The circuit board pattern matching data is rendered based on the process effects of different process layers and the materials used, with reference to the surface effects of the actual product, to obtain the rendered circuit board pattern data. The rendered circuit board pattern data is then overlaid based on the upper and lower coverage relationship of the different process layers (i.e., the order in which the process layers are blocked and overlaid), to obtain a virtual defect-free template image of the flexible circuit board. In this embodiment, the flexible circuit board pattern is overlaid in the order of the top silk screen layer, main pads, solder paste layer, solder mask layer, routing layer, the inner middle layer, the bottom routing layer, solder mask layer, solder paste layer, main pads, surface mount layer, and silk screen layer.

[0061] Step 3: Construct multiple types of surface defect images of the flexible circuit board and obtain the defect feature set;

[0062] Step three is as follows:

[0063] First, different skeleton generation methods are used to generate linear skeletons and regional skeletons respectively. Linear defects, curved defects, and geometric defects are constructed based on the linear skeleton, and irregular regional defects are constructed based on the regional skeleton. Then, after morphological processing of linear defects, curved defects, geometric defects, and irregular regional defects, various defect features are obtained and used to synthesize defect images, thereby forming a defect feature set for the flexible circuit board. The size of each defect is determined by the size of the skeleton, the number of points on the skeleton, and the basic length of the defect generation. The basic length is related to the resolution of the image captured by the industrial CCD camera in step 1 and the electronic magnification of the image capture. In this embodiment, the basic length of each defect is l base The value of and the number of structural points n are determined according to specific needs, and the actual length of the defect is set between 0.3mm and 10mm, such as Figure 4 As shown in (a)-(d).

[0064] In step 3, the structural points of the linear skeleton satisfy the following formula. The position of each structural point is described in the image coordinate system in the form of rectangular coordinates. In the specific implementation, the first structural point is used as the starting point of the defect, and the coordinates are marked as (x0, y0):

[0065]

[0066] Among them, x i ,y i is the x-axis and y-axis coordinate value of the structural point i in the linear skeleton, θ iis the direction angle of the structural point i, Δθ is the change angle randomly distributed within a certain range, Δδ is the mutation angle with probability, r is the basic step size, ε is the offset index, and r·ε is the step size of the skeleton change.

[0067] The structural points of the regional skeleton are determined in polar coordinates with the defect starting point as the origin O. All structural points on the skeleton are distributed near the circumference of a circle with the defect starting point as the center and the base length as the radius. If the skeleton has n structural points, then there are n angles between these structural points and the origin O, and the sum is 2π. The structural points of the regional skeleton satisfy the following formula:

[0068]

[0069] Among them, α i is a random angle, which is a randomly generated angle between 0 and 2π, θ i is the polar angle of the i-th structural point after sorting all the generated random angles from small to large, ρ i is the polar radius of the i-th structural point, which is randomly generated by a Gaussian distribution function with mathematical expectation u and standard deviation σ to determine the fluctuation of the polar radius.

[0070] For linear defects, first, randomly generate the direction angle θ and determine the basic length l base , and then determine the starting point of the linear defect (x s ,y s ) and the end point (x e ,y e ), the formula is as follows:

[0071]

[0072] Among them, random[] represents the function of generating random numbers, normal() represents the normal distribution function, and x s ,y s Indicates the x and y axis coordinate values ​​of the starting point of the linear defect, x e ,y e Indicates the x- and y-axis coordinates of the end point of the linear defect; l is the length of the linear defect;

[0073] According to the starting point of the linear defect (x s ,y s ) and the end point (x e ,y e ) Determine the coordinates of the remaining n-2 structural points using the following formula:

[0074]

[0075] Where i = 1, 2, ..., n, n is the total number of structure points, sort() represents the sorting function, t1, t2, ..., t n-2 are the 1st, 2nd, ..., n-2th structural points, p i is the i-th structural point t i The position coefficient, x i ,y i Represents the i-th structural point t i The x, y axis coordinate values.

[0076] Finally, the unit pixel area where the line segment formed by two adjacent structural points in the linear defect is located is morphologically expanded to obtain a linear defect image with random length and different local width. n-2 ) and the starting point and the end point form a total of n-1 line segments.

[0077] For curve defects, the curve shape is determined according to the linear skeleton, and the smooth curve control points corresponding to the sequentially arranged structural points are generated through the curve parameter equation, and then all the structural points on the curve are generated, and finally a closed or open curve is drawn; the curve is composed of n structural points, and the structural point A i The generation formula of the forward and backward control points is as follows:

[0078]

[0079] Where i=2,...,n-1, α is the proportional coefficient, ||||2 represents the second norm, A i-1 Indicates structural point A i The forward structure point, A i+1 Indicates structural point A i The backward structure point, P i，1 is the structural point P i The forward control point, P i，2 is the structural point P i The backward control point;

[0080] For the nth structural point A n , when the generated curve is a closed curve, the structural point A n is the forward structural point of structural point A1, structural point A1 is structural point A n The backward structure point, structure point A n The above formula is satisfied between the structure point A1;

[0081] When the generated curve is an open curve, the structure point A1 has only backward control points, and the structure point A k There is only a forward control point, and the formula is as follows:

[0082]

[0083] Given n+1 control points (P1, P2, ...P n ), the parametric equation of the curve defined by these control points is as follows:

[0084]

[0085] Among them, P(t) is the coordinate of the point with position coefficient t on the curve, B i，n (t) is the Bernstein basis function:

[0086]

[0087] Finally, a series of discrete points are interpolated between two structural points through the discrete position coefficient t. The corresponding unit pixel areas are connected to form a curved defect. After morphological expansion of each unit pixel area, different curved defects are obtained.

[0088] For geometric defects, the defect shape is determined based on the linear skeleton. Each structural point is used as the center point of a geometric figure. A randomly selected geometric figure is constructed from each structural point to form a point-block feature. The geometric regions of the point-block features generated by two adjacent structural points must intersect or be tangent to each other. Ultimately, the geometric defect is formed from the point-block features corresponding to each structural point. In specific implementations, the geometric figures are shapes such as rectangles and ellipses. To ensure regional continuity, the parameters of the linear skeleton are related to the parameters of the generated geometric figure.

[0089] The geometric parameters of the rectangle are as follows:

[0090]

[0091] Among them, h and w are the width and height of the rectangle, P1 and P2 are the upper left and lower right corner points of the rectangle, that is, the first and second structural points, respectively. base is the basic length, δ is the fluctuation range of the first length;

[0092] The geometric parameters of the ellipse are as follows:

[0093]

[0094] θ i =random[α+θ s ,α+θ e ]

[0095] Among them, α is the rotation angle of the ellipse, θ s is the starting angle of the elliptical area, θ e is the end angle of the ellipse area, β is the minimum angle of the ellipse area, r1 and r2 are the two radii of the ellipse, lbase is the basic length of the geometric defect, σ is the fluctuation range of the second length, θ i is the direction angle. First, the coordinate system is rotated by angle α to transform into an elliptical coordinate system, and then the elliptical area is drawn on this coordinate system according to the starting angle and the ending angle.

[0096] In order to ensure that the area formed by the geometric figures is connected in the process of skeleton generation, when the ellipse shape is selected, the direction angle θ i The setting makes the next structure point fall within the elliptical area.

[0097] For irregular regional defects, the morphology is determined based on the regional skeleton. The curve parameter equation is used to generate smooth curve control points corresponding to the sequentially arranged structural points. This is followed by the generation of all structural points on the curve. Finally, a closed curve is drawn to form a closed region, using the curve generation method used for curve-type defects. After filling the closed region, the irregular regional defect is obtained. The number and relative number of structural points determine the smoothness and degree of variation of the contour of the region.

[0098] Step 4: Simulate the changes in background color information caused by different lighting conditions and different FPC shooting areas to enhance the robustness of the trained deep learning network. Perform image enhancement on the real defect-free template image and the virtual defect-free template image to obtain a set of defect-free template images of the circuit board;

[0099] In step 4, the image enhancement of the true defect-free template image is performed by color enhancement of the true defect-free template image in two color spaces to obtain multiple corresponding enhanced template images. Specifically, the true defect-free template image is processed by inversion, noise, hue saturation, contrast, brightness, and lightness adjustment. In a specific implementation, the original defect-free template image and the image-enhanced template image together constitute a set of circuit board defect-free template images.

[0100] Specifically, offline color enhancement includes RGB color space enhancement and HSV color space enhancement. The RGB color space enhancement method is as follows:

[0101]

[0102] Among them, rand represents a function that randomly selects one of the given numbers, and α is the upper and lower limits of a high and a low The adjustment coefficient, p is the selection coefficient that characterizes whether the color adjustment is a single channel, p i is the single-channel color adjustment probability, c is the single-channel adjustment number, β is the color adjustment matrix, (r, g, b) is the new RGB three-channel value of the pixel point, and the color of the image can be adjusted by the above formula. T represents transpose.

[0103] Gaussian noise follows the characteristics of Gaussian distribution. The method of adding Gaussian noise is as follows:

[0104]

[0105] Where δ is the value of Gaussian noise, which is generated according to a Gaussian distribution with 0 as the mathematical expectation and σ as the standard deviation. By changing the value of σ, the degree of influence of Gaussian noise on the image can be adjusted.

[0106] The HSV color space enhancement method includes brightness contrast adjustment and hue saturation adjustment. The specific adjustment methods are as follows:

[0107]

[0108] Among them, δ is the brightness coefficient randomly selected for each pixel, δ0 is the brightness adjustment limit, α is the linear contrast coefficient selected only once for each image, α0 is the contrast adjustment limit, V1 and V2 are both adjusted brightness. The adjustment of brightness and contrast are not performed at the same time. The order of adjustment has a certain impact on the contrast and brightness of the image. Adjusting the brightness first may lose some information of the highlight area or the extremely dark area.

[0109] Adjusting the hue and saturation is adjusting the H and S channels, which are adjustments to the entire image. Each image only generates one adjustment parameter. The adjustment method is as follows:

[0110]

[0111] Where δ is the chroma coefficient, δ0 is the chroma adjustment limit, α is the saturation contrast coefficient, α0 is the saturation contrast adjustment limit, β is the saturation multiplier coefficient, β0 is the saturation multiplier adjustment limit, and H and S are the adjusted chroma and saturation. Adjusting the chroma value significantly changes the hue of the image, while saturation affects the vividness of the image color.

[0112] Step 5: After fusing the defect-free template image set of the circuit board and the surface defect images in the defect feature set, a target detection image set is obtained;

[0113] In step 5, for each circuit board defect-free template image in the circuit board defect-free template image set, the corresponding target detection image specifically performs the following steps:

[0114] S1: Determine a fixed image size as the image size of the data set, use this size to perform random image segmentation on the defect-free template image of the circuit board, and obtain multiple segmented template images; the defect-free template image of the circuit board is as follows: Figure 6 shown.

[0115] S2: Determine the corresponding defect area to be placed in each segmented image, perform expansion and separation operations on each defect area to be placed, and obtain separated expanded defect areas; specifically, select the defect-free template image area affected by the defect as a mask based on the range of the defect after expansion, and then use the mask to separate the image information of the expanded defect area from the segmented image, blur the texture of the separated expanded defect area through Gaussian filtering, and then fuse it with the corresponding segmented image through weighted superposition to obtain blurred and color-differentiated defect-free images of the local area around the defect; in the specific implementation, the number of defects in each segmented image is set to N, and the defects are virtually generated. The number N of virtual defects is selected according to the selected data set image size and defect generation parameters.

[0116] S3: Blurring the edges of each surface defect image in the defect feature set by a Gaussian filter method to obtain a surface defect image with blurred edges; in this embodiment, the kernel of the Gaussian filter is 5x5.

[0117] S4: After fusing the defect-free image with blurred color in the local area around the defect and at least one type of surface defect image with blurred edges through weighted superposition, the corresponding target detection image is obtained, such as Figure 7 As shown in (a)-(d).

[0118] In this embodiment, 10 virtual defects are prepared and fused on each defect-free template image. A dilation operation is performed on each defect area, 20% of which is adjusted to a bright defect and 80% is adjusted to a dark defect. Correspondingly, the kernels of the dilation operation are 9x9 and 5x5 respectively, and color enhancement adjustment is also performed. The process of forming the defect image in steps 4 and 5 is as follows: Figure 5 shown.

[0119] Step 6: Obtain information about various defects in the target detection image by finding the minimum enclosing contour method and form training data set information. The target detection data set is formed by the target detection image set, data set information and corresponding surface defect images. The target detection data set is input into the deep convolutional neural network in the computer for training to obtain a defect detection model. The defect detection model is used to perform defect detection on the corresponding model of flexible circuit board as the template image.

[0120] In this embodiment, the information of virtual defect images is used to construct datasets in the COCO dataset format and the VOC dataset format, which can be used in mainstream one-stage and two-stage defect detection networks. The number of each defect sample is about 4500. By randomly selecting images from the dataset, the original dataset is divided into a training set, a validation set, and a test set, and the ratio of their image numbers is 7:2:1. In this embodiment, a two-stage deep convolutional neural network combining Faster R-CNN and feature pyramid network FPN is preferably used for defect detection. The detection effect is as follows: Figure 8 shown.

[0121] For any given flexible circuit board that needs to be defect-detected, the above steps can be used to obtain a deep learning neural network that can automatically detect defects generated by this model of flexible circuit board. This can avoid the problems caused by the high production yield of flexible circuit boards, small amounts of defect data, slow accumulation, and poor generalization ability of deep learning networks, thereby effectively improving the detection efficiency and accuracy of flexible circuit boards.

[0122] The above embodiments should not limit the present invention in any way, and any technical solutions obtained by equivalent replacement or equivalent conversion fall within the protection scope of the present invention.

Claims

1. A flexible circuit board defect detection method based on skeleton generation and fusion configuration, characterized in that: The following steps are involved: Step 1: Collect a real defect-free template image of the flexible circuit board; Step 2: Generate a virtual defect-free template image of the flexible circuit board according to the production file of the flexible circuit board; Step 3: Construct multiple types of surface defect images of the flexible circuit board and obtain the defect feature set; The step three is specifically as follows: First, different skeleton generation methods are used to generate linear skeletons and regional skeletons respectively. Linear defects, curved defects, and geometric defects are constructed based on the linear skeleton, and irregular regional defects are constructed based on the regional skeleton. Then, morphological processing is performed on linear defects, curved defects, geometric defects, and irregular regional defects respectively to obtain various defect features and use them to synthesize defect images, thereby forming a defect feature set for flexible circuit boards. Step 4: Perform image enhancement on the real defect-free template image and the virtual defect-free template image respectively, thereby obtaining a set of defect-free template images of the circuit board; Step 5: After fusing the defect-free template image set of the circuit board and the surface defect images in the defect feature set, a target detection image set is obtained; In step 5, for each circuit board defect-free template image in the circuit board defect-free template image set, the corresponding target detection image specifically performs the following steps: S1: Perform image segmentation on the defect-free template image of the circuit board to obtain multiple segmented template images; S2: determining the corresponding defect regions to be placed in each segmented image, performing expansion and separation operations on each defect region to be placed, and obtaining separated expanded defect regions; The texture of the separated expansion defect area is blurred by Gaussian filtering and then fused with the corresponding segmented image by weighted superposition to obtain a defect-free image with blurred color difference in the local area around the defect; S3: blurring the edges of each surface defect image in the defect feature set by using a Gaussian filtering method to obtain a blurred edge surface defect image; S4: fusing the defect-free image with blurred color in the local area around the defect and the surface defect image of at least one type of blurred edge by weighted superposition to obtain the corresponding target detection image; Step 6: Obtain information about various defects in the target detection image by finding the minimum enclosing contour method and form a training data set information. The target detection data set is formed by the target detection image set, data set information and corresponding surface defect images. The target detection data set is input into the deep convolutional neural network for training to obtain a defect detection model, and the defect detection model is used to perform defect detection on the flexible circuit board.

2. A method for detecting defects in a flexible circuit board based on skeleton generation and fusion configuration according to claim 1, characterized in that: The step 2 is specifically as follows: First, the production file of the flexible circuit board is parsed and converted to obtain the part type information of the flexible circuit board; then, the part type information of the flexible circuit board is matched and parsed to obtain the circuit board pattern matching data; then, the circuit board pattern matching data is rendered and pattern synthesized to obtain a virtual defect-free template image of the flexible circuit board.

3. The method for detecting defects in a flexible circuit board based on skeleton generation and fusion configuration according to claim 1, characterized in that: The step three is specifically as follows: Firstly, different skeleton generation methods are used to generate linear skeletons and regional skeletons respectively. Linear defects, curved defects and geometric defects are constructed based on the linear skeleton, and irregular regional defects are constructed based on the regional skeleton. Then, morphological processing is performed on linear defects, curved defects, geometric defects and irregular regional defects respectively to obtain various defect features and use them to synthesize defect images, thereby forming a defect feature set for flexible circuit boards.

4. The method for detecting defects in a flexible circuit board based on skeleton generation and fusion configuration according to claim 3, characterized in that: In step 3, for linear defects, first, randomly generate the direction angle θ and determine the basic length l base , and then determine the starting point of the linear defect (x s ,y s ) and the end point (x e ,y e ), the formula is as follows: Among them, random[] represents the function of generating random numbers, normal() represents the normal distribution function, and x s ,y s Indicates the x,y axis coordinate value of the starting point of the linear defect, x e ,y e Indicates the x and y coordinate values ​​of the end point of the linear defect; l is the length of the linear defect; According to the starting point of the linear defect (x s ,y s ) and the end point (x e ,y e ) Determine the coordinates of the remaining n-2 structural points using the following formula: Where i = 1, 2, ..., n, n is the total number of structure points, sort(·) represents the sorting function, t1, t2, ..., t n-2 are the 1st, 2nd, ..., n-2th structural points respectively, p i is the i-th structural point t i The position coefficient, x i ,y i Represents the i-th structural point t i The x,y axis coordinate values.

5. The method for detecting defects in a flexible circuit board based on skeleton generation and fusion configuration according to claim 3, characterized in that: For curve defects, the curve shape is determined according to the linear skeleton, and the smooth curve control points corresponding to the structure points arranged in sequence are generated through the curve parameter equation, and then all the structure points on the curve are generated. Finally, a closed or open curve is drawn. The structure point A i The generation formula of the forward and backward control points is as follows: Where i=2,...,n-2, α is the proportional coefficient, ‖‖2 represents the second norm, A i-1 Indicates structural point A i The forward structure point, A i+1 Indicates structural point A i The backward structure point, P i,1 is the structural point P i The forward control point, P i,2 is the structural point P i The backward control point; For the nth structural point A n , when the generated curve is a closed curve, the structural point A n is the forward structure point of structure point A1, structure point A1 is the forward structure point of structure point A n The backward structure point, structure point A n The above formula is satisfied between the structure point A1; When the generated curve is an open curve, the structure point A1 has only backward control points, and the structure point A n There is only a forward control point, and the formula is as follows: Finally, a series of discrete points are interpolated between two structural points using the discrete position coefficient t. The discrete points are connected to obtain the corresponding unit pixel areas, and the curve-shaped defects are formed by each unit pixel area.

6. The method for detecting defects in a flexible circuit board based on skeleton generation and fusion configuration according to claim 3, characterized in that: For geometric defects, the defect shape is determined according to the linear skeleton. Each structural point is used as the center point of the geometric figure. Each structural point randomly selects a geometric figure to construct a point block feature. The geometric areas of the point block features generated by two adjacent structural points must intersect or be tangent. Finally, the point block features corresponding to each structural point constitute a geometric defect.

7. The method for detecting defects in a flexible printed circuit board based on skeleton generation and fusion configuration according to claim 3, characterized in that: For irregular regional defects, the shape is determined according to the regional skeleton, and the smooth curve control points corresponding to the structural points arranged in sequence are generated through the curve parameter equation, and then all the structural points on the curve are generated. Finally, a closed curve is drawn to form a closed area. After filling the closed area, the irregular regional defect is obtained.

8. The method for detecting defects in a flexible circuit board based on skeleton generation and fusion configuration according to claim 1, characterized in that: In the step 4, for the image enhancement of the real defect-free template image, the color of the real defect-free template image is enhanced in two color spaces respectively to obtain corresponding template images after multiple image enhancements.

Images