FPC connector appearance defect detection system and method based on image generation

Abstract

This invention relates to the field of image recognition technology, specifically to an image-based FPC connector appearance defect detection system and method. The system includes a constraint definition module, a warping field generation module, a substrate reconstruction module, a poor solder joint void mesh construction module, and a defect synthesis module. In this invention, by extracting the gray-level abrupt change boundary of a defect-free image and constructing mechanically fixed end constraint lines, a warping offset field of the flexible substrate is established using a nonlinear mapping function. Coordinate migration and bilinear interpolation techniques are used to reconstruct the physical deformation characteristics of the substrate, solving the problem of difficult acquisition of actual warping samples. A mapping mesh is constructed based on the radial sector division of the solder pad area. A random walk algorithm is used to simulate dark pixel clusters inside the metal solder pads caused by poor soldering. This allows for the synthesis of highly realistic and diverse composite defect images without the need for real defect samples, filling the gap in specific morphological defect data in the detection sample library.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to an image-based FPC connector appearance defect detection system and method. Background Technology

[0002] Image recognition technology refers to a set of technologies that acquire image information of target objects and perform feature analysis and discrimination to complete target differentiation and status judgment. It includes the selection of image acquisition methods, image data preprocessing, target area localization, appearance feature extraction, and feature-based recognition and judgment process. This technology is widely used in industrial inspection, automated production monitoring, and product quality analysis.

[0003] Among them, the FPC connector appearance defect detection system refers to the detection system established to detect whether there are problems such as damage, deformation, gaps, or contamination on the surface of flexible circuit board connectors. This type of system usually takes pictures of the connector with a fixed light source and camera, obtains image data of the front or side of the connector, analyzes the images, and makes a judgment on the connector terminal arrangement area, welding area, and substrate surface item by item according to the pre-set appearance standard pattern or defect sample, thereby completing the process of identifying appearance defects.

[0004] Existing technologies rely on fixed light sources and cameras to capture images of physical objects and compare them with preset samples for identification. In actual production, the defect morphologies of flexible circuit board connectors vary greatly and samples are extremely scarce. It is difficult to cover all possible warping amplitudes of flexible substrates or the shapes of voids in metal pads through physical acquisition. The limited preset sample library cannot exhaust non-standardized defect features, causing the comparison logic to fail when faced with complex and minute deformations or randomly distributed internal voids that are not included in the data. Over-reliance on the accumulation of physical samples makes appearance inspection difficult to adapt to the changing defect features. Due to the lack of complete abnormal data support, missed detections occur frequently, which cannot meet the quality control requirements of high-precision automated production. Summary of the Invention

[0005] To address the technical problems existing in the prior art, embodiments of the present invention provide an image-generated FPC connector appearance defect detection system and method. The technical solution is as follows:

[0006] On the one hand, an image-based FPC connector appearance defect detection system is provided, which includes:

[0007] The constraint definition module identifies grayscale abrupt change boundaries in the standard reference image of a defect-free FPC connector, extracts the coordinates of the outermost edge pixels on the grayscale abrupt change boundaries, and constructs the constraint line of the mechanical fixed end.

[0008] The warping field generation module identifies the edge contour of the FPC flexible substrate in the standard reference image and determines the warping offset field of the FPC flexible substrate in combination with the mechanical fixing end constraint line.

[0009] The substrate reconstruction module identifies the FPC flexible substrate area in the standard reference image, queries the position index of the pixel in the FPC flexible substrate area and the corresponding displacement value in the FPC flexible substrate warping offset field, and reconstructs the warping shape image of the FPC flexible substrate.

[0010] The module for constructing a mesh for solder voids segments the image of the warped shape of the FPC flexible substrate, extracts the target metal pad area, and analyzes the potential solder void mapping mesh of the target metal pad area.

[0011] The defect synthesis module generates dark pixel clusters that match the number of target solder void pixels within the potential solder void mapping grid and writes them to the corresponding positions in the FPC flexible substrate warping morphology image, outputting a composite defect image.

[0012] As a further aspect of the present invention, the mechanically fixed end constraint line is specifically a linear reference base constructed based on the coordinate set of the outermost edge pixel points on the gray-scale abrupt boundary. The FPC flexible substrate warping offset field includes the perpendicular Euclidean distance between the sampling point position coordinates of the FPC flexible substrate edge contour and the mechanically fixed end constraint line, and the nonlinear mapping relationship of vertical displacement established based on the target value of the maximum warping amplitude of the FPC flexible substrate. The FPC flexible substrate warping morphology image includes the reconstructed target coordinates after pixel position migration, and the pixel gap gray value calculated by bilinear interpolation. The potential solder void mapping grid includes the solder pad radial origin located in the solder pad welding generation area, and the radial sector expanded with the solder pad radial origin as the pole. The composite defect image includes the FPC flexible substrate warping morphology image as the image base, and the dark pixel clusters embedded in the corresponding position of the image base.

[0013] As a further aspect of the present invention, the constraint definition module includes:

[0014] The amplitude distribution calculation submodule obtains a standard reference image of a defect-free FPC connector, traverses each pixel in the standard reference image, performs gray-level difference operations in the horizontal and vertical directions on each pixel, obtains the rate of change in the horizontal and vertical directions, calculates the arithmetic square root of the sum of squares of the rates of change in the horizontal and vertical directions, and generates gradient amplitude distribution data.

[0015] The boundary point set filtering submodule traverses the gradient amplitude distribution data, compares it with the preset gray-scale change judgment threshold, filters the pixel position indexes where the gradient amplitude is greater than the gray-scale change judgment threshold, extracts the horizontal and vertical coordinates of the pixel position in the entire standard reference image, and forms the gray-scale change boundary pixel coordinate set at the boundary between the rigid body of the connector and the flexible substrate of the FPC.

[0016] The constraint line construction submodule filters the pixel coordinates corresponding to the outermost extreme position of the horizontal coordinate in the set of pixel coordinates of the gray-scale change boundary, fits and connects the pixel coordinates according to the linear rule, defines the physical connection boundary between the FPC flexible substrate and the rigid body of the connector, and constructs the mechanical fixed end constraint line.

[0017] As a further aspect of the present invention, the warping field generation module includes:

[0018] The contour sampling and positioning submodule identifies the edge contour of the FPC flexible substrate in the standard reference image, performs discretization processing on the edge contour curve of the FPC flexible substrate according to the preset contour sampling interval, collects the row index and column index of the discrete points in the coordinate system of the standard reference image, and generates the coordinates of the sampling points of the FPC flexible substrate edge contour.

[0019] The perpendicular distance calculation submodule calls the mechanical fixed end constraint line as the reference geometric line, calculates the perpendicular length from the coordinates of the sampling point of the FPC flexible substrate edge contour to the reference geometric line, counts the perpendicular lengths corresponding to all sampling points, reflects the distribution of the physical distance between the position of the FPC flexible substrate and the rigid body of the connector, and obtains the set of perpendicular Euclidean distances.

[0020] The offset mapping construction submodule obtains the preset target value of the maximum warping amplitude of the FPC flexible substrate, constructs a nonlinear mapping function between the vertical length in the set of vertical Euclidean distances and the target value of the maximum warping amplitude of the FPC flexible substrate, calculates the vertical deformation displacement corresponding to the position of each pixel point based on the nonlinear mapping function, and generates the warping offset field of the FPC flexible substrate.

[0021] As a further aspect of the present invention, the substrate reconstruction module includes:

[0022] The coordinate migration calculation submodule identifies the FPC flexible substrate region in the standard reference image, traverses the pixels within the FPC flexible substrate region and reads the displacement values ​​of the pixels in the FPC flexible substrate warping offset field, determines the new position of the pixels after deformation, and obtains the reconstructed target coordinates.

[0023] The gap interpolation filling submodule performs spatial position mapping on the pixels of the standard reference image according to the reconstructed target coordinates, detects the non-integer pixel coordinate positions and pixel gaps caused by the mapping transformation, selects the gray values ​​of four pixels in the neighborhood of the pixel gap and performs a weighted average operation to obtain pixel gap interpolation gray data.

[0024] The morphological image synthesis submodule migrates the pixels of the original FPC flexible substrate area to the position specified by the reconstruction target coordinates, fills the blank areas generated during the mapping process with interpolated grayscale data of the pixel gaps, and generates an FPC flexible substrate warping morphological image.

[0025] As a further aspect of the present invention, the void mesh construction module includes:

[0026] The pad region extraction submodule iterates through the grayscale value of each pixel in the FPC flexible substrate warping shape image, filters out pixels with grayscale values ​​greater than the preset local grayscale segmentation threshold as candidate points for metal pad regions, detects the edge connectivity of the candidate points for metal pad regions and constructs closed regions to obtain the pad welding generation area mask.

[0027] The radial sector partitioning submodule extracts the row and column indices of all pixels within the mask coverage area of ​​the pad welding generation area, calculates the average value of all index values ​​as the region center point, uses the region center point as the pole, and performs radial ray segmentation on the region according to the sector partitioning angle interval and grid radius step size to generate pad radial sector partitioning data.

[0028] The mapping mesh generation submodule discretizes the target metal pad region into multiple spatial mesh blocks based on the sector geometry of the radial sector division data of the pad, assigns a unique identifier to each spatial mesh block, and constructs a mapping mesh for potential void solder joints.

[0029] As a further aspect of the present invention, the defect synthesis module includes:

[0030] The defect quantity quantification submodule counts the total number of pixels in the mask of the solder pad generation area, reads the preset percentage value of the severity of the poor solder joint, calculates the product of the total number of pixels and the percentage value of the severity of the poor solder joint, and performs a floor function on the product result to obtain the total number of defective pixels, which is used as the target number of pixels with poor solder joint voids.

[0031] The dark cluster simulation submodule initializes and generates random paths within the spatial range defined by the potential solder hole mapping grid, and generates continuous preset dark gray value pixels according to the number limit of the target solder hole pixels, simulating the void morphology inside the metal pad caused by poor welding, and generating dark pixel cluster data.

[0032] The composite image output submodule traverses the position and grayscale value of each pixel in the dark pixel cluster data, writes it to the corresponding coordinates of the FPC flexible substrate warping morphology image, and fuses the warping features of the background FPC flexible substrate with the poor soldering features of the metal pads in the foreground to output a composite defect image.

[0033] On the other hand, the image-based FPC connector appearance defect detection method, which is executed based on the aforementioned image-based FPC connector appearance defect detection system, includes the following steps:

[0034] S1: Identify the gray-scale abrupt change boundary of the standard reference image of the defect-free FPC connector, extract the coordinates of the outermost edge pixel on the gray-scale abrupt change boundary, and construct the mechanical fixed end constraint line;

[0035] S2: Identify the edge contour of the FPC flexible substrate in the standard reference image, and determine the warping offset field of the FPC flexible substrate in combination with the mechanical fixing end constraint line;

[0036] S3: Identify the FPC flexible substrate region in the standard reference image, query the position index of the pixel in the FPC flexible substrate region and the corresponding displacement value in the FPC flexible substrate warping offset field, and reconstruct the FPC flexible substrate warping shape image.

[0037] S4: Segment the image of the warping shape of the FPC flexible substrate, extract the target metal pad area, and analyze the potential solder void mapping mesh of the target metal pad area;

[0038] S5: Generate dark pixel clusters matching the number of target solder void pixels within the potential solder void mapping grid and write them to the corresponding positions in the FPC flexible substrate warping morphology image, and output the composite defect image.

[0039] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0040] By extracting the gray-level abrupt change boundary of defect-free images and constructing mechanical fixed end constraint lines, and combining nonlinear mapping functions to establish the warping offset field of flexible substrates, the physical deformation characteristics of the substrate are reconstructed using coordinate migration and bilinear interpolation techniques. This solves the problem of difficult collection of physical warping samples. A mapping grid is constructed based on the radial sector division of the pad area. A random walk algorithm is used to simulate dark pixel clusters inside the metal pads caused by poor welding. This synthesizes highly realistic and diverse composite defect images without the need for real defect samples, filling the gap in the detection sample library for specific morphological defect data and significantly improving the recognition accuracy and robustness of appearance detection algorithms for complex defects. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of an image-based FPC connector appearance defect detection system provided in an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the system framework of the present invention;

[0044] Figure 3 This is a flowchart of the constraint definition module in this invention;

[0045] Figure 4 This is a flowchart of the warping field generation module in this invention;

[0046] Figure 5 This is a flowchart of the substrate reconstruction module in this invention;

[0047] Figure 6 This is a flowchart of the virtual weld void mesh construction module in this invention;

[0048] Figure 7 This is a flowchart of the defect synthesis module in this invention;

[0049] Figure 8 This is a flowchart of the image-based FPC connector appearance defect detection method provided in the embodiments of the present invention. Detailed Implementation

[0050] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0051] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0052] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0053] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0054] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0055] This invention provides an image-based FPC connector appearance defect detection system, such as... Figure 1 The diagram shown illustrates an image-based FPC connector appearance defect detection system. The system includes:

[0056] The constraint definition module identifies grayscale abrupt change boundaries in the standard reference image of a defect-free FPC connector, extracts the coordinates of the outermost edge pixels on the grayscale abrupt change boundaries, and constructs the constraint line of the mechanical fixed end.

[0057] The warping field generation module identifies the edge contour of the FPC flexible substrate in the standard reference image and determines the warping offset field of the FPC flexible substrate by combining the mechanical fixed end constraint line.

[0058] The substrate reconstruction module identifies the FPC flexible substrate area in the standard reference image, queries the position index of the pixel in the FPC flexible substrate area and the corresponding displacement value in the FPC flexible substrate warping offset field, and reconstructs the FPC flexible substrate warping shape image.

[0059] The module for constructing meshes for solder voids segments the image of the warping morphology of the FPC flexible substrate, extracts the target metal pad region, and analyzes the potential solder void mapping mesh of the target metal pad region.

[0060] The defect synthesis module generates dark pixel clusters that match the number of pixels of the target weld void within the potential weld void mapping grid and writes them to the corresponding positions in the FPC flexible substrate warping morphology image, outputting a composite defect image.

[0061] The mechanical fixed end constraint line is specifically a linear reference datum constructed based on the coordinate set of the outermost edge pixel points on the gray-level abrupt boundary. The FPC flexible substrate warping offset field includes the perpendicular Euclidean distance between the sampling point position coordinates of the FPC flexible substrate edge contour and the mechanical fixed end constraint line, and the nonlinear mapping relationship of vertical displacement established based on the target value of the maximum warping amplitude of the FPC flexible substrate. The FPC flexible substrate warping morphology image includes the reconstructed target coordinates after pixel position migration, and the pixel gap gray value calculated by bilinear interpolation. The potential virtual solder void mapping mesh includes the radial origin of the solder pad located in the solder pad generation area, and the radial sector expanded with the radial origin of the solder pad as the pole. The composite defect image includes the FPC flexible substrate warping morphology image as the image base, and the dark pixel clusters embedded in the corresponding position of the image base.

[0062] Please see Figure 2 and Figure 3 The constraint definition module includes:

[0063] The amplitude distribution calculation submodule obtains a standard reference image of a defect-free FPC connector, traverses each pixel in the standard reference image, performs gray-level difference operations in the horizontal and vertical directions on each pixel, obtains the rate of change in the horizontal and vertical directions, calculates the arithmetic square root of the sum of squares of the rates of change in the horizontal and vertical directions, and generates gradient amplitude distribution data.

[0064] For a single-channel grayscale standard reference image of a defect-free FPC connector with a resolution of 2592 x 1944 pixels, each valid pixel within the range of row 1 to row 1943 and column 1 to column 2591 is read sequentially from left to right and top to bottom using a raster scan. The coordinates of the currently traversed pixel are set as the center point, and the grayscale value of the center point, the grayscale value of the pixel adjacent to the right of the center point, and the grayscale value of the pixel adjacent directly below the center point are read. Based on the principle of grayscale change, a difference operation is performed on the read grayscale values. For example, if the grayscale value of the center point is 120, the grayscale value of the pixel adjacent to the right is 128, and the grayscale value of the pixel adjacent directly below is 126, then the horizontal grayscale difference is... The vertical grayscale difference value is Subsequently, the obtained difference values ​​are squared to calculate the square of the rate of change in the horizontal direction. The square of the rate of change in the vertical direction is Adding the squares of the results in both directions yields the gradient energy sum, i.e. Finally, the gradient energy is calculated by performing an arithmetic square root operation. The value 10 represents the gradient magnitude of the current pixel. This calculation result is written to the corresponding coordinate position of a two-dimensional matrix with the same size as the original image, until all pixels have been calculated, forming complete gradient magnitude distribution data.

[0065] The boundary point set filtering submodule traverses the gradient magnitude distribution data, compares it with the preset gray-scale change judgment threshold, filters the pixel position index of the gradient magnitude greater than the gray-scale change judgment threshold, extracts the horizontal and vertical coordinates of the pixel position in the entire standard reference image, and forms the gray-scale change boundary pixel coordinate set at the boundary between the rigid body of the connector and the flexible substrate of the FPC.

[0066] The preset grayscale abrupt change threshold is read. This threshold is set based on statistical analysis of the background region gradients of a batch of 500 defect-free FPC connector images. The arithmetic mean of the background region pixel gradient magnitudes is 5, and the standard deviation is 2. Based on the rule of three standard deviations in statistics, the upper limit of background noise is determined and calculated as follows: To ensure the extracted edges are salient and to exclude texture interference, the grayscale abrupt change threshold is set to four times the upper limit of the background noise, i.e., the calculation... This serves as the final threshold. Iterate through each value in the gradient magnitude distribution data, comparing the gradient magnitude of each pixel with 44. If the gradient magnitude of the current pixel is greater than 44, for example, if the gradient magnitude of a certain point is 85, and... If the gradient magnitude of the current pixel is less than or equal to 44 (for example, a gradient magnitude of 30 at a certain point), then the point is determined to be at a gray-level abrupt change boundary. The row index of the point in the matrix is ​​immediately extracted as the vertical coordinate, and the column index is extracted as the horizontal coordinate. If the condition is not met, the point is determined to be a non-boundary region and is not recorded. The coordinates of all pixels that meet the determination conditions are stored in a dynamic array as coordinate pairs, forming a set of grayscale transition boundary pixel coordinates at the boundary between the rigid body of the connector and the flexible FPC substrate.

[0067] The constraint line construction submodule filters the pixel coordinates corresponding to the outermost extreme position of the horizontal coordinate in the pixel coordinate set of gray-scale abrupt boundary, fits and connects the pixel coordinates according to the linear rule, defines the physical connection boundary between the FPC flexible substrate and the connector rigid body, and constructs the mechanical fixed end constraint line.

[0068] The coordinate data of all pixels in the gray-level abrupt change boundary pixel coordinate set are traversed to extract the horizontal coordinate values ​​of all pixels. A comparison operation is used to select the set of points with the smallest horizontal coordinate value (corresponding to the left edge of the connector) or the set of points with the largest horizontal coordinate value (corresponding to the right edge of the connector) to determine the outermost extreme value position. For example, all boundary points with horizontal coordinates within the interval [200, 205] are selected. The least squares method is used to linearly fit the selected pixel coordinates to establish a linear equation model, where the vertical coordinate equals the slope multiplied by the horizontal coordinate plus the intercept. The optimal slope and intercept parameters are calculated by minimizing the sum of the squares of the distances from all selected points to the fitted line. Assuming the calculated slope is 0.05 and the intercept is 400, the determined linear rule is that the vertical coordinate equals... The straight line trajectory determined by this linear equation in the image coordinate system is defined as the physical connection boundary, i.e., the completed mechanical fixed end constraint line.

[0069] Please see Figure 2 and Figure 4 The uplift field generation module includes:

[0070] The contour sampling and positioning submodule identifies the edge contour of the FPC flexible substrate in the standard reference image, performs discretization processing on the edge contour curve of the FPC flexible substrate according to the preset contour sampling interval, collects the row index and column index of the discrete points in the coordinate system of the standard reference image, and generates the coordinates of the sampling points of the FPC flexible substrate edge contour.

[0071] The Canny edge detection operator is used to process a standard reference image to identify the continuous edge contour of the FPC flexible substrate. A preset contour sampling interval of 10 pixels is read to minimize computation while preserving the macroscopic morphological features of the contour. Equivalent discretization is performed along the path of the edge contour curve, starting from the initial endpoint. A point is acquired every 10 pixels of Euclidean distance. For example, if the total edge contour length is 2000 pixels, 200 discrete points are acquired. For each acquired discrete point, its row and column numbers in the standard reference image coordinate system are directly read. For example, the 5th sampling point has row number 150 and column number 800; the 6th sampling point has row number 160 and column number 805. These acquired row and column index values ​​are combined to generate a series of coordinate pairs for the FPC flexible substrate edge contour sampling points.

[0072] The perpendicular distance calculation submodule calls the mechanical fixed end constraint line as the reference geometric line, calculates the perpendicular length from the coordinates of the sampling point of the FPC flexible substrate edge contour to the reference geometric line, counts the perpendicular lengths corresponding to all sampling points, reflects the distribution of the physical distance between the position of the FPC flexible substrate and the rigid body of the connector, and obtains the set of perpendicular Euclidean distances.

[0073] The linear equation parameters (slope 0.05, intercept 400) of the mechanically fixed end constraint line are used as the reference geometric line. The coordinates of each sampling point in the FPC flexible substrate edge contour are traversed, and the distance formula from the point to the line is used for calculation. Let the horizontal coordinate of the current sampling point be 1000 and the vertical coordinate be 600. The numerator is calculated using absolute value operations: Calculate the arithmetic square root of the sum of squares of the coefficients in the denominator: The length of the perpendicular line is obtained by performing a division operation: This calculation process is repeated for all sampling points. If there are 200 sampling points in total, 200 corresponding vertical length values ​​are obtained. These values ​​objectively reflect the degree of physical deviation of each position of the flexible substrate relative to the rigid body, and are statistically formed into a set of vertical Euclidean distances.

[0074] The offset mapping construction submodule obtains the preset target value of the maximum warping amplitude of the FPC flexible substrate, constructs a nonlinear mapping function between the length of the vertical line in the set of vertical Euclidean distances and the target value of the maximum warping amplitude of the FPC flexible substrate, calculates the vertical deformation displacement corresponding to the position of each pixel point based on the nonlinear mapping function, and generates the warping offset field of the FPC flexible substrate.

[0075] Read the preset target value for the maximum warping amplitude of the FPC flexible substrate, which is set to 50 pixels. This value is based on the maximum allowable deformation tolerance range of the FPC substrate in actual production or the measured extreme values ​​of common defect samples. Retrieve the maximum distance value from the set of perpendicular Euclidean distances, assuming this value is 1000 pixels. Construct a quadratic polynomial nonlinear mapping function, where the vertical deformation displacement is equal to the mapping coefficient multiplied by the square of the perpendicular length. Substitute the maximum distance of 1000 and the maximum warping amplitude of 50 into the equation to solve for the mapping coefficient: The determined mapping function is that the vertical deformation displacement is equal to... The process iterates through each pixel in the image, calculates the distance from that point to the mechanically fixed constraint line, and substitutes this distance into the mapping function. For example, for a position 500 pixels from the constraint line, the vertical deformation displacement is calculated as follows: Pixel. The calculated displacement value is stored in the corresponding pixel position to generate the FPC flexible substrate warping offset field.

[0076] Please see Figure 2 and Figure 5 The substrate reconstruction module includes:

[0077] The coordinate migration calculation submodule identifies the FPC flexible substrate region in the standard reference image, traverses the pixels within the FPC flexible substrate region and reads the displacement values ​​of the pixels in the FPC flexible substrate warping offset field, determines the new position of the pixels after deformation, and obtains the reconstructed target coordinates.

[0078] The FPC flexible substrate region in the standard reference image is identified using an image segmentation mask. Each pixel within this region is traversed, and its original vertical and horizontal coordinates are read. Based on the pixel's coordinates, the corresponding vertical deformation displacement value is read from the FPC flexible substrate warping offset field. An addition operation is performed to determine the new position, adding the original vertical coordinate to the vertical deformation displacement. For example, if a pixel's original vertical coordinate is 600, its horizontal coordinate is 1000, and its read vertical deformation displacement is 12.5, then the reconstructed vertical coordinate is calculated as follows: The horizontal coordinate remains unchanged at 1000 (or a slight horizontal shrinkage is added as needed). This coordinate update calculation is performed on all pixels within the region to obtain a set of reconstructed target coordinates containing non-integer values.

[0079] The gap interpolation filling submodule performs spatial position mapping on the pixels of the standard reference image based on the reconstructed target coordinates, detects the coordinate positions of non-integer pixels and pixel gaps caused by the mapping transformation, selects the gray values ​​of four pixels in the neighborhood of the pixel gap and performs a weighted average operation to obtain pixel gap interpolation gray data.

[0080] The weighted average operation performed on the gray values ​​of four pixels within the pixel gap neighborhood includes:

[0081] Calculate the absolute value of the horizontal coordinate difference between the coordinate position of a non-integer pixel and the coordinate difference between the four pixels in the pixel gap neighborhood, and the absolute value of the vertical coordinate difference.

[0082] The interpolation weight coefficients for the four pixels in the neighborhood of each pixel gap are calculated based on the absolute values ​​of the coordinate differences in the horizontal and vertical directions.

[0083] The gray values ​​of four pixels in the neighborhood of the pixel gap are multiplied by their corresponding interpolation weight coefficients to obtain four independent weighted gray components.

[0084] Perform a summation operation on the four independent weighted grayscale components, and use the summation value as the pixel gap interpolation grayscale data;

[0085] For non-integer coordinates appearing in the reconstructed target coordinates, such as (1000, 612.5), first determine its pixel gap neighborhood within the image raster. This coordinate lies between the vertical coordinates 612 and 613. Select four integer pixels within the neighborhood: (1000, 612), (1001, 612), (1000, 613), and (1001, 613). Calculate the absolute value of the vertical difference between the non-integer coordinate and these integer points, and the difference with the point above 612 is... The difference between this point and the point below 613 is The horizontal difference is 0. Weights are calculated based on bilinear interpolation. Assume the weight of the upper point is 0.5 and the weight of the lower point is 0.5 (simplified example). The grayscale value of the upper point (1000, 612) is read as 100, and the grayscale value of the lower point (1000, 613) is read as 110. Multiplication is performed to obtain the weighted grayscale components: , Perform a summation operation on the weighted grayscale components: The calculation result 105 is the pixel gap interpolation grayscale data corresponding to this non-integer coordinate position.

[0086] The morphological image synthesis submodule migrates the pixels of the original FPC flexible substrate area to the position specified by the reconstruction target coordinates, fills the pixel gaps and interpolates grayscale data for the blank areas generated during the mapping process, and generates the FPC flexible substrate warping morphological image.

[0087] Create a blank image container with a background grayscale value of 0. Based on the mapping relationship calculated by coordinate migration, transfer the pixel grayscale values ​​of the original FPC flexible substrate area to the new coordinate positions. For blank pixel areas caused by coordinate stretching or non-integerization during the mapping process, fill them with pixel gap interpolation grayscale data calculated by the gap interpolation filling submodule. For example, write the calculated grayscale value 105 into the transition position between coordinates (1000, 612) and (1000, 613) (corresponding to the nearest integer point during actual rasterization or filled through reverse mapping). Ensure that all migrated coordinate positions and the resulting gaps are covered by effective grayscale data, complete image reconstruction, and generate an image of the FPC flexible substrate with an upward bending visual characteristic.

[0088] Please see Figure 2 and Figure 6 The module for constructing void meshes in poor solder joints includes:

[0089] The pad region extraction submodule iterates through the grayscale value of each pixel in the FPC flexible substrate warping shape image, filters out pixels with grayscale values ​​greater than the preset local grayscale segmentation threshold as candidate points for metal pad regions, detects the edge connectivity of the candidate points for metal pad regions and constructs closed regions to obtain the pad welding generation area mask.

[0090] Iterate through the grayscale values ​​of each pixel in the image of the warped shape of the FPC flexible substrate. Read the preset local grayscale segmentation threshold, which is set to 180. This threshold is based on grayscale histogram analysis of the FPC image. Due to its high reflectivity, the grayscale peaks of the metal pad area are typically distributed between 200 and 255, while the grayscale of the substrate area is distributed between 50 and 150. The valley value between these two peaks, 180, is selected as the distinction boundary. Perform a numerical comparison judgment: if the grayscale value of the current pixel is greater than 180, for example, if the grayscale value is 210 and... If the grayscale value is less than or equal to 180, for example, if the grayscale value is 120 and... If the pixel is not marked as 0, it is considered background and marked as 0. Connectivity detection is performed on the marked binary image, and adjacent pixels marked as 1 are merged into closed regions. Noise pixels with an area of ​​less than 50 pixels are removed. The final set of closed regions is the pad soldering generation area mask.

[0091] The radial sector partitioning submodule extracts the row and column indices of all pixels within the mask coverage area of ​​the pad soldering generation area, calculates the average value of all index values ​​as the region center point, and performs radial ray segmentation on the region with the region center point as the pole and according to the sector partitioning angle interval and grid radius step size to generate pad radial sector partitioning data.

[0092] Scan the pad soldering generation area mask and extract the row and column indices of all pixels within the mask area. Calculate the arithmetic mean of the row and column index sets to determine the coordinates (xc, yc) of the region's center point. Read the preset sector division angle interval (set to 10 degrees) and grid radius step size (set to 5 pixels), which are determined based on the minimum detectable size of the solder void (approximately 5-10 pixels). Using the region's center point as the polar coordinate origin, emit a virtual ray every 10 degrees starting from 0 degrees, dividing the 360-degree circle into 36 angle intervals; simultaneously, draw concentric rings with a radius of 5 pixels, centered on the region's center point. By calculating the angle and distance of each pixel relative to the center point, divide the pad area into multiple radial sectors. Record the boundary parameters of each sector to generate pad radial sector division data.

[0093] The mapping mesh generation submodule discretizes the target metal pad region into multiple spatial mesh blocks based on the sector geometry of the radial sector division data of the pad, assigns a unique identifier to each spatial mesh block, and constructs a mapping mesh for potential virtual solder voids.

[0094] Based on the angle and radius boundaries in the radial sector partitioning data of the solder pads, the target metal solder pad region is discretized into multiple independent spatial grid blocks. Each region enclosed by two adjacent rays and two adjacent concentric arcs is defined as a mapping grid. A unique identifier is assigned to each grid, such as "Layer 2 - Sector 5". It is checked whether the pixels within each grid block are primarily located within the mask area of ​​the solder pad generation region; if the grid center point is within the mask, the grid is retained. Through this process, a grid system composed of multiple tiny geometric units is established, each unit corresponding to a specific set of pixel coordinates on the image, constructing a mapping grid for potential solder voids.

[0095] Please see Figure 2 and Figure 7 The defect synthesis module includes:

[0096] The defect quantity quantification submodule counts the total number of pixels in the mask of the solder pad generation area, reads the preset percentage value of the severity of the poor solder joint, calculates the product of the total number of pixels and the percentage value of the severity of the poor solder joint, and performs a floor function on the product to obtain the total number of defective pixels, which is used as the target number of pixels with poor solder joint voids.

[0097] Count the total number of pixels marked as 1 within the mask of the solder pad generation area, assuming a count of 5000 pixels. Read the preset percentage value for solder joint severity, set to 15%, based on the IPC-A-610 electronic component acceptability standard's classification of solder joint defect area percentage; 15% represents a moderate level of solder joint failure. Perform a multiplication operation to calculate the target number of solder joint void pixels: The calculation result is rounded down to determine the final number of generated fake solder pixels to be 750. This value of 750 will serve as a constraint on the size of subsequent dark clusters.

[0098] The dark cluster simulation submodule initializes and generates random paths within the spatial range defined by the potential solder hole mapping mesh, and generates continuous preset dark gray value pixels according to the number of target solder hole pixels, simulating the void morphology inside the metal pad caused by poor soldering, and generating dark pixel cluster data.

[0099] Initializing and generating random paths within the spatial bounded by the potential weld void mapping mesh includes:

[0100] Randomly select a block coordinate from all grid blocks of the potential weld void mapping grid as the walk start point, record the walk start point into the current cluster coordinate set and mark it as occupied;

[0101] Establish neighborhood retrieval rules including four directions: up, down, left, and right. Based on the neighborhood retrieval rules, identify the adjacent grid blocks around all coordinate points in the current cluster coordinate set.

[0102] Select adjacent mesh blocks whose attributes are not marked as occupied and whose spatial location is within the boundary of the potential weld void mapped mesh as candidate growth points;

[0103] A target growth point is selected from the candidate growth points using a pseudo-random number generator. The target growth point is then entered into the current cluster coordinate set and the occupancy status flag is updated. The neighborhood search and target growth point selection operations are performed repeatedly until the total number of elements in the current cluster coordinate set is consistent with the number of pixels of the target virtual solder hole.

[0104] Within the spatial range covered by the potential weld void mapping grid, the center coordinates of a grid block are randomly selected as the starting point for the walk, for example, coordinates (500, 500). This coordinate is entered into the current cluster coordinate set and marked as occupied. A preset dark grayscale value of 50 is set, based on the fact that weld voids typically exhibit low grayscale values ​​(usually less than 60) because light cannot be reflected inside. A random walk algorithm is executed: the top, bottom, left, and right neighboring coordinates of all points in the current cluster coordinate set are retrieved. Unoccupied adjacent pixels within the grid range are selected. A target growth point is randomly selected from the selected neighboring points using a pseudo-random number generator, for example, (500, 501), and added to the set. This neighbor retrieval and selection operation is repeated, with the number of elements in the set increasing by 1 in each iteration, until the total number of elements in the set reaches the previously calculated target weld void pixel count of 750. All 750 coordinate points in the set are combined with the set grayscale value of 50 to generate dark pixel cluster data.

[0105] The composite image output submodule iterates through the position and grayscale value of each pixel in the dark pixel cluster data, writes it to the corresponding coordinates of the FPC flexible substrate warping morphology image, and merges the FPC flexible substrate warping features in the background with the metal pad poor soldering features in the foreground to output a composite defect image.

[0106] Using the image of the warped shape of the FPC flexible substrate as a base image, each data item in the dark pixel cluster data is traversed. For each pixel coordinate recorded in the data item, such as (500, 501), the corresponding position in the base image is located. The original pixel grayscale value at that position (which might be the grayscale value 210 for a bright pad) is directly replaced with the dark grayscale value 50 recorded in the data item. This grayscale replacement operation is performed on all 750 cluster pixels. This process integrates the dark features simulating solder joint voids into the image with the warped shape, so that the final image simultaneously possesses the defect features of both substrate deformation and solder joint voids, outputting the final composite defect image.

[0107] Please see Figure 8 The image-based FPC connector appearance defect detection method, which is based on the aforementioned image-based FPC connector appearance defect detection system, includes the following steps:

[0108] S1: Identify the gray-scale abrupt change boundary of the standard reference image of the defect-free FPC connector, extract the coordinates of the outermost edge pixel on the gray-scale abrupt change boundary, and construct the mechanical fixed end constraint line;

[0109] S2: Identify the edge contour of the FPC flexible substrate in the standard reference image, and determine the warping offset field of the FPC flexible substrate by combining the mechanical fixed end constraint line.

[0110] S3: Identify the FPC flexible substrate region in the standard reference image, query the position index of the pixel in the FPC flexible substrate region and the corresponding displacement value in the FPC flexible substrate warping offset field, and reconstruct the FPC flexible substrate warping shape image.

[0111] S4: Segment the image of the warping shape of the FPC flexible substrate, extract the target metal pad area, and analyze the potential poor solder joint void mapping mesh of the target metal pad area;

[0112] S5: Generate dark pixel clusters matching the number of target void pixels within the potential void mapping mesh and write them to the corresponding positions in the FPC flexible substrate warping morphology image, outputting a composite defect image.

[0113] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An image-based FPC connector appearance defect detection system, characterized in that, The system includes: The constraint definition module identifies grayscale abrupt change boundaries in the standard reference image of a defect-free FPC connector, extracts the coordinates of the outermost edge pixels on the grayscale abrupt change boundaries, and constructs the constraint line of the mechanical fixed end. The warping field generation module identifies the edge contour of the FPC flexible substrate in the standard reference image and determines the warping offset field of the FPC flexible substrate in combination with the mechanical fixed end constraint line. The substrate reconstruction module identifies the FPC flexible substrate area in the standard reference image, queries the position index of the pixel in the FPC flexible substrate area and the corresponding displacement value in the FPC flexible substrate warping offset field, and reconstructs the warping shape image of the FPC flexible substrate. The module for constructing a mesh for solder voids segments the image of the warped shape of the FPC flexible substrate, extracts the target metal pad region, and analyzes the potential solder void mapping mesh of the target metal pad region. The defect synthesis module generates dark pixel clusters matching the target number of solder void pixels within the potential solder void mapping grid and writes them to the corresponding positions in the FPC flexible substrate warping morphology image, outputting a composite defect image. The mechanical fixed end constraint line is specifically a linear reference base constructed based on the coordinate set of the outermost edge pixel points on the gray-level abrupt boundary. The FPC flexible substrate warping offset field includes the perpendicular Euclidean distance between the sampling point position coordinates of the FPC flexible substrate edge contour and the mechanical fixed end constraint line, and a nonlinear mapping relationship of vertical displacement established based on the target value of the maximum warping amplitude of the FPC flexible substrate. The FPC flexible substrate warping morphology image includes the reconstructed target coordinates after pixel position migration and the pixel gap gray value calculated by bilinear interpolation. The potential solder void mapping grid includes the radial origin of the solder pad located in the solder pad welding generation area and the radial sector expanded with the radial origin of the solder pad as the pole. The composite defect image includes the FPC flexible substrate warping morphology image as the image base and the dark pixel clusters embedded in the corresponding positions of the image base.

2. The FPC connector appearance defect detection system based on image generation according to claim 1, characterized in that: The constraint definition module includes: The amplitude distribution calculation submodule obtains a standard reference image of a defect-free FPC connector, traverses each pixel in the standard reference image, performs gray-level difference operations in the horizontal and vertical directions on each pixel, obtains the rate of change in the horizontal and vertical directions, calculates the arithmetic square root of the sum of squares of the rates of change in the horizontal and vertical directions, and generates gradient amplitude distribution data. The boundary point set filtering submodule traverses the gradient amplitude distribution data, compares it with the preset gray-scale change judgment threshold, filters the pixel position indexes where the gradient amplitude is greater than the gray-scale change judgment threshold, extracts the horizontal and vertical coordinates of the pixel position in the entire standard reference image, and forms the gray-scale change boundary pixel coordinate set at the boundary between the rigid body of the connector and the flexible substrate of the FPC. The constraint line construction submodule filters the pixel coordinates corresponding to the outermost extreme position of the horizontal coordinate in the set of pixel coordinates of the gray-scale change boundary, fits and connects the pixel coordinates according to the linear rule, defines the physical connection boundary between the FPC flexible substrate and the rigid body of the connector, and constructs the mechanical fixed end constraint line.

3. The image-based FPC connector appearance defect detection system according to claim 1, characterized in that: The warping field generation module includes: The contour sampling and positioning submodule identifies the edge contour of the FPC flexible substrate in the standard reference image, performs discretization processing on the edge contour curve of the FPC flexible substrate according to the preset contour sampling interval, collects the row index and column index of the discrete points in the coordinate system of the standard reference image, and generates the coordinates of the sampling points of the FPC flexible substrate edge contour. The perpendicular distance calculation submodule calls the mechanical fixed end constraint line as the reference geometric line, calculates the perpendicular length from the coordinates of the sampling point of the FPC flexible substrate edge contour to the reference geometric line, counts the perpendicular lengths corresponding to all sampling points, reflects the distribution of the physical distance between the position of the FPC flexible substrate and the rigid body of the connector, and obtains the set of perpendicular Euclidean distances. The offset mapping construction submodule obtains the preset target value of the maximum warping amplitude of the FPC flexible substrate, constructs a nonlinear mapping function between the vertical length in the set of vertical Euclidean distances and the target value of the maximum warping amplitude of the FPC flexible substrate, calculates the vertical deformation displacement corresponding to the position of each pixel point based on the nonlinear mapping function, and generates the warping offset field of the FPC flexible substrate.

4. The FPC connector appearance defect detection system based on image generation according to claim 1, characterized in that: The substrate reconstruction module includes: The coordinate migration calculation submodule identifies the FPC flexible substrate region in the standard reference image, traverses the pixels within the FPC flexible substrate region and reads the displacement values ​​of the pixels in the FPC flexible substrate warping offset field, determines the new position of the pixels after deformation, and obtains the reconstructed target coordinates. The gap interpolation filling submodule performs spatial position mapping on the pixels of the standard reference image according to the reconstructed target coordinates, detects the non-integer pixel coordinate positions and pixel gaps caused by the mapping transformation, selects the gray values ​​of four pixels in the neighborhood of the pixel gap and performs a weighted average operation to obtain pixel gap interpolation gray data. The morphological image synthesis submodule migrates the pixels of the original FPC flexible substrate area to the position specified by the reconstruction target coordinates, fills the blank areas generated during the mapping process with interpolated grayscale data of the pixel gaps, and generates an FPC flexible substrate warping morphological image.

5. The image-based FPC connector appearance defect detection system according to claim 1, characterized in that: The module for constructing the mesh of voids in welded connections includes: The pad region extraction submodule iterates through the grayscale value of each pixel in the FPC flexible substrate warping shape image, filters out pixels with grayscale values ​​greater than the preset local grayscale segmentation threshold as candidate points for metal pad regions, detects the edge connectivity of the candidate points for metal pad regions and constructs closed regions to obtain the pad welding generation area mask. The radial sector partitioning submodule extracts the row and column indices of all pixels within the mask coverage area of ​​the pad welding generation area, calculates the average value of all index values ​​as the region center point, uses the region center point as the pole, and performs radial ray segmentation on the region according to the sector partitioning angle interval and grid radius step size to generate pad radial sector partitioning data. The mapping mesh generation submodule discretizes the target metal pad region into multiple spatial mesh blocks based on the sector geometry of the radial sector division data of the pad, assigns a unique identifier to each spatial mesh block, and constructs a mapping mesh for potential void solder joints.

6. The FPC connector appearance defect detection system based on image generation according to claim 1, characterized in that: The defect synthesis module includes: The defect quantity quantification submodule counts the total number of pixels in the mask of the solder pad generation area, reads the preset percentage value of the severity of the poor solder joint, calculates the product of the total number of pixels and the percentage value of the severity of the poor solder joint, and performs a floor function on the product result to obtain the total number of defective pixels, which is used as the target number of pixels with poor solder joint voids. The dark cluster simulation submodule initializes and generates random paths within the spatial range defined by the potential solder hole mapping grid, and generates continuous preset dark gray value pixels according to the number limit of the target solder hole pixels, simulating the void morphology inside the metal pad caused by poor welding, and generating dark pixel cluster data. The composite image output submodule traverses the position and grayscale value of each pixel in the dark pixel cluster data, writes it to the corresponding coordinates of the FPC flexible substrate warping morphology image, and fuses the warping features of the background FPC flexible substrate with the poor soldering features of the metal pads in the foreground to output a composite defect image.

7. The image-based FPC connector appearance defect detection system according to claim 4, characterized in that: The weighted average operation performed on the gray values ​​of four pixels within the pixel gap neighborhood includes: Calculate the absolute value of the horizontal coordinate difference between the coordinate position of a non-integer pixel and the coordinate difference between the four pixels in the pixel gap neighborhood, and the absolute value of the vertical coordinate difference. The interpolation weight coefficients for the four pixels in the neighborhood of each pixel gap are calculated based on the absolute values ​​of the coordinate differences in the horizontal and vertical directions. The gray values ​​of the four pixels in the neighborhood of the pixel gap are multiplied by the corresponding interpolation weight coefficients to obtain four independent weighted gray components. Perform a summation operation on the four independent weighted grayscale components, and use the summation value as the pixel gap interpolation grayscale data.

8. The image-based FPC connector appearance defect detection system according to claim 6, characterized in that: Initializing and generating random paths within the spatial bounded by the potential weld void mapping mesh includes: Randomly select a block coordinate from all grid blocks of the potential weld void mapping grid as the walk start point, record the walk start point into the current cluster coordinate set and mark it as occupied; Establish neighborhood retrieval rules including four directions: up, down, left, and right. Based on the neighborhood retrieval rules, identify the adjacent grid blocks around all coordinate points in the current cluster coordinate set. Select adjacent mesh blocks whose attributes are not marked as occupied and whose spatial location is within the boundary of the potential weld void mapped mesh as candidate growth points; A target growth point is selected from the candidate growth points using a pseudo-random number generator. The target growth point is then entered into the current cluster coordinate set and the occupancy status flag is updated. The neighborhood search and target growth point selection operations are performed repeatedly until the total number of elements in the current cluster coordinate set matches the number of pixels of the target virtual solder hole.

9. A method for detecting appearance defects in FPC connectors based on image generation, characterized in that, The image-based FPC connector appearance defect detection system according to any one of claims 1-8 includes the following steps: S1: Identify the gray-scale abrupt change boundary of the standard reference image of the defect-free FPC connector, extract the coordinates of the outermost edge pixel on the gray-scale abrupt change boundary, and construct the mechanical fixed end constraint line; S2: Identify the edge contour of the FPC flexible substrate in the standard reference image, and determine the warping offset field of the FPC flexible substrate in combination with the mechanical fixing end constraint line; S3: Identify the FPC flexible substrate region in the standard reference image, query the position index of the pixel in the FPC flexible substrate region and the corresponding displacement value in the FPC flexible substrate warping offset field, and reconstruct the FPC flexible substrate warping shape image. S4: Segment the image of the warping shape of the FPC flexible substrate, extract the target metal pad area, and analyze the potential solder void mapping mesh of the target metal pad area; S5: Generate dark pixel clusters matching the number of target solder void pixels within the potential solder void mapping grid and write them to the corresponding positions in the FPC flexible substrate warping morphology image, and output the composite defect image.

Images